2024-04-20T01:24:56.881 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-20T01:24:56.882 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102 --description rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} --owner scheduled_yuriw@teuthology --verbose --name yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi -- /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102/orig.config.yaml 2024-04-20T01:24:56.925 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T01:24:57.006 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102 branch: wip-yuri7-testing-2024-04-18-1351-reef description: rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} email: yweinste@redhat.com first_in_suite: false job_id: '7664102' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-yuri7-testing-2024-04-18-1351-reef ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 1 ms inject socket failures: 5000 osd_object_clean_region_max_num_intervals: 1000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: stupid bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.3 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd heartbeat use min delay socket: true osd max backfills: 3 osd max markdown count: 1000 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - slow request - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: 15eecde2bab9803889d929c7e423d07970d8cbed ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 15eecde2bab9803889d929c7e423d07970d8cbed selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri7-testing-2024-04-18-1351-reef sha1: 15eecde2bab9803889d929c7e423d07970d8cbed owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - osd.4 - osd.8 - osd.12 - - mon.b - osd.1 - osd.5 - osd.9 - osd.13 - - mon.c - osd.2 - osd.6 - osd.10 - osd.14 - - mgr.x - osd.3 - osd.7 - osd.11 - osd.15 - client.0 seed: 6867 sha1: 15eecde2bab9803889d929c7e423d07970d8cbed sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri7-testing-2024-04-18-1351-reef suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_15eecde2bab9803889d929c7e423d07970d8cbed/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 15eecde2bab9803889d929c7e423d07970d8cbed targets: smithi044.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMIgcP+N0MinUz7O0/WVH1eLrCamp4Ij8HNr9X+vM9WdQmAJ8bkMbCgenNT1rvXHF0KHA6Xxv4NfoM242eSHH8w= smithi121.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOLY3BspgQuGnvmUB46pety3e+CT7KjzA4Ny5DYo9lunqK1EvskSncHv7IU5e1ykYNkoPQYRgG6GyTwKkyvQ/h8= smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEkjsevvi9EwRZSOFfgnl7OgcuAhGvgOA5spMkNiPuteKF0rrQG98huMZbkMCR1fGcHkctosA9IRsGL4E7YQA+k= smithi165.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIoo/dIkI4rR3vYVmBf83hvaLdIycYt2SJgvAkGKlLeTPh0uuczc7xa9bfxlij62cdXlNmHyaZiYn4kWMdU3GGQ= tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: aggressive_pg_num_changes: false chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 min_in: 8 timeout: 1200 - rados: clients: - client.0 ec_pool: true erasure_code_profile: c: 2 crush-failure-domain: osd k: 4 m: 3 name: shecprofile plugin: shec objects: 50 op_weights: append: 100 copy_from: 50 delete: 50 read: 100 rmattr: 25 rollback: 50 setattr: 25 snap_create: 50 snap_remove: 50 write: 0 ops: 400 write_append_excl: false teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-20_01:10:46 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-20T01:24:57.007 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_15eecde2bab9803889d929c7e423d07970d8cbed/qa; will attempt to use it 2024-04-20T01:24:57.007 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_15eecde2bab9803889d929c7e423d07970d8cbed/qa/tasks 2024-04-20T01:24:57.008 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-20T01:24:57.009 INFO:teuthology.task.internal:Checking packages... 2024-04-20T01:24:57.035 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '15eecde2bab9803889d929c7e423d07970d8cbed' 2024-04-20T01:24:57.035 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-20T01:24:57.035 INFO:teuthology.packaging:ref: None 2024-04-20T01:24:57.035 INFO:teuthology.packaging:tag: None 2024-04-20T01:24:57.035 INFO:teuthology.packaging:branch: wip-yuri7-testing-2024-04-18-1351-reef 2024-04-20T01:24:57.035 INFO:teuthology.packaging:sha1: 15eecde2bab9803889d929c7e423d07970d8cbed 2024-04-20T01:24:57.036 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri7-testing-2024-04-18-1351-reef 2024-04-20T01:24:57.251 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1252.g15eecde2 2024-04-20T01:24:57.252 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-20T01:24:57.259 INFO:teuthology.task.internal:no buildpackages task found 2024-04-20T01:24:57.259 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-20T01:24:57.266 INFO:teuthology.task.internal:Saving configuration 2024-04-20T01:24:57.283 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-20T01:24:57.289 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T01:24:57.313 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 01:13:10.753195', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMIgcP+N0MinUz7O0/WVH1eLrCamp4Ij8HNr9X+vM9WdQmAJ8bkMbCgenNT1rvXHF0KHA6Xxv4NfoM242eSHH8w='} 2024-04-20T01:24:57.335 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 01:13:10.755690', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOLY3BspgQuGnvmUB46pety3e+CT7KjzA4Ny5DYo9lunqK1EvskSncHv7IU5e1ykYNkoPQYRgG6GyTwKkyvQ/h8='} 2024-04-20T01:24:57.361 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 01:13:10.756715', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEkjsevvi9EwRZSOFfgnl7OgcuAhGvgOA5spMkNiPuteKF0rrQG98huMZbkMCR1fGcHkctosA9IRsGL4E7YQA+k='} 2024-04-20T01:24:57.386 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi165.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 01:13:10.754472', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIoo/dIkI4rR3vYVmBf83hvaLdIycYt2SJgvAkGKlLeTPh0uuczc7xa9bfxlij62cdXlNmHyaZiYn4kWMdU3GGQ='} 2024-04-20T01:24:57.386 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-20T01:24:57.393 INFO:teuthology.task.internal:roles: ubuntu@smithi044.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'] 2024-04-20T01:24:57.393 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'] 2024-04-20T01:24:57.393 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'] 2024-04-20T01:24:57.393 INFO:teuthology.task.internal:roles: ubuntu@smithi165.front.sepia.ceph.com - ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0'] 2024-04-20T01:24:57.393 INFO:teuthology.run_tasks:Running task console_log... 2024-04-20T01:24:57.529 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f654b9e0f70>, signals=[15]) 2024-04-20T01:24:57.529 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-20T01:24:57.536 INFO:teuthology.task.internal:Opening connections... 2024-04-20T01:24:57.536 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi044.front.sepia.ceph.com 2024-04-20T01:24:57.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T01:24:57.618 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi121.front.sepia.ceph.com 2024-04-20T01:24:57.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T01:24:57.692 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-04-20T01:24:57.693 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T01:24:57.769 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi165.front.sepia.ceph.com 2024-04-20T01:24:57.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T01:24:57.851 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-20T01:24:57.861 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2024-04-20T01:24:57.883 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2024-04-20T01:24:57.884 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2024-04-20T01:24:57.944 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="Red Hat Enterprise Linux" 2024-04-20T01:24:57.945 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="8.6 (Ootpa)" 2024-04-20T01:24:57.945 INFO:teuthology.orchestra.run.smithi044.stdout:ID="rhel" 2024-04-20T01:24:57.945 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="fedora" 2024-04-20T01:24:57.945 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="8.6" 2024-04-20T01:24:57.945 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el8" 2024-04-20T01:24:57.945 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-20T01:24:57.945 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2024-04-20T01:24:57.945 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-20T01:24:57.945 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://www.redhat.com/" 2024-04-20T01:24:57.945 INFO:teuthology.orchestra.run.smithi044.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-20T01:24:57.946 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T01:24:57.946 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:24:57.946 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-20T01:24:57.946 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-20T01:24:57.946 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-20T01:24:57.946 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-20T01:24:57.946 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2024-04-20T01:24:57.969 DEBUG:teuthology.orchestra.run.smithi121:> uname -m 2024-04-20T01:24:57.991 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2024-04-20T01:24:57.991 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2024-04-20T01:24:58.051 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="Red Hat Enterprise Linux" 2024-04-20T01:24:58.051 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="8.6 (Ootpa)" 2024-04-20T01:24:58.051 INFO:teuthology.orchestra.run.smithi121.stdout:ID="rhel" 2024-04-20T01:24:58.051 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="fedora" 2024-04-20T01:24:58.051 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="8.6" 2024-04-20T01:24:58.051 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el8" 2024-04-20T01:24:58.051 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-20T01:24:58.051 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2024-04-20T01:24:58.051 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-20T01:24:58.052 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://www.redhat.com/" 2024-04-20T01:24:58.052 INFO:teuthology.orchestra.run.smithi121.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-20T01:24:58.052 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T01:24:58.052 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:24:58.052 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-20T01:24:58.052 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-20T01:24:58.052 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-20T01:24:58.052 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-20T01:24:58.052 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2024-04-20T01:24:58.103 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-04-20T01:24:58.124 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-04-20T01:24:58.125 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-04-20T01:24:58.185 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="Red Hat Enterprise Linux" 2024-04-20T01:24:58.185 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="8.6 (Ootpa)" 2024-04-20T01:24:58.185 INFO:teuthology.orchestra.run.smithi123.stdout:ID="rhel" 2024-04-20T01:24:58.185 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="fedora" 2024-04-20T01:24:58.185 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="8.6" 2024-04-20T01:24:58.185 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el8" 2024-04-20T01:24:58.185 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-20T01:24:58.185 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-04-20T01:24:58.185 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-20T01:24:58.185 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://www.redhat.com/" 2024-04-20T01:24:58.185 INFO:teuthology.orchestra.run.smithi123.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-20T01:24:58.185 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T01:24:58.186 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:24:58.186 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-20T01:24:58.186 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-20T01:24:58.186 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-20T01:24:58.186 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-20T01:24:58.186 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-04-20T01:24:58.336 DEBUG:teuthology.orchestra.run.smithi165:> uname -m 2024-04-20T01:24:58.356 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64 2024-04-20T01:24:58.357 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release 2024-04-20T01:24:58.417 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="Red Hat Enterprise Linux" 2024-04-20T01:24:58.417 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="8.6 (Ootpa)" 2024-04-20T01:24:58.417 INFO:teuthology.orchestra.run.smithi165.stdout:ID="rhel" 2024-04-20T01:24:58.417 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="fedora" 2024-04-20T01:24:58.417 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="8.6" 2024-04-20T01:24:58.417 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el8" 2024-04-20T01:24:58.418 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-20T01:24:58.418 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31" 2024-04-20T01:24:58.418 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-20T01:24:58.418 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://www.redhat.com/" 2024-04-20T01:24:58.418 INFO:teuthology.orchestra.run.smithi165.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-20T01:24:58.418 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T01:24:58.418 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:24:58.418 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-20T01:24:58.418 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-20T01:24:58.418 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-20T01:24:58.418 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-20T01:24:58.419 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2024-04-20T01:24:58.444 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-20T01:24:58.454 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-20T01:24:58.460 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-20T01:24:58.461 DEBUG:teuthology.orchestra.run.smithi044:> test '!' -e /home/ubuntu/cephtest 2024-04-20T01:24:58.463 DEBUG:teuthology.orchestra.run.smithi121:> test '!' -e /home/ubuntu/cephtest 2024-04-20T01:24:58.465 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-04-20T01:24:58.468 DEBUG:teuthology.orchestra.run.smithi165:> test '!' -e /home/ubuntu/cephtest 2024-04-20T01:24:58.486 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-20T01:24:58.494 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-20T01:24:58.494 DEBUG:teuthology.orchestra.run.smithi044:> test -z $(ls -A /var/lib/ceph) 2024-04-20T01:24:58.524 DEBUG:teuthology.orchestra.run.smithi121:> test -z $(ls -A /var/lib/ceph) 2024-04-20T01:24:58.526 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-04-20T01:24:58.528 DEBUG:teuthology.orchestra.run.smithi165:> test -z $(ls -A /var/lib/ceph) 2024-04-20T01:24:58.579 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-20T01:24:58.679 INFO:teuthology.run_tasks:Running task kernel... 2024-04-20T01:24:58.693 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-20T01:24:58.693 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-20T01:24:58.694 DEBUG:teuthology.orchestra.run.smithi044:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T01:24:58.694 DEBUG:teuthology.orchestra.run.smithi121:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T01:24:58.695 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T01:24:58.695 DEBUG:teuthology.orchestra.run.smithi165:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T01:24:58.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T01:24:58.715 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-04-20T01:24:58.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T01:24:58.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T01:24:58.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T01:24:58.717 DEBUG:teuthology.orchestra.run.smithi044:> uname -r 2024-04-20T01:24:58.717 DEBUG:teuthology.orchestra.run.smithi165:> uname -r 2024-04-20T01:24:58.718 DEBUG:teuthology.orchestra.run.smithi121:> uname -r 2024-04-20T01:24:58.776 INFO:teuthology.orchestra.run.smithi123.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:24:58.777 INFO:teuthology.task.kernel:Running kernel on smithi123: 4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:24:58.777 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum install -y kernel 2024-04-20T01:24:58.779 INFO:teuthology.orchestra.run.smithi121.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:24:58.779 INFO:teuthology.orchestra.run.smithi165.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:24:58.779 INFO:teuthology.task.kernel:Running kernel on smithi121: 4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:24:58.779 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum install -y kernel 2024-04-20T01:24:58.779 INFO:teuthology.task.kernel:Running kernel on smithi165: 4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:24:58.780 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum install -y kernel 2024-04-20T01:24:58.781 INFO:teuthology.orchestra.run.smithi044.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:24:58.781 INFO:teuthology.task.kernel:Running kernel on smithi044: 4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:24:58.781 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum install -y kernel 2024-04-20T01:25:00.230 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:25:00.231 INFO:teuthology.orchestra.run.smithi123.stdout:Unable to read consumer identity 2024-04-20T01:25:00.281 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:25:00.281 INFO:teuthology.orchestra.run.smithi165.stdout:Unable to read consumer identity 2024-04-20T01:25:00.338 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:25:00.338 INFO:teuthology.orchestra.run.smithi121.stdout:Unable to read consumer identity 2024-04-20T01:25:00.368 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:25:00.368 INFO:teuthology.orchestra.run.smithi123.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-20T01:25:00.368 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:25:00.431 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:25:00.431 INFO:teuthology.orchestra.run.smithi165.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-20T01:25:00.431 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:25:00.469 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:25:00.469 INFO:teuthology.orchestra.run.smithi044.stdout:Unable to read consumer identity 2024-04-20T01:25:00.470 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:25:00.470 INFO:teuthology.orchestra.run.smithi121.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-20T01:25:00.470 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:25:00.599 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:25:00.599 INFO:teuthology.orchestra.run.smithi044.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-20T01:25:00.600 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:25:02.150 INFO:teuthology.orchestra.run.smithi165.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-04-20T01:25:02.169 INFO:teuthology.orchestra.run.smithi123.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-04-20T01:25:02.307 INFO:teuthology.orchestra.run.smithi121.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2024-04-20T01:25:02.407 INFO:teuthology.orchestra.run.smithi044.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2024-04-20T01:25:02.817 INFO:teuthology.orchestra.run.smithi165.stdout:Extra Packages for Enterprise Linux 26 MB/s | 16 MB 00:00 2024-04-20T01:25:03.793 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2024-04-20T01:25:03.869 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2024-04-20T01:25:06.409 INFO:teuthology.orchestra.run.smithi165.stdout:lab-extras 568 kB/s | 24 kB 00:00 2024-04-20T01:25:07.145 INFO:teuthology.orchestra.run.smithi165.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-04-20T01:25:07.145 INFO:teuthology.orchestra.run.smithi165.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-04-20T01:25:07.228 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:25:07.229 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-20T01:25:07.229 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:25:07.363 DEBUG:teuthology.orchestra.run.smithi165:> echo no | sudo yum reinstall kernel || true 2024-04-20T01:25:07.390 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 545 kB/s | 24 kB 00:00 2024-04-20T01:25:07.701 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:25:07.701 INFO:teuthology.orchestra.run.smithi165.stdout:Unable to read consumer identity 2024-04-20T01:25:07.772 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:25:07.772 INFO:teuthology.orchestra.run.smithi165.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-20T01:25:07.773 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:25:07.941 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 308 kB/s | 24 kB 00:00 2024-04-20T01:25:08.086 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:02 ago on Sat 20 Apr 2024 01:25:06 AM UTC. 2024-04-20T01:25:08.118 INFO:teuthology.orchestra.run.smithi121.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-04-20T01:25:08.118 INFO:teuthology.orchestra.run.smithi121.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-04-20T01:25:08.172 INFO:teuthology.orchestra.run.smithi165.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-04-20T01:25:08.173 INFO:teuthology.orchestra.run.smithi165.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-04-20T01:25:08.198 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:25:08.199 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-04-20T01:25:08.199 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:25:08.223 INFO:teuthology.orchestra.run.smithi165.stderr:Error: No packages marked for reinstall. 2024-04-20T01:25:08.271 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum reinstall -y kernel || true 2024-04-20T01:25:08.316 DEBUG:teuthology.orchestra.run.smithi121:> echo no | sudo yum reinstall kernel || true 2024-04-20T01:25:08.606 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:25:08.606 INFO:teuthology.orchestra.run.smithi165.stdout:Unable to read consumer identity 2024-04-20T01:25:08.673 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:25:08.673 INFO:teuthology.orchestra.run.smithi165.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-20T01:25:08.673 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:25:08.690 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:25:08.690 INFO:teuthology.orchestra.run.smithi121.stdout:Unable to read consumer identity 2024-04-20T01:25:08.739 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:25:08.739 INFO:teuthology.orchestra.run.smithi121.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-20T01:25:08.739 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:25:08.744 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-04-20T01:25:08.744 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-04-20T01:25:08.837 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:25:08.838 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-20T01:25:08.838 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:25:08.982 DEBUG:teuthology.orchestra.run.smithi044:> echo no | sudo yum reinstall kernel || true 2024-04-20T01:25:08.987 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:02 ago on Sat 20 Apr 2024 01:25:06 AM UTC. 2024-04-20T01:25:09.048 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:02 ago on Sat 20 Apr 2024 01:25:07 AM UTC. 2024-04-20T01:25:09.050 INFO:teuthology.orchestra.run.smithi165.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-04-20T01:25:09.050 INFO:teuthology.orchestra.run.smithi165.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-04-20T01:25:09.097 INFO:teuthology.orchestra.run.smithi165.stderr:Error: No packages marked for reinstall. 2024-04-20T01:25:09.145 INFO:teuthology.orchestra.run.smithi121.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-04-20T01:25:09.150 DEBUG:teuthology.orchestra.run.smithi165:> rpm -q kernel | sort -rV | head -n 1 2024-04-20T01:25:09.152 INFO:teuthology.orchestra.run.smithi121.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-04-20T01:25:09.198 INFO:teuthology.orchestra.run.smithi121.stderr:Error: No packages marked for reinstall. 2024-04-20T01:25:09.244 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum reinstall -y kernel || true 2024-04-20T01:25:09.311 INFO:teuthology.orchestra.run.smithi165.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:25:09.311 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:25:09.311 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-20T01:25:09.312 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-04-20T01:25:09.312 DEBUG:teuthology.orchestra.run.smithi165:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-20T01:25:09.346 INFO:teuthology.orchestra.run.smithi165.stdout:ttyS1 2024-04-20T01:25:09.361 DEBUG:teuthology.parallel:result is None 2024-04-20T01:25:09.367 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:25:09.367 INFO:teuthology.orchestra.run.smithi044.stdout:Unable to read consumer identity 2024-04-20T01:25:09.432 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:25:09.432 INFO:teuthology.orchestra.run.smithi044.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-20T01:25:09.432 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:25:09.579 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:25:09.579 INFO:teuthology.orchestra.run.smithi121.stdout:Unable to read consumer identity 2024-04-20T01:25:09.631 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:25:09.631 INFO:teuthology.orchestra.run.smithi121.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-20T01:25:09.631 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:25:09.794 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:02 ago on Sat 20 Apr 2024 01:25:07 AM UTC. 2024-04-20T01:25:09.881 INFO:teuthology.orchestra.run.smithi044.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-04-20T01:25:09.882 INFO:teuthology.orchestra.run.smithi044.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-04-20T01:25:09.923 INFO:teuthology.orchestra.run.smithi044.stderr:Error: No packages marked for reinstall. 2024-04-20T01:25:09.936 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:02 ago on Sat 20 Apr 2024 01:25:07 AM UTC. 2024-04-20T01:25:09.979 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum reinstall -y kernel || true 2024-04-20T01:25:09.998 INFO:teuthology.orchestra.run.smithi121.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-04-20T01:25:09.998 INFO:teuthology.orchestra.run.smithi121.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-04-20T01:25:10.030 INFO:teuthology.orchestra.run.smithi121.stderr:Error: No packages marked for reinstall. 2024-04-20T01:25:10.081 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q kernel | sort -rV | head -n 1 2024-04-20T01:25:10.235 INFO:teuthology.orchestra.run.smithi121.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:25:10.235 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:25:10.236 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-20T01:25:10.236 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-20T01:25:10.236 DEBUG:teuthology.orchestra.run.smithi121:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-20T01:25:10.271 INFO:teuthology.orchestra.run.smithi121.stdout:ttyS1 2024-04-20T01:25:10.284 DEBUG:teuthology.parallel:result is None 2024-04-20T01:25:10.363 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:25:10.363 INFO:teuthology.orchestra.run.smithi044.stdout:Unable to read consumer identity 2024-04-20T01:25:10.434 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:25:10.434 INFO:teuthology.orchestra.run.smithi044.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-20T01:25:10.434 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:25:10.789 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:03 ago on Sat 20 Apr 2024 01:25:07 AM UTC. 2024-04-20T01:25:10.864 INFO:teuthology.orchestra.run.smithi044.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-04-20T01:25:10.864 INFO:teuthology.orchestra.run.smithi044.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-04-20T01:25:10.915 INFO:teuthology.orchestra.run.smithi044.stderr:Error: No packages marked for reinstall. 2024-04-20T01:25:10.970 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q kernel | sort -rV | head -n 1 2024-04-20T01:25:11.135 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:25:11.136 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:25:11.136 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-20T01:25:11.136 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-20T01:25:11.137 DEBUG:teuthology.orchestra.run.smithi044:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-20T01:25:11.174 INFO:teuthology.orchestra.run.smithi044.stdout:ttyS1 2024-04-20T01:25:11.193 DEBUG:teuthology.parallel:result is None 2024-04-20T01:25:22.351 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 842 kB/s | 16 MB 00:19 2024-04-20T01:25:25.992 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 415 kB/s | 24 kB 00:00 2024-04-20T01:25:26.720 INFO:teuthology.orchestra.run.smithi123.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-04-20T01:25:26.720 INFO:teuthology.orchestra.run.smithi123.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-04-20T01:25:26.824 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:25:26.825 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-04-20T01:25:26.825 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:25:26.943 DEBUG:teuthology.orchestra.run.smithi123:> echo no | sudo yum reinstall kernel || true 2024-04-20T01:25:27.279 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:25:27.279 INFO:teuthology.orchestra.run.smithi123.stdout:Unable to read consumer identity 2024-04-20T01:25:27.349 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:25:27.349 INFO:teuthology.orchestra.run.smithi123.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-20T01:25:27.349 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:25:27.658 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:00:02 ago on Sat 20 Apr 2024 01:25:25 AM UTC. 2024-04-20T01:25:27.739 INFO:teuthology.orchestra.run.smithi123.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-04-20T01:25:27.740 INFO:teuthology.orchestra.run.smithi123.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-04-20T01:25:27.777 INFO:teuthology.orchestra.run.smithi123.stderr:Error: No packages marked for reinstall. 2024-04-20T01:25:27.823 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum reinstall -y kernel || true 2024-04-20T01:25:28.157 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:25:28.157 INFO:teuthology.orchestra.run.smithi123.stdout:Unable to read consumer identity 2024-04-20T01:25:28.234 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:25:28.234 INFO:teuthology.orchestra.run.smithi123.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-20T01:25:28.234 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:25:28.538 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:00:03 ago on Sat 20 Apr 2024 01:25:25 AM UTC. 2024-04-20T01:25:28.607 INFO:teuthology.orchestra.run.smithi123.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-04-20T01:25:28.607 INFO:teuthology.orchestra.run.smithi123.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-04-20T01:25:28.640 INFO:teuthology.orchestra.run.smithi123.stderr:Error: No packages marked for reinstall. 2024-04-20T01:25:28.688 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q kernel | sort -rV | head -n 1 2024-04-20T01:25:28.876 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:25:28.876 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-04-20T01:25:28.877 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-20T01:25:28.877 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-04-20T01:25:28.877 DEBUG:teuthology.orchestra.run.smithi123:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-20T01:25:28.909 INFO:teuthology.orchestra.run.smithi123.stdout:ttyS1 2024-04-20T01:25:28.933 DEBUG:teuthology.parallel:result is None 2024-04-20T01:25:28.933 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-20T01:25:28.940 INFO:teuthology.task.internal:Creating test directory... 2024-04-20T01:25:28.940 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T01:25:28.943 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T01:25:28.945 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T01:25:28.976 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T01:25:28.997 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-20T01:25:29.004 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-20T01:25:29.013 INFO:teuthology.task.internal:Creating archive directory... 2024-04-20T01:25:29.014 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T01:25:29.017 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T01:25:29.020 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T01:25:29.032 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T01:25:29.077 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-20T01:25:29.085 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-20T01:25:29.085 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T01:25:29.088 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T01:25:29.094 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T01:25:29.096 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T01:25:29.122 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T01:25:29.129 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T01:25:29.132 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T01:25:29.139 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T01:25:29.143 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T01:25:29.152 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T01:25:29.158 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T01:25:29.170 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T01:25:29.172 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-20T01:25:29.179 INFO:teuthology.task.internal:Configuring sudo... 2024-04-20T01:25:29.179 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T01:25:29.188 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T01:25:29.191 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T01:25:29.193 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T01:25:29.224 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-20T01:25:29.233 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-20T01:25:29.233 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T01:25:29.263 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T01:25:29.265 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T01:25:29.268 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T01:25:29.286 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T01:25:29.376 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T01:25:29.427 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T01:25:29.516 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T01:25:29.566 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:25:29.567 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T01:25:29.641 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T01:25:29.689 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T01:25:29.751 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T01:25:29.792 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T01:25:29.841 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T01:25:29.842 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T01:25:29.912 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T01:25:29.957 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T01:25:30.017 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T01:25:30.103 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T01:25:30.146 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T01:25:30.146 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T01:25:30.214 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T01:25:30.264 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T01:25:30.318 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T01:25:30.403 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T01:25:30.449 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T01:25:30.449 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T01:25:30.518 DEBUG:teuthology.orchestra.run.smithi044:> sudo service rsyslog restart 2024-04-20T01:25:30.521 DEBUG:teuthology.orchestra.run.smithi121:> sudo service rsyslog restart 2024-04-20T01:25:30.523 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-04-20T01:25:30.526 DEBUG:teuthology.orchestra.run.smithi165:> sudo service rsyslog restart 2024-04-20T01:25:30.593 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T01:25:30.595 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T01:25:30.599 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T01:25:30.638 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T01:25:31.028 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-20T01:25:31.036 INFO:teuthology.task.internal:Starting timer... 2024-04-20T01:25:31.036 INFO:teuthology.run_tasks:Running task pcp... 2024-04-20T01:25:31.056 INFO:teuthology.run_tasks:Running task selinux... 2024-04-20T01:25:31.066 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-04-20T01:25:31.066 DEBUG:teuthology.orchestra.run.smithi044:> sudo service auditd rotate 2024-04-20T01:25:31.140 INFO:teuthology.orchestra.run.smithi044.stdout:Rotating logs: 2024-04-20T01:25:31.142 DEBUG:teuthology.orchestra.run.smithi121:> sudo service auditd rotate 2024-04-20T01:25:31.214 INFO:teuthology.orchestra.run.smithi121.stdout:Rotating logs: 2024-04-20T01:25:31.217 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2024-04-20T01:25:31.280 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2024-04-20T01:25:31.282 DEBUG:teuthology.orchestra.run.smithi165:> sudo service auditd rotate 2024-04-20T01:25:31.330 INFO:teuthology.orchestra.run.smithi165.stdout:Rotating logs: 2024-04-20T01:25:31.332 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-20T01:25:31.333 DEBUG:teuthology.orchestra.run.smithi044:> /usr/sbin/getenforce 2024-04-20T01:25:31.365 INFO:teuthology.orchestra.run.smithi044.stdout:Permissive 2024-04-20T01:25:31.365 DEBUG:teuthology.orchestra.run.smithi121:> /usr/sbin/getenforce 2024-04-20T01:25:31.390 INFO:teuthology.orchestra.run.smithi121.stdout:Permissive 2024-04-20T01:25:31.390 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2024-04-20T01:25:31.437 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2024-04-20T01:25:31.437 DEBUG:teuthology.orchestra.run.smithi165:> /usr/sbin/getenforce 2024-04-20T01:25:31.467 INFO:teuthology.orchestra.run.smithi165.stdout:Permissive 2024-04-20T01:25:31.468 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi044.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi121.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi123.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi165.front.sepia.ceph.com': 'permissive'} 2024-04-20T01:25:31.468 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-20T01:25:31.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T01:25:31.499 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-20T01:25:31.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T01:25:31.527 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-20T01:25:31.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T01:25:31.553 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-20T01:25:31.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T01:25:31.583 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-20T01:25:31.584 DEBUG:teuthology.orchestra.run.smithi044:> sudo /usr/sbin/setenforce permissive 2024-04-20T01:25:31.625 DEBUG:teuthology.orchestra.run.smithi121:> sudo /usr/sbin/setenforce permissive 2024-04-20T01:25:31.667 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2024-04-20T01:25:31.707 DEBUG:teuthology.orchestra.run.smithi165:> sudo /usr/sbin/setenforce permissive 2024-04-20T01:25:31.734 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-20T01:25:31.755 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-20T01:25:31.779 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-20T01:25:31.781 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi044.front.sepia.ceph.com,smithi121.front.sepia.ceph.com,smithi123.front.sepia.ceph.com,smithi165.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-20T01:36:39.483 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi044.front.sepia.ceph.com'), Remote(name='ubuntu@smithi121.front.sepia.ceph.com'), Remote(name='ubuntu@smithi123.front.sepia.ceph.com'), Remote(name='ubuntu@smithi165.front.sepia.ceph.com')] 2024-04-20T01:36:39.484 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-04-20T01:36:39.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T01:36:39.565 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-04-20T01:36:39.645 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-04-20T01:36:39.645 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-04-20T01:36:39.646 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T01:36:39.721 DEBUG:teuthology.orchestra.run.smithi121:> true 2024-04-20T01:36:39.791 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-04-20T01:36:39.792 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-20T01:36:39.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T01:36:39.873 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-04-20T01:36:39.940 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-20T01:36:39.940 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-04-20T01:36:39.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T01:36:40.025 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-04-20T01:36:40.097 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-04-20T01:36:40.099 INFO:teuthology.run_tasks:Running task clock... 2024-04-20T01:36:40.109 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-20T01:36:40.109 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T01:36:40.110 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T01:36:40.112 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T01:36:40.112 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T01:36:40.115 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T01:36:40.115 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T01:36:40.117 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T01:36:40.117 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T01:36:40.148 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T01:36:40.149 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T01:36:40.149 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T01:36:40.163 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T01:36:40.164 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T01:36:40.167 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T01:36:40.175 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T01:36:40.191 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T01:36:40.235 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2024-04-20T01:36:40.244 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: ntpd: command not found 2024-04-20T01:36:40.245 INFO:teuthology.orchestra.run.smithi044.stderr:sudo: ntpd: command not found 2024-04-20T01:36:40.247 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2024-04-20T01:36:40.256 INFO:teuthology.orchestra.run.smithi121.stdout:506 Cannot talk to daemon 2024-04-20T01:36:40.259 INFO:teuthology.orchestra.run.smithi044.stdout:506 Cannot talk to daemon 2024-04-20T01:36:40.262 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T01:36:40.271 INFO:teuthology.orchestra.run.smithi165.stderr:sudo: ntpd: command not found 2024-04-20T01:36:40.271 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T01:36:40.276 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T01:36:40.277 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T01:36:40.284 INFO:teuthology.orchestra.run.smithi165.stdout:506 Cannot talk to daemon 2024-04-20T01:36:40.285 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T01:36:40.294 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T01:36:40.299 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T01:36:40.314 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T01:36:40.355 INFO:teuthology.orchestra.run.smithi123.stderr:bash: ntpq: command not found 2024-04-20T01:36:40.359 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T01:36:40.359 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-04-20T01:36:40.359 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T01:36:40.359 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T01:36:40.359 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T01:36:40.359 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T01:36:40.367 INFO:teuthology.orchestra.run.smithi044.stderr:bash: ntpq: command not found 2024-04-20T01:36:40.367 INFO:teuthology.orchestra.run.smithi121.stderr:bash: ntpq: command not found 2024-04-20T01:36:40.371 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T01:36:40.371 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2024-04-20T01:36:40.371 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T01:36:40.371 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T01:36:40.371 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T01:36:40.371 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T01:36:40.372 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T01:36:40.372 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-04-20T01:36:40.372 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T01:36:40.372 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T01:36:40.372 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T01:36:40.372 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T01:36:40.388 INFO:teuthology.orchestra.run.smithi165.stderr:bash: ntpq: command not found 2024-04-20T01:36:40.391 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T01:36:40.391 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-04-20T01:36:40.391 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T01:36:40.391 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T01:36:40.391 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T01:36:40.392 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T01:36:40.392 INFO:teuthology.run_tasks:Running task install... 2024-04-20T01:36:40.401 DEBUG:teuthology.task.install:project ceph 2024-04-20T01:36:40.401 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed'}} 2024-04-20T01:36:40.401 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed'} 2024-04-20T01:36:40.401 INFO:teuthology.task.install:Using flavor: default 2024-04-20T01:36:40.407 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-20T01:36:40.407 INFO:teuthology.task.install:extra packages: [] 2024-04-20T01:36:40.407 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed', 'tag': None, 'wait_for_package': False} 2024-04-20T01:36:40.408 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=15eecde2bab9803889d929c7e423d07970d8cbed 2024-04-20T01:36:40.410 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed', 'tag': None, 'wait_for_package': False} 2024-04-20T01:36:40.410 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=15eecde2bab9803889d929c7e423d07970d8cbed 2024-04-20T01:36:40.411 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed', 'tag': None, 'wait_for_package': False} 2024-04-20T01:36:40.411 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=15eecde2bab9803889d929c7e423d07970d8cbed 2024-04-20T01:36:40.412 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed', 'tag': None, 'wait_for_package': False} 2024-04-20T01:36:40.412 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=15eecde2bab9803889d929c7e423d07970d8cbed 2024-04-20T01:36:40.563 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/ 2024-04-20T01:36:40.564 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1252.g15eecde2 2024-04-20T01:36:40.568 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/ 2024-04-20T01:36:40.569 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1252.g15eecde2 2024-04-20T01:36:40.630 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/ 2024-04-20T01:36:40.630 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1252.g15eecde2 2024-04-20T01:36:40.634 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/ 2024-04-20T01:36:40.634 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1252.g15eecde2 2024-04-20T01:36:40.679 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T01:36:40.679 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T01:36:40.679 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T01:36:40.706 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-20T01:36:40.706 DEBUG:teuthology.orchestra.run.smithi121:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/15eecde2bab9803889d929c7e423d07970d8cbed/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T01:36:40.761 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T01:36:40.761 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T01:36:40.761 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T01:36:40.774 DEBUG:teuthology.orchestra.run.smithi121:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T01:36:40.789 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-20T01:36:40.789 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/15eecde2bab9803889d929c7e423d07970d8cbed/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T01:36:40.826 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T01:36:40.827 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:36:40.827 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T01:36:40.851 DEBUG:teuthology.orchestra.run.smithi121:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T01:36:40.856 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T01:36:40.863 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-20T01:36:40.863 DEBUG:teuthology.orchestra.run.smithi044:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/15eecde2bab9803889d929c7e423d07970d8cbed/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T01:36:40.917 INFO:teuthology.orchestra.run.smithi121.stdout:check_obsoletes = 1 2024-04-20T01:36:40.919 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum clean all 2024-04-20T01:36:40.934 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T01:36:40.939 DEBUG:teuthology.orchestra.run.smithi044:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T01:36:41.000 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2024-04-20T01:36:41.001 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2024-04-20T01:36:41.027 DEBUG:teuthology.orchestra.run.smithi044:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T01:36:41.059 INFO:teuthology.orchestra.run.smithi044.stdout:check_obsoletes = 1 2024-04-20T01:36:41.061 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum clean all 2024-04-20T01:36:41.295 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:36:41.377 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:36:41.497 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:36:41.799 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-04-18-1351-reef/15eecde2bab9803889d929c7e423d07970d8cbed/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T01:36:41.800 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T01:36:41.800 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T01:36:41.834 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-20T01:36:41.834 DEBUG:teuthology.orchestra.run.smithi165:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/15eecde2bab9803889d929c7e423d07970d8cbed/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T01:36:41.908 DEBUG:teuthology.orchestra.run.smithi165:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T01:36:41.992 DEBUG:teuthology.orchestra.run.smithi165:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T01:36:42.061 INFO:teuthology.orchestra.run.smithi165.stdout:check_obsoletes = 1 2024-04-20T01:36:42.062 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum clean all 2024-04-20T01:36:42.443 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:36:42.749 INFO:teuthology.orchestra.run.smithi044.stdout:63 files removed 2024-04-20T01:36:42.804 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-radosgw 2024-04-20T01:36:42.829 INFO:teuthology.orchestra.run.smithi123.stdout:63 files removed 2024-04-20T01:36:42.850 INFO:teuthology.orchestra.run.smithi121.stdout:63 files removed 2024-04-20T01:36:42.880 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw 2024-04-20T01:36:42.903 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-radosgw 2024-04-20T01:36:43.187 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:36:43.216 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:36:43.240 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:36:43.791 INFO:teuthology.orchestra.run.smithi165.stdout:63 files removed 2024-04-20T01:36:43.849 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-radosgw 2024-04-20T01:36:44.186 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:36:45.315 INFO:teuthology.orchestra.run.smithi121.stdout:ceph packages for x86_64 158 kB/s | 80 kB 00:00 2024-04-20T01:36:45.782 INFO:teuthology.orchestra.run.smithi121.stdout:ceph noarch packages 26 kB/s | 11 kB 00:00 2024-04-20T01:36:45.852 INFO:teuthology.orchestra.run.smithi044.stdout:ceph packages for x86_64 63 kB/s | 80 kB 00:01 2024-04-20T01:36:45.925 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 84 kB/s | 80 kB 00:00 2024-04-20T01:36:46.088 INFO:teuthology.orchestra.run.smithi165.stdout:ceph packages for x86_64 198 kB/s | 80 kB 00:00 2024-04-20T01:36:46.142 INFO:teuthology.orchestra.run.smithi121.stdout:ceph source packages 4.5 kB/s | 1.5 kB 00:00 2024-04-20T01:36:46.353 INFO:teuthology.orchestra.run.smithi044.stdout:ceph noarch packages 25 kB/s | 11 kB 00:00 2024-04-20T01:36:46.514 INFO:teuthology.orchestra.run.smithi165.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-04-20T01:36:46.525 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 20 kB/s | 11 kB 00:00 2024-04-20T01:36:46.526 INFO:teuthology.orchestra.run.smithi121.stdout:Copr repo for python3-asyncssh owned by ceph 9.6 kB/s | 3.5 kB 00:00 2024-04-20T01:36:47.006 INFO:teuthology.orchestra.run.smithi165.stdout:ceph source packages 3.2 kB/s | 1.5 kB 00:00 2024-04-20T01:36:47.070 INFO:teuthology.orchestra.run.smithi044.stdout:ceph source packages 2.2 kB/s | 1.5 kB 00:00 2024-04-20T01:36:47.223 INFO:teuthology.orchestra.run.smithi165.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2024-04-20T01:36:47.292 INFO:teuthology.orchestra.run.smithi044.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2024-04-20T01:36:47.293 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 2.0 kB/s | 1.5 kB 00:00 2024-04-20T01:36:47.501 INFO:teuthology.orchestra.run.smithi123.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2024-04-20T01:36:47.819 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2024-04-20T01:36:48.321 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:00 2024-04-20T01:36:48.668 INFO:teuthology.orchestra.run.smithi165.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2024-04-20T01:36:51.460 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 349 kB/s | 24 kB 00:00 2024-04-20T01:36:52.265 INFO:teuthology.orchestra.run.smithi165.stdout:lab-extras 386 kB/s | 24 kB 00:00 2024-04-20T01:36:52.288 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 379 kB/s | 24 kB 00:00 2024-04-20T01:36:52.779 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 48 MB/s | 60 MB 00:01 2024-04-20T01:36:53.115 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 12 MB/s | 8.8 MB 00:00 2024-04-20T01:36:53.617 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 47 MB/s | 60 MB 00:01 2024-04-20T01:36:56.416 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 42 MB/s | 67 MB 00:01 2024-04-20T01:37:04.359 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 15 MB/s | 8.8 MB 00:00 2024-04-20T01:37:04.620 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 52 MB/s | 67 MB 00:01 2024-04-20T01:37:05.444 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 930 kB/s | 16 MB 00:17 2024-04-20T01:37:07.376 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 67 MB 00:01 2024-04-20T01:37:08.719 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 47 MB/s | 60 MB 00:01 2024-04-20T01:37:09.024 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 522 kB/s | 24 kB 00:00 2024-04-20T01:37:09.651 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 16 MB/s | 8.8 MB 00:00 2024-04-20T01:37:12.519 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 60 MB 00:01 2024-04-20T01:37:15.194 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2024-04-20T01:37:22.401 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:37:22.403 INFO:teuthology.orchestra.run.smithi121.stdout:==================================================================================================== 2024-04-20T01:37:22.403 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-04-20T01:37:22.403 INFO:teuthology.orchestra.run.smithi121.stdout:==================================================================================================== 2024-04-20T01:37:22.404 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-04-20T01:37:22.404 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 13 M 2024-04-20T01:37:22.404 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrading: 2024-04-20T01:37:22.404 INFO:teuthology.orchestra.run.smithi121.stdout: librados2 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 3.7 M 2024-04-20T01:37:22.404 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 3.6 M 2024-04-20T01:37:22.404 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-04-20T01:37:22.404 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 5.5 M 2024-04-20T01:37:22.404 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 22 M 2024-04-20T01:37:22.404 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 24 k 2024-04-20T01:37:22.404 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-04-20T01:37:22.404 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 781 k 2024-04-20T01:37:22.404 INFO:teuthology.orchestra.run.smithi121.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-04-20T01:37:22.404 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-04-20T01:37:22.405 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 474 k 2024-04-20T01:37:22.405 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-04-20T01:37:22.405 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 6.5 M 2024-04-20T01:37:22.405 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-04-20T01:37:22.405 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-04-20T01:37:22.405 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 45 k 2024-04-20T01:37:22.405 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 129 k 2024-04-20T01:37:22.405 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 197 k 2024-04-20T01:37:22.405 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-04-20T01:37:22.405 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 372 k 2024-04-20T01:37:22.405 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 364 k 2024-04-20T01:37:22.405 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 107 k 2024-04-20T01:37:22.405 INFO:teuthology.orchestra.run.smithi121.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-04-20T01:37:22.405 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:37:22.406 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-20T01:37:22.406 INFO:teuthology.orchestra.run.smithi121.stdout:==================================================================================================== 2024-04-20T01:37:22.406 INFO:teuthology.orchestra.run.smithi121.stdout:Install 21 Packages 2024-04-20T01:37:22.406 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrade 2 Packages 2024-04-20T01:37:22.406 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:37:22.406 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 59 M 2024-04-20T01:37:22.406 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-20T01:37:22.655 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:37:22.658 INFO:teuthology.orchestra.run.smithi165.stdout:==================================================================================================== 2024-04-20T01:37:22.658 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-20T01:37:22.658 INFO:teuthology.orchestra.run.smithi165.stdout:==================================================================================================== 2024-04-20T01:37:22.658 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-20T01:37:22.658 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 13 M 2024-04-20T01:37:22.658 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrading: 2024-04-20T01:37:22.658 INFO:teuthology.orchestra.run.smithi165.stdout: librados2 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 3.7 M 2024-04-20T01:37:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 3.6 M 2024-04-20T01:37:22.659 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-20T01:37:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 5.5 M 2024-04-20T01:37:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 22 M 2024-04-20T01:37:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 24 k 2024-04-20T01:37:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-04-20T01:37:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 781 k 2024-04-20T01:37:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-04-20T01:37:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-04-20T01:37:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 474 k 2024-04-20T01:37:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-04-20T01:37:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 6.5 M 2024-04-20T01:37:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-04-20T01:37:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-04-20T01:37:22.660 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 45 k 2024-04-20T01:37:22.660 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 129 k 2024-04-20T01:37:22.660 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 197 k 2024-04-20T01:37:22.660 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-04-20T01:37:22.660 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 372 k 2024-04-20T01:37:22.660 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 364 k 2024-04-20T01:37:22.660 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 107 k 2024-04-20T01:37:22.660 INFO:teuthology.orchestra.run.smithi165.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-04-20T01:37:22.660 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:37:22.660 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-20T01:37:22.660 INFO:teuthology.orchestra.run.smithi165.stdout:==================================================================================================== 2024-04-20T01:37:22.660 INFO:teuthology.orchestra.run.smithi165.stdout:Install 21 Packages 2024-04-20T01:37:22.660 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrade 2 Packages 2024-04-20T01:37:22.660 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:37:22.661 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 59 M 2024-04-20T01:37:22.661 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-20T01:37:23.950 INFO:teuthology.orchestra.run.smithi165.stdout:(1/23): ceph-base-18.2.2-1252.g15eecde2.el8.x86 4.5 MB/s | 5.5 MB 00:01 2024-04-20T01:37:23.986 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 67 MB 00:01 2024-04-20T01:37:24.017 INFO:teuthology.orchestra.run.smithi165.stdout:(2/23): ceph-selinux-18.2.2-1252.g15eecde2.el8. 366 kB/s | 24 kB 00:00 2024-04-20T01:37:24.134 INFO:teuthology.orchestra.run.smithi165.stdout:(3/23): libcephfs2-18.2.2-1252.g15eecde2.el8.x8 6.5 MB/s | 781 kB 00:00 2024-04-20T01:37:24.178 INFO:teuthology.orchestra.run.smithi121.stdout:(1/23): ceph-common-18.2.2-1252.g15eecde2.el8.x 13 MB/s | 22 MB 00:01 2024-04-20T01:37:24.218 INFO:teuthology.orchestra.run.smithi165.stdout:(4/23): libradosstriper1-18.2.2-1252.g15eecde2. 5.6 MB/s | 474 kB 00:00 2024-04-20T01:37:24.228 INFO:teuthology.orchestra.run.smithi121.stdout:(2/23): ceph-selinux-18.2.2-1252.g15eecde2.el8. 489 kB/s | 24 kB 00:00 2024-04-20T01:37:24.387 INFO:teuthology.orchestra.run.smithi121.stdout:(3/23): libcephfs2-18.2.2-1252.g15eecde2.el8.x8 4.8 MB/s | 781 kB 00:00 2024-04-20T01:37:24.487 INFO:teuthology.orchestra.run.smithi121.stdout:(4/23): libradosstriper1-18.2.2-1252.g15eecde2. 4.6 MB/s | 474 kB 00:00 2024-04-20T01:37:25.059 INFO:teuthology.orchestra.run.smithi165.stdout:(5/23): librgw2-18.2.2-1252.g15eecde2.el8.x86_6 7.8 MB/s | 6.5 MB 00:00 2024-04-20T01:37:25.168 INFO:teuthology.orchestra.run.smithi165.stdout:(6/23): python3-ceph-argparse-18.2.2-1252.g15ee 412 kB/s | 45 kB 00:00 2024-04-20T01:37:25.171 INFO:teuthology.orchestra.run.smithi121.stdout:(5/23): ceph-base-18.2.2-1252.g15eecde2.el8.x86 2.1 MB/s | 5.5 MB 00:02 2024-04-20T01:37:25.243 INFO:teuthology.orchestra.run.smithi165.stdout:(7/23): python3-ceph-common-18.2.2-1252.g15eecd 1.7 MB/s | 129 kB 00:00 2024-04-20T01:37:25.329 INFO:teuthology.orchestra.run.smithi121.stdout:(6/23): librgw2-18.2.2-1252.g15eecde2.el8.x86_6 7.8 MB/s | 6.5 MB 00:00 2024-04-20T01:37:25.343 INFO:teuthology.orchestra.run.smithi165.stdout:(8/23): python3-cephfs-18.2.2-1252.g15eecde2.el 1.9 MB/s | 197 kB 00:00 2024-04-20T01:37:25.354 INFO:teuthology.orchestra.run.smithi121.stdout:(7/23): python3-ceph-argparse-18.2.2-1252.g15ee 243 kB/s | 45 kB 00:00 2024-04-20T01:37:25.380 INFO:teuthology.orchestra.run.smithi121.stdout:(8/23): python3-ceph-common-18.2.2-1252.g15eecd 2.5 MB/s | 129 kB 00:00 2024-04-20T01:37:25.435 INFO:teuthology.orchestra.run.smithi165.stdout:(9/23): python3-rados-18.2.2-1252.g15eecde2.el8 4.0 MB/s | 372 kB 00:00 2024-04-20T01:37:25.447 INFO:teuthology.orchestra.run.smithi121.stdout:(9/23): python3-rados-18.2.2-1252.g15eecde2.el8 5.5 MB/s | 372 kB 00:00 2024-04-20T01:37:25.472 INFO:teuthology.orchestra.run.smithi121.stdout:(10/23): python3-cephfs-18.2.2-1252.g15eecde2.e 1.6 MB/s | 197 kB 00:00 2024-04-20T01:37:25.510 INFO:teuthology.orchestra.run.smithi165.stdout:(10/23): python3-rbd-18.2.2-1252.g15eecde2.el8. 4.7 MB/s | 364 kB 00:00 2024-04-20T01:37:25.547 INFO:teuthology.orchestra.run.smithi121.stdout:(11/23): python3-rbd-18.2.2-1252.g15eecde2.el8. 3.5 MB/s | 364 kB 00:00 2024-04-20T01:37:25.573 INFO:teuthology.orchestra.run.smithi121.stdout:(12/23): python3-rgw-18.2.2-1252.g15eecde2.el8. 1.0 MB/s | 107 kB 00:00 2024-04-20T01:37:25.577 INFO:teuthology.orchestra.run.smithi165.stdout:(11/23): python3-rgw-18.2.2-1252.g15eecde2.el8. 1.6 MB/s | 107 kB 00:00 2024-04-20T01:37:25.748 INFO:teuthology.orchestra.run.smithi121.stdout:(13/23): liboath-2.6.2-3.el8.x86_64.rpm 336 kB/s | 59 kB 00:00 2024-04-20T01:37:25.803 INFO:teuthology.orchestra.run.smithi165.stdout:(12/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.3 MB/s | 306 kB 00:00 2024-04-20T01:37:25.861 INFO:teuthology.orchestra.run.smithi165.stdout:(13/23): liboath-2.6.2-3.el8.x86_64.rpm 1.0 MB/s | 59 kB 00:00 2024-04-20T01:37:25.873 INFO:teuthology.orchestra.run.smithi121.stdout:(14/23): gperftools-libs-2.7-9.el8.x86_64.rpm 940 kB/s | 306 kB 00:00 2024-04-20T01:37:25.899 INFO:teuthology.orchestra.run.smithi121.stdout:(15/23): libunwind-1.3.1-3.el8.x86_64.rpm 499 kB/s | 75 kB 00:00 2024-04-20T01:37:25.937 INFO:teuthology.orchestra.run.smithi165.stdout:(14/23): libunwind-1.3.1-3.el8.x86_64.rpm 1.0 MB/s | 75 kB 00:00 2024-04-20T01:37:26.091 INFO:teuthology.orchestra.run.smithi121.stdout:(16/23): python3-prettytable-0.7.2-14.el8.noarc 229 kB/s | 44 kB 00:00 2024-04-20T01:37:26.240 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:37:26.242 INFO:teuthology.orchestra.run.smithi044.stdout:==================================================================================================== 2024-04-20T01:37:26.242 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-20T01:37:26.243 INFO:teuthology.orchestra.run.smithi044.stdout:==================================================================================================== 2024-04-20T01:37:26.243 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-20T01:37:26.243 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 13 M 2024-04-20T01:37:26.243 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrading: 2024-04-20T01:37:26.243 INFO:teuthology.orchestra.run.smithi044.stdout: librados2 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 3.7 M 2024-04-20T01:37:26.243 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 3.6 M 2024-04-20T01:37:26.243 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-20T01:37:26.243 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 5.5 M 2024-04-20T01:37:26.243 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 22 M 2024-04-20T01:37:26.243 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 24 k 2024-04-20T01:37:26.243 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-04-20T01:37:26.243 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 781 k 2024-04-20T01:37:26.243 INFO:teuthology.orchestra.run.smithi044.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-04-20T01:37:26.244 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-04-20T01:37:26.244 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 474 k 2024-04-20T01:37:26.244 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-04-20T01:37:26.244 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 6.5 M 2024-04-20T01:37:26.244 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-04-20T01:37:26.244 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-04-20T01:37:26.244 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 45 k 2024-04-20T01:37:26.244 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 129 k 2024-04-20T01:37:26.244 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 197 k 2024-04-20T01:37:26.244 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-04-20T01:37:26.244 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 372 k 2024-04-20T01:37:26.244 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 364 k 2024-04-20T01:37:26.244 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 107 k 2024-04-20T01:37:26.245 INFO:teuthology.orchestra.run.smithi044.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-04-20T01:37:26.245 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:37:26.245 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-20T01:37:26.245 INFO:teuthology.orchestra.run.smithi044.stdout:==================================================================================================== 2024-04-20T01:37:26.245 INFO:teuthology.orchestra.run.smithi044.stdout:Install 21 Packages 2024-04-20T01:37:26.245 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrade 2 Packages 2024-04-20T01:37:26.245 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:37:26.245 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 59 M 2024-04-20T01:37:26.245 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-20T01:37:26.266 INFO:teuthology.orchestra.run.smithi121.stdout:(17/23): ceph-radosgw-18.2.2-1252.g15eecde2.el8 3.5 MB/s | 13 MB 00:03 2024-04-20T01:37:26.291 INFO:teuthology.orchestra.run.smithi121.stdout:(18/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.3 MB/s | 259 kB 00:00 2024-04-20T01:37:26.425 INFO:teuthology.orchestra.run.smithi121.stdout:(19/23): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 368 kB/s | 47 kB 00:00 2024-04-20T01:37:26.484 INFO:teuthology.orchestra.run.smithi121.stdout:(20/23): librdkafka-0.11.4-3.el8.x86_64.rpm 1.6 MB/s | 354 kB 00:00 2024-04-20T01:37:26.734 INFO:teuthology.orchestra.run.smithi121.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 2.0 MB/s | 1.7 MB 00:00 2024-04-20T01:37:26.845 INFO:teuthology.orchestra.run.smithi165.stdout:(15/23): ceph-common-18.2.2-1252.g15eecde2.el8. 5.3 MB/s | 22 MB 00:04 2024-04-20T01:37:26.951 INFO:teuthology.orchestra.run.smithi121.stdout:(22/23): librbd1-18.2.2-1252.g15eecde2.el8.x86_ 7.6 MB/s | 3.6 MB 00:00 2024-04-20T01:37:27.021 INFO:teuthology.orchestra.run.smithi165.stdout:(16/23): python3-prettytable-0.7.2-14.el8.noarc 251 kB/s | 44 kB 00:00 2024-04-20T01:37:27.162 INFO:teuthology.orchestra.run.smithi165.stdout:(17/23): ceph-radosgw-18.2.2-1252.g15eecde2.el8 2.9 MB/s | 13 MB 00:04 2024-04-20T01:37:27.193 INFO:teuthology.orchestra.run.smithi121.stdout:(23/23): librados2-18.2.2-1252.g15eecde2.el8.x8 4.8 MB/s | 3.7 MB 00:00 2024-04-20T01:37:27.194 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:37:27.195 INFO:teuthology.orchestra.run.smithi121.stdout:Total 12 MB/s | 59 MB 00:04 2024-04-20T01:37:27.196 INFO:teuthology.orchestra.run.smithi165.stdout:(18/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.4 MB/s | 259 kB 00:00 2024-04-20T01:37:27.241 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-20T01:37:27.286 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-20T01:37:27.286 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-20T01:37:27.378 INFO:teuthology.orchestra.run.smithi165.stdout:(19/23): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 267 kB/s | 47 kB 00:00 2024-04-20T01:37:27.422 INFO:teuthology.orchestra.run.smithi165.stdout:(20/23): librdkafka-0.11.4-3.el8.x86_64.rpm 1.3 MB/s | 354 kB 00:00 2024-04-20T01:37:27.508 INFO:teuthology.orchestra.run.smithi044.stdout:(1/23): ceph-base-18.2.2-1252.g15eecde2.el8.x86 5.1 MB/s | 5.5 MB 00:01 2024-04-20T01:37:27.567 INFO:teuthology.orchestra.run.smithi044.stdout:(2/23): ceph-selinux-18.2.2-1252.g15eecde2.el8. 419 kB/s | 24 kB 00:00 2024-04-20T01:37:27.616 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-20T01:37:27.625 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-20T01:37:27.775 INFO:teuthology.orchestra.run.smithi044.stdout:(3/23): libcephfs2-18.2.2-1252.g15eecde2.el8.x8 3.7 MB/s | 781 kB 00:00 2024-04-20T01:37:27.855 INFO:teuthology.orchestra.run.smithi165.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 926 kB/s | 1.7 MB 00:01 2024-04-20T01:37:27.925 INFO:teuthology.orchestra.run.smithi044.stdout:(4/23): libradosstriper1-18.2.2-1252.g15eecde2. 3.1 MB/s | 474 kB 00:00 2024-04-20T01:37:28.264 INFO:teuthology.orchestra.run.smithi165.stdout:(22/23): librados2-18.2.2-1252.g15eecde2.el8.x8 4.2 MB/s | 3.7 MB 00:00 2024-04-20T01:37:28.330 INFO:teuthology.orchestra.run.smithi165.stdout:(23/23): librbd1-18.2.2-1252.g15eecde2.el8.x86_ 3.9 MB/s | 3.6 MB 00:00 2024-04-20T01:37:28.334 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:37:28.334 INFO:teuthology.orchestra.run.smithi165.stdout:Total 10 MB/s | 59 MB 00:05 2024-04-20T01:37:28.387 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-20T01:37:28.432 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-20T01:37:28.432 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-20T01:37:28.534 INFO:teuthology.orchestra.run.smithi044.stdout:(5/23): ceph-radosgw-18.2.2-1252.g15eecde2.el8. 6.2 MB/s | 13 MB 00:02 2024-04-20T01:37:28.593 INFO:teuthology.orchestra.run.smithi044.stdout:(6/23): python3-ceph-argparse-18.2.2-1252.g15ee 770 kB/s | 45 kB 00:00 2024-04-20T01:37:28.643 INFO:teuthology.orchestra.run.smithi044.stdout:(7/23): python3-ceph-common-18.2.2-1252.g15eecd 2.5 MB/s | 129 kB 00:00 2024-04-20T01:37:28.718 INFO:teuthology.orchestra.run.smithi044.stdout:(8/23): python3-cephfs-18.2.2-1252.g15eecde2.el 2.6 MB/s | 197 kB 00:00 2024-04-20T01:37:28.762 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-20T01:37:28.771 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-20T01:37:28.793 INFO:teuthology.orchestra.run.smithi044.stdout:(9/23): python3-rados-18.2.2-1252.g15eecde2.el8 4.9 MB/s | 372 kB 00:00 2024-04-20T01:37:28.870 INFO:teuthology.orchestra.run.smithi044.stdout:(10/23): python3-rbd-18.2.2-1252.g15eecde2.el8. 4.7 MB/s | 364 kB 00:00 2024-04-20T01:37:28.933 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-20T01:37:28.978 INFO:teuthology.orchestra.run.smithi044.stdout:(11/23): librgw2-18.2.2-1252.g15eecde2.el8.x86_ 6.2 MB/s | 6.5 MB 00:01 2024-04-20T01:37:29.003 INFO:teuthology.orchestra.run.smithi044.stdout:(12/23): python3-rgw-18.2.2-1252.g15eecde2.el8. 798 kB/s | 107 kB 00:00 2024-04-20T01:37:29.165 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2024-04-20T01:37:29.307 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-04-20T01:37:29.353 INFO:teuthology.orchestra.run.smithi044.stdout:(13/23): liboath-2.6.2-3.el8.x86_64.rpm 168 kB/s | 59 kB 00:00 2024-04-20T01:37:29.462 INFO:teuthology.orchestra.run.smithi044.stdout:(14/23): gperftools-libs-2.7-9.el8.x86_64.rpm 633 kB/s | 306 kB 00:00 2024-04-20T01:37:29.562 INFO:teuthology.orchestra.run.smithi044.stdout:(15/23): libunwind-1.3.1-3.el8.x86_64.rpm 360 kB/s | 75 kB 00:00 2024-04-20T01:37:29.737 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-04-20T01:37:29.896 INFO:teuthology.orchestra.run.smithi044.stdout:(16/23): ceph-common-18.2.2-1252.g15eecde2.el8. 6.2 MB/s | 22 MB 00:03 2024-04-20T01:37:29.921 INFO:teuthology.orchestra.run.smithi044.stdout:(17/23): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 131 kB/s | 47 kB 00:00 2024-04-20T01:37:30.015 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 3/25 2024-04-20T01:37:30.059 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-20T01:37:30.071 INFO:teuthology.orchestra.run.smithi044.stdout:(18/23): python3-prettytable-0.7.2-14.el8.noarc 251 kB/s | 44 kB 00:00 2024-04-20T01:37:30.097 INFO:teuthology.orchestra.run.smithi044.stdout:(19/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.5 MB/s | 259 kB 00:00 2024-04-20T01:37:30.100 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 3/25 2024-04-20T01:37:30.188 INFO:teuthology.orchestra.run.smithi044.stdout:(20/23): librdkafka-0.11.4-3.el8.x86_64.rpm 3.0 MB/s | 354 kB 00:00 2024-04-20T01:37:30.254 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2024-04-20T01:37:30.264 INFO:teuthology.orchestra.run.smithi044.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 2.2 MB/s | 1.7 MB 00:00 2024-04-20T01:37:30.321 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 4/25 2024-04-20T01:37:30.420 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-04-20T01:37:30.571 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/25 2024-04-20T01:37:30.789 INFO:teuthology.orchestra.run.smithi044.stdout:(22/23): librbd1-18.2.2-1252.g15eecde2.el8.x86_ 5.9 MB/s | 3.6 MB 00:00 2024-04-20T01:37:30.806 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 6/25 2024-04-20T01:37:30.824 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-04-20T01:37:30.872 INFO:teuthology.orchestra.run.smithi044.stdout:(23/23): librados2-18.2.2-1252.g15eecde2.el8.x8 4.8 MB/s | 3.7 MB 00:00 2024-04-20T01:37:30.875 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:37:30.875 INFO:teuthology.orchestra.run.smithi044.stdout:Total 13 MB/s | 59 MB 00:04 2024-04-20T01:37:30.927 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-20T01:37:30.978 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-20T01:37:30.978 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-20T01:37:31.061 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 3/25 2024-04-20T01:37:31.179 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 3/25 2024-04-20T01:37:31.198 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 6/25 2024-04-20T01:37:31.334 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 4/25 2024-04-20T01:37:31.344 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-20T01:37:31.354 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-20T01:37:31.440 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 7/25 2024-04-20T01:37:31.506 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 7/25 2024-04-20T01:37:31.584 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/25 2024-04-20T01:37:31.729 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 8/25 2024-04-20T01:37:31.786 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 6/25 2024-04-20T01:37:31.948 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2024-04-20T01:37:32.019 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2024-04-20T01:37:32.178 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 6/25 2024-04-20T01:37:32.186 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-argparse-2:18.2.2-1252.g15eecde2.el8. 10/25 2024-04-20T01:37:32.344 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 7/25 2024-04-20T01:37:32.399 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 11/25 2024-04-20T01:37:32.410 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 7/25 2024-04-20T01:37:32.611 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-20T01:37:32.634 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 8/25 2024-04-20T01:37:32.665 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 12/25 2024-04-20T01:37:32.790 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2024-04-20T01:37:32.819 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2024-04-20T01:37:32.872 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2024-04-20T01:37:32.897 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_6 13/25 2024-04-20T01:37:32.925 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-04-20T01:37:32.972 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_6 13/25 2024-04-20T01:37:33.014 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-argparse-2:18.2.2-1252.g15eecde2.el8. 10/25 2024-04-20T01:37:33.156 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 14/25 2024-04-20T01:37:33.229 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 11/25 2024-04-20T01:37:33.381 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 15/25 2024-04-20T01:37:33.388 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-04-20T01:37:33.445 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 12/25 2024-04-20T01:37:33.611 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_6 13/25 2024-04-20T01:37:33.634 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 3/25 2024-04-20T01:37:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_6 13/25 2024-04-20T01:37:33.735 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 3/25 2024-04-20T01:37:33.916 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 4/25 2024-04-20T01:37:34.137 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 14/25 2024-04-20T01:37:34.140 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/25 2024-04-20T01:37:34.219 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 16/25 2024-04-20T01:37:34.351 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 6/25 2024-04-20T01:37:34.438 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 15/25 2024-04-20T01:37:34.447 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 17/25 2024-04-20T01:37:34.529 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 17/25 2024-04-20T01:37:34.780 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 6/25 2024-04-20T01:37:34.820 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 18/25 2024-04-20T01:37:34.950 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 7/25 2024-04-20T01:37:34.998 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-common-2:18.2.2-1252.g15eecde2.el8.x8 19/25 2024-04-20T01:37:35.051 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 7/25 2024-04-20T01:37:35.242 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 8/25 2024-04-20T01:37:35.254 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 16/25 2024-04-20T01:37:35.314 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 8/25 2024-04-20T01:37:35.410 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 17/25 2024-04-20T01:37:35.466 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 9/25 2024-04-20T01:37:35.481 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 17/25 2024-04-20T01:37:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-argparse-2:18.2.2-1252.g15eecde2.el8. 10/25 2024-04-20T01:37:35.748 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 18/25 2024-04-20T01:37:35.824 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 11/25 2024-04-20T01:37:35.928 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-common-2:18.2.2-1252.g15eecde2.el8.x8 19/25 2024-04-20T01:37:36.042 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 12/25 2024-04-20T01:37:36.208 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_6 13/25 2024-04-20T01:37:36.271 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_6 13/25 2024-04-20T01:37:36.628 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 14/25 2024-04-20T01:37:36.814 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 15/25 2024-04-20T01:37:37.795 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 16/25 2024-04-20T01:37:37.950 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 17/25 2024-04-20T01:37:38.027 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 17/25 2024-04-20T01:37:38.291 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 18/25 2024-04-20T01:37:38.442 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-common-2:18.2.2-1252.g15eecde2.el8.x8 19/25 2024-04-20T01:37:38.850 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 20/25 2024-04-20T01:37:39.130 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 20/25 2024-04-20T01:37:39.151 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:37:39.154 INFO:teuthology.orchestra.run.smithi123.stdout:==================================================================================================== 2024-04-20T01:37:39.154 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-04-20T01:37:39.154 INFO:teuthology.orchestra.run.smithi123.stdout:==================================================================================================== 2024-04-20T01:37:39.154 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-20T01:37:39.154 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 13 M 2024-04-20T01:37:39.154 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-04-20T01:37:39.154 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 3.7 M 2024-04-20T01:37:39.154 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 3.6 M 2024-04-20T01:37:39.154 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-04-20T01:37:39.154 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 5.5 M 2024-04-20T01:37:39.154 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 22 M 2024-04-20T01:37:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 24 k 2024-04-20T01:37:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-04-20T01:37:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 781 k 2024-04-20T01:37:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-04-20T01:37:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-04-20T01:37:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 474 k 2024-04-20T01:37:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-04-20T01:37:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 6.5 M 2024-04-20T01:37:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-04-20T01:37:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-04-20T01:37:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 45 k 2024-04-20T01:37:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 129 k 2024-04-20T01:37:39.155 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 197 k 2024-04-20T01:37:39.156 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-04-20T01:37:39.156 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 372 k 2024-04-20T01:37:39.156 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 364 k 2024-04-20T01:37:39.156 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 107 k 2024-04-20T01:37:39.156 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-04-20T01:37:39.156 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:37:39.156 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-20T01:37:39.156 INFO:teuthology.orchestra.run.smithi123.stdout:==================================================================================================== 2024-04-20T01:37:39.156 INFO:teuthology.orchestra.run.smithi123.stdout:Install 21 Packages 2024-04-20T01:37:39.156 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 2 Packages 2024-04-20T01:37:39.156 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:37:39.157 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 59 M 2024-04-20T01:37:39.157 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-20T01:37:39.610 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 20/25 2024-04-20T01:37:39.724 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 20/25 2024-04-20T01:37:39.827 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 20/25 2024-04-20T01:37:40.043 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 21/25 2024-04-20T01:37:40.114 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 21/25 2024-04-20T01:37:40.327 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 22/25 2024-04-20T01:37:40.369 INFO:teuthology.orchestra.run.smithi123.stdout:(1/23): ceph-base-18.2.2-1252.g15eecde2.el8.x86 6.1 MB/s | 5.5 MB 00:00 2024-04-20T01:37:40.421 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 20/25 2024-04-20T01:37:40.444 INFO:teuthology.orchestra.run.smithi123.stdout:(2/23): ceph-selinux-18.2.2-1252.g15eecde2.el8. 326 kB/s | 24 kB 00:00 2024-04-20T01:37:40.611 INFO:teuthology.orchestra.run.smithi123.stdout:(3/23): libcephfs2-18.2.2-1252.g15eecde2.el8.x8 4.6 MB/s | 781 kB 00:00 2024-04-20T01:37:40.701 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 21/25 2024-04-20T01:37:40.728 INFO:teuthology.orchestra.run.smithi123.stdout:(4/23): libradosstriper1-18.2.2-1252.g15eecde2. 4.0 MB/s | 474 kB 00:00 2024-04-20T01:37:40.791 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 21/25 2024-04-20T01:37:40.980 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 22/25 2024-04-20T01:37:41.645 INFO:teuthology.orchestra.run.smithi123.stdout:(5/23): librgw2-18.2.2-1252.g15eecde2.el8.x86_6 7.1 MB/s | 6.5 MB 00:00 2024-04-20T01:37:41.703 INFO:teuthology.orchestra.run.smithi123.stdout:(6/23): python3-ceph-argparse-18.2.2-1252.g15ee 770 kB/s | 45 kB 00:00 2024-04-20T01:37:41.812 INFO:teuthology.orchestra.run.smithi123.stdout:(7/23): python3-ceph-common-18.2.2-1252.g15eecd 1.2 MB/s | 129 kB 00:00 2024-04-20T01:37:41.879 INFO:teuthology.orchestra.run.smithi123.stdout:(8/23): python3-cephfs-18.2.2-1252.g15eecde2.el 2.9 MB/s | 197 kB 00:00 2024-04-20T01:37:41.954 INFO:teuthology.orchestra.run.smithi123.stdout:(9/23): python3-rados-18.2.2-1252.g15eecde2.el8 4.9 MB/s | 372 kB 00:00 2024-04-20T01:37:42.065 INFO:teuthology.orchestra.run.smithi123.stdout:(10/23): python3-rbd-18.2.2-1252.g15eecde2.el8. 3.2 MB/s | 364 kB 00:00 2024-04-20T01:37:42.146 INFO:teuthology.orchestra.run.smithi123.stdout:(11/23): python3-rgw-18.2.2-1252.g15eecde2.el8. 1.3 MB/s | 107 kB 00:00 2024-04-20T01:37:42.546 INFO:teuthology.orchestra.run.smithi123.stdout:(12/23): gperftools-libs-2.7-9.el8.x86_64.rpm 766 kB/s | 306 kB 00:00 2024-04-20T01:37:42.568 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 20/25 2024-04-20T01:37:42.638 INFO:teuthology.orchestra.run.smithi123.stdout:(13/23): liboath-2.6.2-3.el8.x86_64.rpm 642 kB/s | 59 kB 00:00 2024-04-20T01:37:42.731 INFO:teuthology.orchestra.run.smithi123.stdout:(14/23): libunwind-1.3.1-3.el8.x86_64.rpm 820 kB/s | 75 kB 00:00 2024-04-20T01:37:42.784 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 20/25 2024-04-20T01:37:42.981 INFO:teuthology.orchestra.run.smithi123.stdout:(15/23): thrift-0.13.0-2.el8.x86_64.rpm 6.9 MB/s | 1.7 MB 00:00 2024-04-20T01:37:43.139 INFO:teuthology.orchestra.run.smithi123.stdout:(16/23): python3-prettytable-0.7.2-14.el8.noarc 278 kB/s | 44 kB 00:00 2024-04-20T01:37:43.281 INFO:teuthology.orchestra.run.smithi123.stdout:(17/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.8 MB/s | 259 kB 00:00 2024-04-20T01:37:43.349 INFO:teuthology.orchestra.run.smithi123.stdout:(18/23): librdkafka-0.11.4-3.el8.x86_64.rpm 5.2 MB/s | 354 kB 00:00 2024-04-20T01:37:43.443 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 20/25 2024-04-20T01:37:43.532 INFO:teuthology.orchestra.run.smithi123.stdout:(19/23): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 257 kB/s | 47 kB 00:00 2024-04-20T01:37:43.711 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 21/25 2024-04-20T01:37:43.791 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 21/25 2024-04-20T01:37:43.974 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 22/25 2024-04-20T01:37:44.599 INFO:teuthology.orchestra.run.smithi123.stdout:(20/23): librados2-18.2.2-1252.g15eecde2.el8.x8 3.5 MB/s | 3.7 MB 00:01 2024-04-20T01:37:45.324 INFO:teuthology.orchestra.run.smithi123.stdout:(21/23): librbd1-18.2.2-1252.g15eecde2.el8.x86_ 4.9 MB/s | 3.6 MB 00:00 2024-04-20T01:37:56.973 INFO:teuthology.orchestra.run.smithi123.stdout:(22/23): ceph-radosgw-18.2.2-1252.g15eecde2.el8 765 kB/s | 13 MB 00:17 2024-04-20T01:37:57.740 INFO:teuthology.orchestra.run.smithi123.stdout:(23/23): ceph-common-18.2.2-1252.g15eecde2.el8. 1.2 MB/s | 22 MB 00:18 2024-04-20T01:37:57.742 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:37:57.743 INFO:teuthology.orchestra.run.smithi123.stdout:Total 3.2 MB/s | 59 MB 00:18 2024-04-20T01:37:57.790 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-20T01:37:57.835 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-20T01:37:57.835 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-20T01:37:58.166 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-20T01:37:58.176 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-20T01:37:59.458 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-20T01:37:59.462 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 22/25 2024-04-20T01:37:59.463 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /sys 2024-04-20T01:37:59.463 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /proc 2024-04-20T01:37:59.463 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /dev 2024-04-20T01:37:59.463 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /run 2024-04-20T01:37:59.463 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /mnt 2024-04-20T01:37:59.463 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /var/tmp 2024-04-20T01:37:59.463 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /home 2024-04-20T01:37:59.463 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /tmp 2024-04-20T01:37:59.463 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /dev 2024-04-20T01:37:59.463 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:37:59.648 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2024-04-20T01:37:59.823 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-04-20T01:38:00.200 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 22/25 2024-04-20T01:38:00.200 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /sys 2024-04-20T01:38:00.200 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /proc 2024-04-20T01:38:00.200 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /dev 2024-04-20T01:38:00.200 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /run 2024-04-20T01:38:00.200 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /mnt 2024-04-20T01:38:00.200 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /var/tmp 2024-04-20T01:38:00.200 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /home 2024-04-20T01:38:00.200 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /tmp 2024-04-20T01:38:00.200 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /dev 2024-04-20T01:38:00.200 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:38:00.231 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-04-20T01:38:00.465 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 3/25 2024-04-20T01:38:00.565 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 3/25 2024-04-20T01:38:00.745 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 4/25 2024-04-20T01:38:00.814 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 23/25 2024-04-20T01:38:00.836 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 23/25 2024-04-20T01:38:00.836 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-20T01:38:00.987 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/25 2024-04-20T01:38:01.093 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-20T01:38:01.094 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-20T01:38:01.206 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 6/25 2024-04-20T01:38:01.556 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 23/25 2024-04-20T01:38:01.578 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 23/25 2024-04-20T01:38:01.578 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-20T01:38:01.601 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 6/25 2024-04-20T01:38:01.800 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-20T01:38:01.800 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-20T01:38:01.822 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-20T01:38:01.822 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 1/25 2024-04-20T01:38:01.822 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 2/25 2024-04-20T01:38:01.822 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 3/25 2024-04-20T01:38:01.823 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 4/25 2024-04-20T01:38:01.823 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 5/25 2024-04-20T01:38:01.823 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_6 6/25 2024-04-20T01:38:01.823 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 7/25 2024-04-20T01:38:01.823 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1252.g15eecde2.el8. 8/25 2024-04-20T01:38:01.823 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-common-2:18.2.2-1252.g15eecde2.el8.x8 9/25 2024-04-20T01:38:01.823 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 10/25 2024-04-20T01:38:01.823 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 11/25 2024-04-20T01:38:01.823 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 12/25 2024-04-20T01:38:01.823 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 13/25 2024-04-20T01:38:01.823 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2024-04-20T01:38:01.823 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2024-04-20T01:38:01.823 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2024-04-20T01:38:01.823 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2024-04-20T01:38:01.823 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-04-20T01:38:01.824 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 19/25 2024-04-20T01:38:01.824 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 20/25 2024-04-20T01:38:01.824 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 21/25 2024-04-20T01:38:01.824 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 22/25 2024-04-20T01:38:01.824 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-04-20T01:38:01.824 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 24/25 2024-04-20T01:38:01.839 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 7/25 2024-04-20T01:38:01.886 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 7/25 2024-04-20T01:38:02.112 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 8/25 2024-04-20T01:38:02.312 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2024-04-20T01:38:02.390 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2024-04-20T01:38:02.475 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-20T01:38:02.475 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 1/25 2024-04-20T01:38:02.475 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 2/25 2024-04-20T01:38:02.475 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 3/25 2024-04-20T01:38:02.475 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 4/25 2024-04-20T01:38:02.475 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 5/25 2024-04-20T01:38:02.475 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_6 6/25 2024-04-20T01:38:02.475 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 7/25 2024-04-20T01:38:02.475 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1252.g15eecde2.el8. 8/25 2024-04-20T01:38:02.475 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-common-2:18.2.2-1252.g15eecde2.el8.x8 9/25 2024-04-20T01:38:02.475 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 10/25 2024-04-20T01:38:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 11/25 2024-04-20T01:38:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 12/25 2024-04-20T01:38:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 13/25 2024-04-20T01:38:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2024-04-20T01:38:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2024-04-20T01:38:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2024-04-20T01:38:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2024-04-20T01:38:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-04-20T01:38:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 19/25 2024-04-20T01:38:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 20/25 2024-04-20T01:38:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 21/25 2024-04-20T01:38:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 22/25 2024-04-20T01:38:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-04-20T01:38:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 24/25 2024-04-20T01:38:02.534 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:18.2.2-1252.g15eecde2.el8. 10/25 2024-04-20T01:38:02.758 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 11/25 2024-04-20T01:38:03.023 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 12/25 2024-04-20T01:38:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_6 13/25 2024-04-20T01:38:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_6 13/25 2024-04-20T01:38:03.484 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 14/25 2024-04-20T01:38:03.698 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 15/25 2024-04-20T01:38:04.531 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 16/25 2024-04-20T01:38:04.730 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 17/25 2024-04-20T01:38:04.816 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 17/25 2024-04-20T01:38:05.106 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 18/25 2024-04-20T01:38:05.289 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:18.2.2-1252.g15eecde2.el8.x8 19/25 2024-04-20T01:38:06.098 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 22/25 2024-04-20T01:38:06.098 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /sys 2024-04-20T01:38:06.098 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /proc 2024-04-20T01:38:06.098 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /dev 2024-04-20T01:38:06.098 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /run 2024-04-20T01:38:06.098 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /mnt 2024-04-20T01:38:06.098 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /var/tmp 2024-04-20T01:38:06.098 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /home 2024-04-20T01:38:06.098 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /tmp 2024-04-20T01:38:06.099 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /dev 2024-04-20T01:38:06.099 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:38:07.549 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 23/25 2024-04-20T01:38:07.575 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 23/25 2024-04-20T01:38:07.575 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-20T01:38:07.808 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-20T01:38:07.808 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-20T01:38:08.565 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-20T01:38:08.565 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 1/25 2024-04-20T01:38:08.565 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 2/25 2024-04-20T01:38:08.565 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 3/25 2024-04-20T01:38:08.565 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 4/25 2024-04-20T01:38:08.565 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 5/25 2024-04-20T01:38:08.565 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_6 6/25 2024-04-20T01:38:08.565 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 7/25 2024-04-20T01:38:08.565 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1252.g15eecde2.el8. 8/25 2024-04-20T01:38:08.565 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-common-2:18.2.2-1252.g15eecde2.el8.x8 9/25 2024-04-20T01:38:08.565 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 10/25 2024-04-20T01:38:08.565 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 11/25 2024-04-20T01:38:08.565 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 12/25 2024-04-20T01:38:08.565 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 13/25 2024-04-20T01:38:08.566 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2024-04-20T01:38:08.566 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2024-04-20T01:38:08.566 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2024-04-20T01:38:08.566 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2024-04-20T01:38:08.566 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 18/25 2024-04-20T01:38:08.566 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 19/25 2024-04-20T01:38:08.566 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 20/25 2024-04-20T01:38:08.566 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 21/25 2024-04-20T01:38:08.566 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 22/25 2024-04-20T01:38:08.566 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-04-20T01:38:08.566 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 24/25 2024-04-20T01:38:09.292 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 20/25 2024-04-20T01:38:09.547 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 20/25 2024-04-20T01:38:10.138 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 20/25 2024-04-20T01:38:10.434 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 21/25 2024-04-20T01:38:10.518 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 21/25 2024-04-20T01:38:10.715 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 22/25 2024-04-20T01:38:14.329 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-04-20T01:38:14.329 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2024-04-20T01:38:14.886 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:38:14.887 INFO:teuthology.orchestra.run.smithi121.stdout:Upgraded: 2024-04-20T01:38:14.887 INFO:teuthology.orchestra.run.smithi121.stdout: librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:14.887 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:14.887 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-04-20T01:38:14.887 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:14.887 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:14.887 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:14.887 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:14.887 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-04-20T01:38:14.887 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:14.887 INFO:teuthology.orchestra.run.smithi121.stdout: liboath-2.6.2-3.el8.x86_64 2024-04-20T01:38:14.887 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-04-20T01:38:14.887 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:14.888 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-04-20T01:38:14.888 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:14.888 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind-1.3.1-3.el8.x86_64 2024-04-20T01:38:14.888 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-04-20T01:38:14.888 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:14.888 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:14.888 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:14.888 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-04-20T01:38:14.888 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:14.888 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:14.888 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:14.888 INFO:teuthology.orchestra.run.smithi121.stdout: thrift-0.13.0-2.el8.x86_64 2024-04-20T01:38:14.888 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:38:14.888 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:38:14.962 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-04-20T01:38:14.962 INFO:teuthology.orchestra.run.smithi165.stdout:Installed products updated. 2024-04-20T01:38:15.119 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-test 2024-04-20T01:38:15.473 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:38:15.519 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:38:15.519 INFO:teuthology.orchestra.run.smithi165.stdout:Upgraded: 2024-04-20T01:38:15.519 INFO:teuthology.orchestra.run.smithi165.stdout: librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:15.519 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:15.519 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-20T01:38:15.519 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:15.519 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:15.519 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:15.519 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:15.520 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-04-20T01:38:15.520 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:15.520 INFO:teuthology.orchestra.run.smithi165.stdout: liboath-2.6.2-3.el8.x86_64 2024-04-20T01:38:15.520 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-04-20T01:38:15.520 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:15.520 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-04-20T01:38:15.520 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:15.520 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind-1.3.1-3.el8.x86_64 2024-04-20T01:38:15.520 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-04-20T01:38:15.520 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:15.520 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:15.520 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:15.520 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-04-20T01:38:15.520 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:15.520 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:15.520 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:15.521 INFO:teuthology.orchestra.run.smithi165.stdout: thrift-0.13.0-2.el8.x86_64 2024-04-20T01:38:15.521 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:38:15.521 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:38:15.714 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-test 2024-04-20T01:38:16.066 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:38:17.314 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 23 kB/s | 4.5 kB 00:00 2024-04-20T01:38:17.723 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 kB/s | 4.5 kB 00:00 2024-04-20T01:38:17.943 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 33 kB/s | 4.1 kB 00:00 2024-04-20T01:38:18.343 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 kB/s | 4.1 kB 00:00 2024-04-20T01:38:20.243 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:38:20.244 INFO:teuthology.orchestra.run.smithi121.stdout:============================================================================================ 2024-04-20T01:38:20.244 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-04-20T01:38:20.244 INFO:teuthology.orchestra.run.smithi121.stdout:============================================================================================ 2024-04-20T01:38:20.244 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-04-20T01:38:20.244 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 47 M 2024-04-20T01:38:20.244 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-04-20T01:38:20.244 INFO:teuthology.orchestra.run.smithi121.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-04-20T01:38:20.244 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 142 k 2024-04-20T01:38:20.245 INFO:teuthology.orchestra.run.smithi121.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-04-20T01:38:20.245 INFO:teuthology.orchestra.run.smithi121.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-04-20T01:38:20.245 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-04-20T01:38:20.245 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:38:20.245 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-20T01:38:20.245 INFO:teuthology.orchestra.run.smithi121.stdout:============================================================================================ 2024-04-20T01:38:20.245 INFO:teuthology.orchestra.run.smithi121.stdout:Install 6 Packages 2024-04-20T01:38:20.245 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:38:20.246 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 48 M 2024-04-20T01:38:20.246 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 213 M 2024-04-20T01:38:20.246 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-20T01:38:20.285 INFO:teuthology.orchestra.run.smithi121.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.8 MB/s | 70 kB 00:00 2024-04-20T01:38:20.411 INFO:teuthology.orchestra.run.smithi121.stdout:(2/6): socat-1.7.4.1-1.el8.x86_64.rpm 2.6 MB/s | 323 kB 00:00 2024-04-20T01:38:20.486 INFO:teuthology.orchestra.run.smithi121.stdout:(3/6): libcephsqlite-18.2.2-1252.g15eecde2.el8. 593 kB/s | 142 kB 00:00 2024-04-20T01:38:20.528 INFO:teuthology.orchestra.run.smithi121.stdout:(4/6): jq-1.6-7.el8.x86_64.rpm 1.7 MB/s | 202 kB 00:00 2024-04-20T01:38:20.633 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:38:20.634 INFO:teuthology.orchestra.run.smithi165.stdout:============================================================================================ 2024-04-20T01:38:20.634 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-20T01:38:20.634 INFO:teuthology.orchestra.run.smithi165.stdout:============================================================================================ 2024-04-20T01:38:20.634 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-20T01:38:20.634 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 47 M 2024-04-20T01:38:20.634 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-20T01:38:20.634 INFO:teuthology.orchestra.run.smithi165.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-04-20T01:38:20.634 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 142 k 2024-04-20T01:38:20.634 INFO:teuthology.orchestra.run.smithi165.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-04-20T01:38:20.635 INFO:teuthology.orchestra.run.smithi165.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-04-20T01:38:20.635 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-04-20T01:38:20.635 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:38:20.635 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-20T01:38:20.635 INFO:teuthology.orchestra.run.smithi165.stdout:============================================================================================ 2024-04-20T01:38:20.635 INFO:teuthology.orchestra.run.smithi165.stdout:Install 6 Packages 2024-04-20T01:38:20.635 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:38:20.635 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 48 M 2024-04-20T01:38:20.636 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 213 M 2024-04-20T01:38:20.636 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-20T01:38:20.661 INFO:teuthology.orchestra.run.smithi121.stdout:(5/6): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-04-20T01:38:20.669 INFO:teuthology.orchestra.run.smithi165.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.1 MB/s | 70 kB 00:00 2024-04-20T01:38:20.894 INFO:teuthology.orchestra.run.smithi165.stdout:(2/6): socat-1.7.4.1-1.el8.x86_64.rpm 1.4 MB/s | 323 kB 00:00 2024-04-20T01:38:20.978 INFO:teuthology.orchestra.run.smithi165.stdout:(3/6): libcephsqlite-18.2.2-1252.g15eecde2.el8. 415 kB/s | 142 kB 00:00 2024-04-20T01:38:21.028 INFO:teuthology.orchestra.run.smithi165.stdout:(4/6): jq-1.6-7.el8.x86_64.rpm 1.5 MB/s | 202 kB 00:00 2024-04-20T01:38:21.112 INFO:teuthology.orchestra.run.smithi165.stdout:(5/6): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-04-20T01:38:21.791 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-04-20T01:38:21.791 INFO:teuthology.orchestra.run.smithi044.stdout:Installed products updated. 2024-04-20T01:38:22.429 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:38:22.429 INFO:teuthology.orchestra.run.smithi044.stdout:Upgraded: 2024-04-20T01:38:22.429 INFO:teuthology.orchestra.run.smithi044.stdout: librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:22.429 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:22.429 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-20T01:38:22.429 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:22.429 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:22.429 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:22.429 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:22.429 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-04-20T01:38:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: liboath-2.6.2-3.el8.x86_64 2024-04-20T01:38:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-04-20T01:38:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-04-20T01:38:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind-1.3.1-3.el8.x86_64 2024-04-20T01:38:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-04-20T01:38:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-04-20T01:38:22.430 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:22.431 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:22.431 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:22.431 INFO:teuthology.orchestra.run.smithi044.stdout: thrift-0.13.0-2.el8.x86_64 2024-04-20T01:38:22.431 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:38:22.431 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:38:22.635 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-test 2024-04-20T01:38:23.047 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:38:25.393 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 33 kB/s | 4.1 kB 00:00 2024-04-20T01:38:25.757 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 33 kB/s | 4.5 kB 00:00 2024-04-20T01:38:26.327 INFO:teuthology.orchestra.run.smithi121.stdout:(6/6): ceph-test-18.2.2-1252.g15eecde2.el8.x86_ 7.8 MB/s | 47 MB 00:06 2024-04-20T01:38:26.329 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:38:26.330 INFO:teuthology.orchestra.run.smithi121.stdout:Total 7.9 MB/s | 48 MB 00:06 2024-04-20T01:38:26.367 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-20T01:38:26.388 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-20T01:38:26.388 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-20T01:38:26.655 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-20T01:38:26.664 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-20T01:38:27.169 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-20T01:38:27.319 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 1/6 2024-04-20T01:38:27.327 INFO:teuthology.orchestra.run.smithi165.stdout:(6/6): ceph-test-18.2.2-1252.g15eecde2.el8.x86_ 7.0 MB/s | 47 MB 00:06 2024-04-20T01:38:27.330 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:38:27.330 INFO:teuthology.orchestra.run.smithi165.stdout:Total 7.2 MB/s | 48 MB 00:06 2024-04-20T01:38:27.370 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-20T01:38:27.391 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-20T01:38:27.392 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-20T01:38:27.432 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 1/6 2024-04-20T01:38:27.630 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : jq-1.6-7.el8.x86_64 2/6 2024-04-20T01:38:27.661 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-20T01:38:27.670 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-20T01:38:27.836 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 3/6 2024-04-20T01:38:28.056 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2024-04-20T01:38:28.133 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-20T01:38:28.202 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 5/6 2024-04-20T01:38:28.275 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 1/6 2024-04-20T01:38:28.397 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 1/6 2024-04-20T01:38:28.507 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:38:28.508 INFO:teuthology.orchestra.run.smithi044.stdout:============================================================================================ 2024-04-20T01:38:28.508 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-20T01:38:28.508 INFO:teuthology.orchestra.run.smithi044.stdout:============================================================================================ 2024-04-20T01:38:28.508 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-20T01:38:28.508 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 47 M 2024-04-20T01:38:28.508 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-20T01:38:28.508 INFO:teuthology.orchestra.run.smithi044.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-04-20T01:38:28.508 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 142 k 2024-04-20T01:38:28.509 INFO:teuthology.orchestra.run.smithi044.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-04-20T01:38:28.509 INFO:teuthology.orchestra.run.smithi044.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-04-20T01:38:28.509 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-04-20T01:38:28.509 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:38:28.509 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-20T01:38:28.509 INFO:teuthology.orchestra.run.smithi044.stdout:============================================================================================ 2024-04-20T01:38:28.509 INFO:teuthology.orchestra.run.smithi044.stdout:Install 6 Packages 2024-04-20T01:38:28.509 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:38:28.510 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 48 M 2024-04-20T01:38:28.510 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 213 M 2024-04-20T01:38:28.510 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-20T01:38:28.553 INFO:teuthology.orchestra.run.smithi044.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2024-04-20T01:38:28.579 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : jq-1.6-7.el8.x86_64 2/6 2024-04-20T01:38:28.661 INFO:teuthology.orchestra.run.smithi044.stdout:(2/6): socat-1.7.4.1-1.el8.x86_64.rpm 3.2 MB/s | 323 kB 00:00 2024-04-20T01:38:28.712 INFO:teuthology.orchestra.run.smithi044.stdout:(3/6): libcephsqlite-18.2.2-1252.g15eecde2.el8. 705 kB/s | 142 kB 00:00 2024-04-20T01:38:28.776 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 3/6 2024-04-20T01:38:28.779 INFO:teuthology.orchestra.run.smithi044.stdout:(4/6): jq-1.6-7.el8.x86_64.rpm 1.7 MB/s | 202 kB 00:00 2024-04-20T01:38:28.879 INFO:teuthology.orchestra.run.smithi044.stdout:(5/6): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-04-20T01:38:28.929 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2024-04-20T01:38:29.100 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 5/6 2024-04-20T01:38:29.761 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 22/25 2024-04-20T01:38:29.761 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2024-04-20T01:38:29.761 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2024-04-20T01:38:29.761 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /dev 2024-04-20T01:38:29.761 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /run 2024-04-20T01:38:29.761 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2024-04-20T01:38:29.761 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2024-04-20T01:38:29.761 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2024-04-20T01:38:29.761 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2024-04-20T01:38:29.761 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /dev 2024-04-20T01:38:29.761 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:38:31.147 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 23/25 2024-04-20T01:38:31.169 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 23/25 2024-04-20T01:38:31.169 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-20T01:38:31.403 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-20T01:38:31.404 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-20T01:38:31.579 INFO:teuthology.orchestra.run.smithi044.stdout:(6/6): ceph-test-18.2.2-1252.g15eecde2.el8.x86_ 15 MB/s | 47 MB 00:03 2024-04-20T01:38:31.581 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:38:31.581 INFO:teuthology.orchestra.run.smithi044.stdout:Total 16 MB/s | 48 MB 00:03 2024-04-20T01:38:31.622 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-20T01:38:31.646 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-20T01:38:31.646 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-20T01:38:31.945 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-20T01:38:31.954 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-20T01:38:32.082 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-20T01:38:32.082 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 1/25 2024-04-20T01:38:32.082 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 2/25 2024-04-20T01:38:32.082 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 3/25 2024-04-20T01:38:32.082 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 4/25 2024-04-20T01:38:32.082 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 5/25 2024-04-20T01:38:32.082 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_6 6/25 2024-04-20T01:38:32.083 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 7/25 2024-04-20T01:38:32.083 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1252.g15eecde2.el8. 8/25 2024-04-20T01:38:32.083 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:18.2.2-1252.g15eecde2.el8.x8 9/25 2024-04-20T01:38:32.083 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 10/25 2024-04-20T01:38:32.083 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 11/25 2024-04-20T01:38:32.083 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 12/25 2024-04-20T01:38:32.083 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 13/25 2024-04-20T01:38:32.083 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2024-04-20T01:38:32.083 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2024-04-20T01:38:32.083 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2024-04-20T01:38:32.083 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2024-04-20T01:38:32.083 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-04-20T01:38:32.083 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 19/25 2024-04-20T01:38:32.083 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 20/25 2024-04-20T01:38:32.083 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 21/25 2024-04-20T01:38:32.083 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 22/25 2024-04-20T01:38:32.084 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-04-20T01:38:32.084 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 24/25 2024-04-20T01:38:32.460 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-20T01:38:32.601 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 1/6 2024-04-20T01:38:32.677 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 1/6 2024-04-20T01:38:32.679 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 5/6 2024-04-20T01:38:32.903 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : jq-1.6-7.el8.x86_64 2/6 2024-04-20T01:38:32.907 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 6/6 2024-04-20T01:38:33.087 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 3/6 2024-04-20T01:38:33.191 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 6/6 2024-04-20T01:38:33.192 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 1/6 2024-04-20T01:38:33.192 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 2/6 2024-04-20T01:38:33.192 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2024-04-20T01:38:33.192 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/6 2024-04-20T01:38:33.192 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : jq-1.6-7.el8.x86_64 5/6 2024-04-20T01:38:33.240 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2024-04-20T01:38:33.409 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 5/6 2024-04-20T01:38:33.604 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 5/6 2024-04-20T01:38:33.872 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 6/6 2024-04-20T01:38:34.136 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 6/6 2024-04-20T01:38:34.136 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 1/6 2024-04-20T01:38:34.136 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 2/6 2024-04-20T01:38:34.136 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2024-04-20T01:38:34.136 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/6 2024-04-20T01:38:34.136 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : jq-1.6-7.el8.x86_64 5/6 2024-04-20T01:38:38.394 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 5/6 2024-04-20T01:38:38.616 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 6/6 2024-04-20T01:38:38.902 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 6/6 2024-04-20T01:38:38.902 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 1/6 2024-04-20T01:38:38.902 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 2/6 2024-04-20T01:38:38.903 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2024-04-20T01:38:38.903 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/6 2024-04-20T01:38:38.903 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : jq-1.6-7.el8.x86_64 5/6 2024-04-20T01:38:45.466 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-04-20T01:38:45.466 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2024-04-20T01:38:46.037 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:38:46.037 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-04-20T01:38:46.037 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:46.037 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:46.037 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-20T01:38:46.037 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:46.037 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:46.037 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:46.037 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:46.037 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-04-20T01:38:46.038 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:46.038 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.2-3.el8.x86_64 2024-04-20T01:38:46.038 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-04-20T01:38:46.038 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:46.038 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-04-20T01:38:46.038 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:46.038 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.3.1-3.el8.x86_64 2024-04-20T01:38:46.038 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-04-20T01:38:46.038 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:46.038 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:46.038 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:46.038 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-04-20T01:38:46.039 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:46.039 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:46.039 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:46.039 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.13.0-2.el8.x86_64 2024-04-20T01:38:46.039 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:38:46.039 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:38:46.259 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-test 2024-04-20T01:38:46.613 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:38:48.512 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 6/6 2024-04-20T01:38:48.512 INFO:teuthology.orchestra.run.smithi165.stdout:Installed products updated. 2024-04-20T01:38:48.595 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 6/6 2024-04-20T01:38:48.595 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2024-04-20T01:38:48.697 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-04-20T01:38:49.050 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:38:49.050 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-20T01:38:49.050 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:49.050 INFO:teuthology.orchestra.run.smithi165.stdout: jq-1.6-7.el8.x86_64 2024-04-20T01:38:49.050 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:49.050 INFO:teuthology.orchestra.run.smithi165.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-04-20T01:38:49.050 INFO:teuthology.orchestra.run.smithi165.stdout: socat-1.7.4.1-1.el8.x86_64 2024-04-20T01:38:49.051 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-04-20T01:38:49.051 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:38:49.051 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:38:49.142 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:38:49.143 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-04-20T01:38:49.143 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:49.143 INFO:teuthology.orchestra.run.smithi121.stdout: jq-1.6-7.el8.x86_64 2024-04-20T01:38:49.143 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:49.143 INFO:teuthology.orchestra.run.smithi121.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-04-20T01:38:49.143 INFO:teuthology.orchestra.run.smithi121.stdout: socat-1.7.4.1-1.el8.x86_64 2024-04-20T01:38:49.143 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-04-20T01:38:49.143 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:38:49.143 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:38:49.223 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph 2024-04-20T01:38:49.318 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph 2024-04-20T01:38:49.336 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 24 kB/s | 4.1 kB 00:00 2024-04-20T01:38:49.563 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:38:49.657 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:38:51.395 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 33 kB/s | 4.5 kB 00:00 2024-04-20T01:38:51.492 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 30 kB/s | 4.5 kB 00:00 2024-04-20T01:38:51.692 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:38:51.693 INFO:teuthology.orchestra.run.smithi123.stdout:============================================================================================ 2024-04-20T01:38:51.693 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-04-20T01:38:51.694 INFO:teuthology.orchestra.run.smithi123.stdout:============================================================================================ 2024-04-20T01:38:51.694 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-20T01:38:51.694 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 47 M 2024-04-20T01:38:51.694 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-04-20T01:38:51.694 INFO:teuthology.orchestra.run.smithi123.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-04-20T01:38:51.694 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 142 k 2024-04-20T01:38:51.694 INFO:teuthology.orchestra.run.smithi123.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-04-20T01:38:51.694 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-04-20T01:38:51.694 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-04-20T01:38:51.694 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:38:51.694 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-20T01:38:51.694 INFO:teuthology.orchestra.run.smithi123.stdout:============================================================================================ 2024-04-20T01:38:51.694 INFO:teuthology.orchestra.run.smithi123.stdout:Install 6 Packages 2024-04-20T01:38:51.695 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:38:51.695 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 48 M 2024-04-20T01:38:51.695 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 213 M 2024-04-20T01:38:51.695 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-20T01:38:51.766 INFO:teuthology.orchestra.run.smithi123.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 989 kB/s | 70 kB 00:00 2024-04-20T01:38:51.948 INFO:teuthology.orchestra.run.smithi123.stdout:(2/6): socat-1.7.4.1-1.el8.x86_64.rpm 1.8 MB/s | 323 kB 00:00 2024-04-20T01:38:52.006 INFO:teuthology.orchestra.run.smithi123.stdout:(3/6): jq-1.6-7.el8.x86_64.rpm 3.4 MB/s | 202 kB 00:00 2024-04-20T01:38:52.039 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 6/6 2024-04-20T01:38:52.039 INFO:teuthology.orchestra.run.smithi044.stdout:Installed products updated. 2024-04-20T01:38:52.077 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 30 kB/s | 4.1 kB 00:00 2024-04-20T01:38:52.140 INFO:teuthology.orchestra.run.smithi123.stdout:(4/6): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-04-20T01:38:52.165 INFO:teuthology.orchestra.run.smithi123.stdout:(5/6): libcephsqlite-18.2.2-1252.g15eecde2.el8. 302 kB/s | 142 kB 00:00 2024-04-20T01:38:52.197 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 21 kB/s | 4.1 kB 00:00 2024-04-20T01:38:52.645 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:38:52.646 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-20T01:38:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: jq-1.6-7.el8.x86_64 2024-04-20T01:38:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:38:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-04-20T01:38:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: socat-1.7.4.1-1.el8.x86_64 2024-04-20T01:38:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-04-20T01:38:52.646 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:38:52.646 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:38:52.823 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph 2024-04-20T01:38:53.211 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:38:54.358 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:38:54.361 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================================= 2024-04-20T01:38:54.361 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-20T01:38:54.361 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================================= 2024-04-20T01:38:54.361 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-20T01:38:54.361 INFO:teuthology.orchestra.run.smithi165.stdout: ceph x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 6.4 k 2024-04-20T01:38:54.361 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-20T01:38:54.361 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 2.3 M 2024-04-20T01:38:54.361 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 1.6 M 2024-04-20T01:38:54.361 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 246 k 2024-04-20T01:38:54.362 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 4.8 M 2024-04-20T01:38:54.362 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 18 M 2024-04-20T01:38:54.362 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-04-20T01:38:54.362 INFO:teuthology.orchestra.run.smithi165.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-04-20T01:38:54.362 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-04-20T01:38:54.362 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-04-20T01:38:54.362 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-04-20T01:38:54.362 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-04-20T01:38:54.362 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-04-20T01:38:54.362 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-04-20T01:38:54.362 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-04-20T01:38:54.362 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-04-20T01:38:54.362 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-04-20T01:38:54.363 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-04-20T01:38:54.363 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-04-20T01:38:54.363 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-04-20T01:38:54.363 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-04-20T01:38:54.363 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-04-20T01:38:54.363 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-04-20T01:38:54.363 INFO:teuthology.orchestra.run.smithi165.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-04-20T01:38:54.363 INFO:teuthology.orchestra.run.smithi165.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-04-20T01:38:54.363 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-04-20T01:38:54.363 INFO:teuthology.orchestra.run.smithi165.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-04-20T01:38:54.363 INFO:teuthology.orchestra.run.smithi165.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-04-20T01:38:54.363 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-04-20T01:38:54.363 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-04-20T01:38:54.363 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-04-20T01:38:54.364 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-04-20T01:38:54.364 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:38:54.364 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-20T01:38:54.364 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================================= 2024-04-20T01:38:54.364 INFO:teuthology.orchestra.run.smithi165.stdout:Install 32 Packages 2024-04-20T01:38:54.364 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:38:54.364 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 31 M 2024-04-20T01:38:54.365 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 124 M 2024-04-20T01:38:54.365 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-20T01:38:54.550 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:38:54.554 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================= 2024-04-20T01:38:54.554 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-04-20T01:38:54.554 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================= 2024-04-20T01:38:54.554 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-04-20T01:38:54.554 INFO:teuthology.orchestra.run.smithi121.stdout: ceph x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 6.4 k 2024-04-20T01:38:54.554 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-04-20T01:38:54.554 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 2.3 M 2024-04-20T01:38:54.554 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 1.6 M 2024-04-20T01:38:54.554 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 246 k 2024-04-20T01:38:54.554 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 4.8 M 2024-04-20T01:38:54.554 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 18 M 2024-04-20T01:38:54.554 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-04-20T01:38:54.555 INFO:teuthology.orchestra.run.smithi121.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-04-20T01:38:54.555 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-04-20T01:38:54.555 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-04-20T01:38:54.555 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-04-20T01:38:54.555 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-04-20T01:38:54.555 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-04-20T01:38:54.555 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-04-20T01:38:54.555 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-04-20T01:38:54.555 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-04-20T01:38:54.555 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-04-20T01:38:54.555 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-04-20T01:38:54.555 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-04-20T01:38:54.555 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-04-20T01:38:54.555 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-04-20T01:38:54.555 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-04-20T01:38:54.556 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-04-20T01:38:54.556 INFO:teuthology.orchestra.run.smithi121.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-04-20T01:38:54.556 INFO:teuthology.orchestra.run.smithi121.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-04-20T01:38:54.556 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-04-20T01:38:54.556 INFO:teuthology.orchestra.run.smithi121.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-04-20T01:38:54.556 INFO:teuthology.orchestra.run.smithi121.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-04-20T01:38:54.556 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-04-20T01:38:54.556 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-04-20T01:38:54.556 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-04-20T01:38:54.556 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-04-20T01:38:54.556 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:38:54.556 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-20T01:38:54.556 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================= 2024-04-20T01:38:54.556 INFO:teuthology.orchestra.run.smithi121.stdout:Install 32 Packages 2024-04-20T01:38:54.557 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:38:54.557 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 31 M 2024-04-20T01:38:54.557 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 124 M 2024-04-20T01:38:54.557 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-20T01:38:54.943 INFO:teuthology.orchestra.run.smithi165.stdout:(1/32): ceph-18.2.2-1252.g15eecde2.el8.x86_64.r 23 kB/s | 6.4 kB 00:00 2024-04-20T01:38:55.151 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 34 kB/s | 4.1 kB 00:00 2024-04-20T01:38:55.193 INFO:teuthology.orchestra.run.smithi165.stdout:(2/32): ceph-mds-18.2.2-1252.g15eecde2.el8.x86_ 4.3 MB/s | 2.3 MB 00:00 2024-04-20T01:38:55.508 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-04-20T01:38:55.901 INFO:teuthology.orchestra.run.smithi165.stdout:(3/32): ceph-mgr-18.2.2-1252.g15eecde2.el8.x86_ 1.3 MB/s | 1.6 MB 00:01 2024-04-20T01:38:56.068 INFO:teuthology.orchestra.run.smithi165.stdout:(4/32): ceph-mgr-modules-core-18.2.2-1252.g15ee 1.4 MB/s | 246 kB 00:00 2024-04-20T01:38:56.093 INFO:teuthology.orchestra.run.smithi121.stdout:(1/32): ceph-18.2.2-1252.g15eecde2.el8.x86_64.r 25 kB/s | 6.4 kB 00:00 2024-04-20T01:38:56.503 INFO:teuthology.orchestra.run.smithi165.stdout:(5/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 100 kB/s | 44 kB 00:00 2024-04-20T01:38:56.686 INFO:teuthology.orchestra.run.smithi165.stdout:(6/32): ceph-osd-18.2.2-1252.g15eecde2.el8.x86_ 12 MB/s | 18 MB 00:01 2024-04-20T01:38:56.794 INFO:teuthology.orchestra.run.smithi165.stdout:(7/32): python3-beautifulsoup4-4.6.3-2.el8.1.no 637 kB/s | 185 kB 00:00 2024-04-20T01:38:57.011 INFO:teuthology.orchestra.run.smithi165.stdout:(8/32): python3-cherrypy-18.4.0-1.el8.noarch.rp 1.7 MB/s | 384 kB 00:00 2024-04-20T01:38:57.128 INFO:teuthology.orchestra.run.smithi165.stdout:(9/32): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2024-04-20T01:38:57.169 INFO:teuthology.orchestra.run.smithi121.stdout:(2/32): ceph-mds-18.2.2-1252.g15eecde2.el8.x86_ 1.7 MB/s | 2.3 MB 00:01 2024-04-20T01:38:57.236 INFO:teuthology.orchestra.run.smithi165.stdout:(10/32): ceph-mon-18.2.2-1252.g15eecde2.el8.x86 2.1 MB/s | 4.8 MB 00:02 2024-04-20T01:38:57.268 INFO:teuthology.orchestra.run.smithi121.stdout:(3/32): ceph-mgr-18.2.2-1252.g15eecde2.el8.x86_ 1.1 MB/s | 1.6 MB 00:01 2024-04-20T01:38:57.278 INFO:teuthology.orchestra.run.smithi165.stdout:(11/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 291 kB/s | 173 kB 00:00 2024-04-20T01:38:57.304 INFO:teuthology.orchestra.run.smithi165.stdout:(12/32): python3-jaraco-functools-2.0-4.el8.noa 100 kB/s | 18 kB 00:00 2024-04-20T01:38:57.396 INFO:teuthology.orchestra.run.smithi165.stdout:(13/32): python3-more-itertools-7.2.0-3.el8.noa 502 kB/s | 59 kB 00:00 2024-04-20T01:38:57.446 INFO:teuthology.orchestra.run.smithi165.stdout:(14/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.9 MB/s | 283 kB 00:00 2024-04-20T01:38:57.460 INFO:teuthology.orchestra.run.smithi121.stdout:(4/32): ceph-mgr-modules-core-18.2.2-1252.g15ee 1.3 MB/s | 246 kB 00:00 2024-04-20T01:38:57.538 INFO:teuthology.orchestra.run.smithi165.stdout:(15/32): python3-portend-2.6-1.el8.noarch.rpm 110 kB/s | 16 kB 00:00 2024-04-20T01:38:57.563 INFO:teuthology.orchestra.run.smithi165.stdout:(16/32): python3-simplegeneric-0.8.1-17.el8.noa 160 kB/s | 19 kB 00:00 2024-04-20T01:38:57.588 INFO:teuthology.orchestra.run.smithi165.stdout:(17/32): python3-logutils-0.3.5-11.el8.noarch.r 138 kB/s | 49 kB 00:00 2024-04-20T01:38:57.630 INFO:teuthology.orchestra.run.smithi165.stdout:(18/32): python3-singledispatch-3.4.0.3-18.el8. 260 kB/s | 24 kB 00:00 2024-04-20T01:38:57.655 INFO:teuthology.orchestra.run.smithi165.stdout:(19/32): python3-tempora-1.14.1-5.el8.noarch.rp 317 kB/s | 29 kB 00:00 2024-04-20T01:38:57.705 INFO:teuthology.orchestra.run.smithi121.stdout:(5/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 178 kB/s | 44 kB 00:00 2024-04-20T01:38:57.714 INFO:teuthology.orchestra.run.smithi165.stdout:(20/32): python3-trustme-0.6.0-4.el8.noarch.rpm 219 kB/s | 27 kB 00:00 2024-04-20T01:38:57.756 INFO:teuthology.orchestra.run.smithi165.stdout:(21/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.4 MB/s | 251 kB 00:00 2024-04-20T01:38:57.802 INFO:teuthology.orchestra.run.smithi121.stdout:(6/32): python3-beautifulsoup4-4.6.3-2.el8.1.no 1.9 MB/s | 185 kB 00:00 2024-04-20T01:38:57.806 INFO:teuthology.orchestra.run.smithi165.stdout:(22/32): python3-waitress-1.4.3-1.el8.noarch.rp 1.4 MB/s | 255 kB 00:00 2024-04-20T01:38:57.852 INFO:teuthology.orchestra.run.smithi121.stdout:(7/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 3.4 MB/s | 173 kB 00:00 2024-04-20T01:38:57.881 INFO:teuthology.orchestra.run.smithi165.stdout:(23/32): python3-zc-lockfile-2.0-2.el8.noarch.r 183 kB/s | 23 kB 00:00 2024-04-20T01:38:57.919 INFO:teuthology.orchestra.run.smithi121.stdout:(8/32): python3-cherrypy-18.4.0-1.el8.noarch.rp 5.6 MB/s | 384 kB 00:00 2024-04-20T01:38:57.949 INFO:teuthology.orchestra.run.smithi165.stdout:(24/32): python3-werkzeug-0.12.2-4.el8.noarch.r 3.1 MB/s | 457 kB 00:00 2024-04-20T01:38:57.990 INFO:teuthology.orchestra.run.smithi165.stdout:(25/32): python3-webtest-2.0.33-1.el8.noarch.rp 309 kB/s | 85 kB 00:00 2024-04-20T01:38:58.003 INFO:teuthology.orchestra.run.smithi121.stdout:(9/32): python3-jaraco-6.2-6.el8.noarch.rpm 133 kB/s | 11 kB 00:00 2024-04-20T01:38:58.016 INFO:teuthology.orchestra.run.smithi165.stdout:(26/32): python3-markupsafe-0.23-19.el8.x86_64. 289 kB/s | 39 kB 00:00 2024-04-20T01:38:58.036 INFO:teuthology.orchestra.run.smithi121.stdout:(10/32): python3-jaraco-functools-2.0-4.el8.noa 530 kB/s | 18 kB 00:00 2024-04-20T01:38:58.070 INFO:teuthology.orchestra.run.smithi121.stdout:(11/32): python3-logutils-0.3.5-11.el8.noarch.r 1.4 MB/s | 49 kB 00:00 2024-04-20T01:38:58.099 INFO:teuthology.orchestra.run.smithi165.stdout:(27/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 688 kB/s | 103 kB 00:00 2024-04-20T01:38:58.103 INFO:teuthology.orchestra.run.smithi121.stdout:(12/32): python3-more-itertools-7.2.0-3.el8.noa 1.7 MB/s | 59 kB 00:00 2024-04-20T01:38:58.154 INFO:teuthology.orchestra.run.smithi121.stdout:(13/32): python3-pecan-1.3.2-9.el8.noarch.rpm 5.6 MB/s | 283 kB 00:00 2024-04-20T01:38:58.180 INFO:teuthology.orchestra.run.smithi165.stdout:(28/32): python3-pytz-2017.2-11.el8.noarch.rpm 343 kB/s | 54 kB 00:00 2024-04-20T01:38:58.196 INFO:teuthology.orchestra.run.smithi121.stdout:(14/32): python3-portend-2.6-1.el8.noarch.rpm 374 kB/s | 16 kB 00:00 2024-04-20T01:38:58.242 INFO:teuthology.orchestra.run.smithi165.stdout:(29/32): python3-pycparser-2.14-14.el8.noarch.r 770 kB/s | 109 kB 00:00 2024-04-20T01:38:58.271 INFO:teuthology.orchestra.run.smithi121.stdout:(15/32): python3-simplegeneric-0.8.1-17.el8.noa 250 kB/s | 19 kB 00:00 2024-04-20T01:38:58.285 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:38:58.289 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================================= 2024-04-20T01:38:58.289 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-20T01:38:58.289 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================================= 2024-04-20T01:38:58.289 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-20T01:38:58.289 INFO:teuthology.orchestra.run.smithi044.stdout: ceph x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 6.4 k 2024-04-20T01:38:58.289 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-20T01:38:58.290 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 2.3 M 2024-04-20T01:38:58.290 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 1.6 M 2024-04-20T01:38:58.290 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 246 k 2024-04-20T01:38:58.290 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 4.8 M 2024-04-20T01:38:58.290 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 18 M 2024-04-20T01:38:58.290 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-04-20T01:38:58.290 INFO:teuthology.orchestra.run.smithi044.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-04-20T01:38:58.290 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-04-20T01:38:58.290 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-04-20T01:38:58.290 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-04-20T01:38:58.291 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-04-20T01:38:58.291 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-04-20T01:38:58.291 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-04-20T01:38:58.291 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-04-20T01:38:58.291 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-04-20T01:38:58.291 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-04-20T01:38:58.291 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-04-20T01:38:58.291 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-04-20T01:38:58.291 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-04-20T01:38:58.291 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-04-20T01:38:58.291 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-04-20T01:38:58.292 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-04-20T01:38:58.292 INFO:teuthology.orchestra.run.smithi044.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-04-20T01:38:58.292 INFO:teuthology.orchestra.run.smithi044.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-04-20T01:38:58.292 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-04-20T01:38:58.292 INFO:teuthology.orchestra.run.smithi044.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-04-20T01:38:58.292 INFO:teuthology.orchestra.run.smithi044.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-04-20T01:38:58.292 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-04-20T01:38:58.292 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-04-20T01:38:58.292 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-04-20T01:38:58.292 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-04-20T01:38:58.292 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:38:58.292 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-20T01:38:58.292 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================================= 2024-04-20T01:38:58.293 INFO:teuthology.orchestra.run.smithi044.stdout:Install 32 Packages 2024-04-20T01:38:58.293 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:38:58.293 INFO:teuthology.orchestra.run.smithi165.stdout:(30/32): python3-mako-1.0.6-14.el8.noarch.rpm 522 kB/s | 157 kB 00:00 2024-04-20T01:38:58.294 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 31 M 2024-04-20T01:38:58.294 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 124 M 2024-04-20T01:38:58.294 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-20T01:38:58.313 INFO:teuthology.orchestra.run.smithi121.stdout:(16/32): python3-singledispatch-3.4.0.3-18.el8. 575 kB/s | 24 kB 00:00 2024-04-20T01:38:58.325 INFO:teuthology.orchestra.run.smithi165.stdout:(31/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.6 MB/s | 238 kB 00:00 2024-04-20T01:38:58.347 INFO:teuthology.orchestra.run.smithi121.stdout:(17/32): python3-tempora-1.14.1-5.el8.noarch.rp 877 kB/s | 29 kB 00:00 2024-04-20T01:38:58.380 INFO:teuthology.orchestra.run.smithi121.stdout:(18/32): python3-trustme-0.6.0-4.el8.noarch.rpm 826 kB/s | 27 kB 00:00 2024-04-20T01:38:58.422 INFO:teuthology.orchestra.run.smithi121.stdout:(19/32): python3-waitress-1.4.3-1.el8.noarch.rp 6.0 MB/s | 255 kB 00:00 2024-04-20T01:38:58.473 INFO:teuthology.orchestra.run.smithi121.stdout:(20/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 4.9 MB/s | 251 kB 00:00 2024-04-20T01:38:58.475 INFO:teuthology.orchestra.run.smithi165.stdout:(32/32): python3-cryptography-3.2.1-7.el8_9.x86 2.3 MB/s | 559 kB 00:00 2024-04-20T01:38:58.477 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:38:58.477 INFO:teuthology.orchestra.run.smithi165.stdout:Total 7.5 MB/s | 31 MB 00:04 2024-04-20T01:38:58.548 INFO:teuthology.orchestra.run.smithi121.stdout:(21/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.1 MB/s | 85 kB 00:00 2024-04-20T01:38:58.574 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-20T01:38:58.581 INFO:teuthology.orchestra.run.smithi121.stdout:(22/32): python3-zc-lockfile-2.0-2.el8.noarch.r 689 kB/s | 23 kB 00:00 2024-04-20T01:38:58.611 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-20T01:38:58.612 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-20T01:38:58.715 INFO:teuthology.orchestra.run.smithi121.stdout:(23/32): python3-werkzeug-0.12.2-4.el8.noarch.r 3.3 MB/s | 457 kB 00:00 2024-04-20T01:38:58.838 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-20T01:38:58.847 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-20T01:38:58.858 INFO:teuthology.orchestra.run.smithi121.stdout:(24/32): python3-markupsafe-0.23-19.el8.x86_64. 274 kB/s | 39 kB 00:00 2024-04-20T01:38:58.940 INFO:teuthology.orchestra.run.smithi044.stdout:(1/32): ceph-18.2.2-1252.g15eecde2.el8.x86_64.r 35 kB/s | 6.4 kB 00:00 2024-04-20T01:38:58.958 INFO:teuthology.orchestra.run.smithi121.stdout:(25/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 1.0 MB/s | 103 kB 00:00 2024-04-20T01:38:59.083 INFO:teuthology.orchestra.run.smithi121.stdout:(26/32): python3-mako-1.0.6-14.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2024-04-20T01:38:59.150 INFO:teuthology.orchestra.run.smithi121.stdout:(27/32): ceph-mon-18.2.2-1252.g15eecde2.el8.x86 1.6 MB/s | 4.8 MB 00:03 2024-04-20T01:38:59.175 INFO:teuthology.orchestra.run.smithi121.stdout:(28/32): python3-pytz-2017.2-11.el8.noarch.rpm 591 kB/s | 54 kB 00:00 2024-04-20T01:38:59.309 INFO:teuthology.orchestra.run.smithi121.stdout:(29/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.8 MB/s | 238 kB 00:00 2024-04-20T01:38:59.367 INFO:teuthology.orchestra.run.smithi121.stdout:(30/32): python3-pycparser-2.14-14.el8.noarch.r 502 kB/s | 109 kB 00:00 2024-04-20T01:38:59.543 INFO:teuthology.orchestra.run.smithi121.stdout:(31/32): ceph-osd-18.2.2-1252.g15eecde2.el8.x86 7.8 MB/s | 18 MB 00:02 2024-04-20T01:38:59.557 INFO:teuthology.orchestra.run.smithi044.stdout:(2/32): ceph-mgr-18.2.2-1252.g15eecde2.el8.x86_ 1.9 MB/s | 1.6 MB 00:00 2024-04-20T01:38:59.576 INFO:teuthology.orchestra.run.smithi121.stdout:(32/32): python3-cryptography-3.2.1-7.el8_9.x86 2.0 MB/s | 559 kB 00:00 2024-04-20T01:38:59.579 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:38:59.580 INFO:teuthology.orchestra.run.smithi121.stdout:Total 6.1 MB/s | 31 MB 00:05 2024-04-20T01:38:59.676 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-20T01:38:59.713 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-20T01:38:59.713 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-20T01:38:59.723 INFO:teuthology.orchestra.run.smithi044.stdout:(3/32): ceph-mds-18.2.2-1252.g15eecde2.el8.x86_ 2.3 MB/s | 2.3 MB 00:00 2024-04-20T01:38:59.861 INFO:teuthology.orchestra.run.smithi044.stdout:(4/32): ceph-mgr-modules-core-18.2.2-1252.g15ee 1.7 MB/s | 246 kB 00:00 2024-04-20T01:38:59.938 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-20T01:38:59.947 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-20T01:39:00.009 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-20T01:39:00.082 INFO:teuthology.orchestra.run.smithi044.stdout:(5/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 197 kB/s | 44 kB 00:00 2024-04-20T01:39:00.224 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-04-20T01:39:00.249 INFO:teuthology.orchestra.run.smithi044.stdout:(6/32): python3-beautifulsoup4-4.6.3-2.el8.1.no 1.1 MB/s | 185 kB 00:00 2024-04-20T01:39:00.333 INFO:teuthology.orchestra.run.smithi044.stdout:(7/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 2.0 MB/s | 173 kB 00:00 2024-04-20T01:39:00.431 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-04-20T01:39:00.467 INFO:teuthology.orchestra.run.smithi044.stdout:(8/32): python3-cherrypy-18.4.0-1.el8.noarch.rp 2.8 MB/s | 384 kB 00:00 2024-04-20T01:39:00.550 INFO:teuthology.orchestra.run.smithi044.stdout:(9/32): python3-jaraco-6.2-6.el8.noarch.rpm 133 kB/s | 11 kB 00:00 2024-04-20T01:39:00.651 INFO:teuthology.orchestra.run.smithi044.stdout:(10/32): python3-jaraco-functools-2.0-4.el8.noa 176 kB/s | 18 kB 00:00 2024-04-20T01:39:00.663 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 3/32 2024-04-20T01:39:00.743 INFO:teuthology.orchestra.run.smithi044.stdout:(11/32): python3-logutils-0.3.5-11.el8.noarch.r 531 kB/s | 49 kB 00:00 2024-04-20T01:39:00.835 INFO:teuthology.orchestra.run.smithi044.stdout:(12/32): python3-more-itertools-7.2.0-3.el8.noa 642 kB/s | 59 kB 00:00 2024-04-20T01:39:00.927 INFO:teuthology.orchestra.run.smithi044.stdout:(13/32): python3-pecan-1.3.2-9.el8.noarch.rpm 3.0 MB/s | 283 kB 00:00 2024-04-20T01:39:01.002 INFO:teuthology.orchestra.run.smithi044.stdout:(14/32): ceph-mon-18.2.2-1252.g15eecde2.el8.x86 2.3 MB/s | 4.8 MB 00:02 2024-04-20T01:39:01.016 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 4/32 2024-04-20T01:39:01.019 INFO:teuthology.orchestra.run.smithi044.stdout:(15/32): python3-portend-2.6-1.el8.noarch.rpm 170 kB/s | 16 kB 00:00 2024-04-20T01:39:01.086 INFO:teuthology.orchestra.run.smithi044.stdout:(16/32): python3-simplegeneric-0.8.1-17.el8.noa 224 kB/s | 19 kB 00:00 2024-04-20T01:39:01.178 INFO:teuthology.orchestra.run.smithi044.stdout:(17/32): python3-tempora-1.14.1-5.el8.noarch.rp 319 kB/s | 29 kB 00:00 2024-04-20T01:39:01.262 INFO:teuthology.orchestra.run.smithi044.stdout:(18/32): python3-singledispatch-3.4.0.3-18.el8. 99 kB/s | 24 kB 00:00 2024-04-20T01:39:01.279 INFO:teuthology.orchestra.run.smithi044.stdout:(19/32): python3-trustme-0.6.0-4.el8.noarch.rpm 273 kB/s | 27 kB 00:00 2024-04-20T01:39:01.305 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 5/32 2024-04-20T01:39:01.371 INFO:teuthology.orchestra.run.smithi044.stdout:(20/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.7 MB/s | 251 kB 00:00 2024-04-20T01:39:01.446 INFO:teuthology.orchestra.run.smithi044.stdout:(21/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.1 MB/s | 85 kB 00:00 2024-04-20T01:39:01.471 INFO:teuthology.orchestra.run.smithi044.stdout:(22/32): python3-waitress-1.4.3-1.el8.noarch.rp 1.2 MB/s | 255 kB 00:00 2024-04-20T01:39:01.522 INFO:teuthology.orchestra.run.smithi044.stdout:(23/32): python3-zc-lockfile-2.0-2.el8.noarch.r 306 kB/s | 23 kB 00:00 2024-04-20T01:39:01.647 INFO:teuthology.orchestra.run.smithi044.stdout:(24/32): python3-pycparser-2.14-14.el8.noarch.r 622 kB/s | 109 kB 00:00 2024-04-20T01:39:01.681 INFO:teuthology.orchestra.run.smithi044.stdout:(25/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2024-04-20T01:39:01.914 INFO:teuthology.orchestra.run.smithi044.stdout:(26/32): python3-cryptography-3.2.1-7.el8_9.x86 2.0 MB/s | 559 kB 00:00 2024-04-20T01:39:02.055 INFO:teuthology.orchestra.run.smithi044.stdout:(27/32): python3-werkzeug-0.12.2-4.el8.noarch.r 1.2 MB/s | 457 kB 00:00 2024-04-20T01:39:02.081 INFO:teuthology.orchestra.run.smithi044.stdout:(28/32): python3-markupsafe-0.23-19.el8.x86_64. 233 kB/s | 39 kB 00:00 2024-04-20T01:39:02.132 INFO:teuthology.orchestra.run.smithi044.stdout:(29/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 1.3 MB/s | 103 kB 00:00 2024-04-20T01:39:02.190 INFO:teuthology.orchestra.run.smithi044.stdout:(30/32): python3-mako-1.0.6-14.el8.noarch.rpm 1.4 MB/s | 157 kB 00:00 2024-04-20T01:39:02.257 INFO:teuthology.orchestra.run.smithi044.stdout:(31/32): python3-pytz-2017.2-11.el8.noarch.rpm 433 kB/s | 54 kB 00:00 2024-04-20T01:39:02.450 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 6/32 2024-04-20T01:39:02.583 INFO:teuthology.orchestra.run.smithi044.stdout:(32/32): ceph-osd-18.2.2-1252.g15eecde2.el8.x86 6.1 MB/s | 18 MB 00:03 2024-04-20T01:39:02.585 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:39:02.585 INFO:teuthology.orchestra.run.smithi044.stdout:Total 7.2 MB/s | 31 MB 00:04 2024-04-20T01:39:02.688 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-20T01:39:02.730 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-20T01:39:02.730 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-20T01:39:02.983 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-20T01:39:02.993 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-20T01:39:03.533 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-20T01:39:03.630 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 7/32 2024-04-20T01:39:03.808 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-04-20T01:39:03.834 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 8/32 2024-04-20T01:39:04.032 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 9/32 2024-04-20T01:39:04.064 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-04-20T01:39:04.165 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-20T01:39:04.293 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 10/32 2024-04-20T01:39:04.312 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 3/32 2024-04-20T01:39:04.414 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-04-20T01:39:04.606 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-04-20T01:39:04.634 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 11/32 2024-04-20T01:39:04.709 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 4/32 2024-04-20T01:39:04.787 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 3/32 2024-04-20T01:39:04.852 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 12/32 2024-04-20T01:39:04.996 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 5/32 2024-04-20T01:39:05.003 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/32 2024-04-20T01:39:05.104 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 13/32 2024-04-20T01:39:05.201 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 6/32 2024-04-20T01:39:05.358 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 5/32 2024-04-20T01:39:05.360 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 14/32 2024-04-20T01:39:05.402 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 7/32 2024-04-20T01:39:05.506 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 15/32 2024-04-20T01:39:05.556 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 6/32 2024-04-20T01:39:05.658 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 8/32 2024-04-20T01:39:05.738 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 16/32 2024-04-20T01:39:05.791 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 7/32 2024-04-20T01:39:05.814 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 9/32 2024-04-20T01:39:05.930 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 17/32 2024-04-20T01:39:06.055 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 10/32 2024-04-20T01:39:06.083 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 18/32 2024-04-20T01:39:06.234 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 8/32 2024-04-20T01:39:06.272 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 19/32 2024-04-20T01:39:06.384 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 11/32 2024-04-20T01:39:06.455 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 20/32 2024-04-20T01:39:06.571 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 9/32 2024-04-20T01:39:06.635 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 12/32 2024-04-20T01:39:06.674 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-04-20T01:39:06.731 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 10/32 2024-04-20T01:39:06.877 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 13/32 2024-04-20T01:39:06.904 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-04-20T01:39:06.933 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 11/32 2024-04-20T01:39:07.135 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 14/32 2024-04-20T01:39:07.156 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 12/32 2024-04-20T01:39:07.164 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-04-20T01:39:07.347 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 15/32 2024-04-20T01:39:07.427 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 24/32 2024-04-20T01:39:07.444 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 13/32 2024-04-20T01:39:07.586 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 16/32 2024-04-20T01:39:07.698 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 25/32 2024-04-20T01:39:07.771 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 17/32 2024-04-20T01:39:07.999 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 18/32 2024-04-20T01:39:08.059 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-04-20T01:39:08.239 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 19/32 2024-04-20T01:39:08.455 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 20/32 2024-04-20T01:39:08.464 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1252.g15eecde2.el8. 27/32 2024-04-20T01:39:08.682 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-04-20T01:39:08.701 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 28/32 2024-04-20T01:39:08.914 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-04-20T01:39:09.173 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-04-20T01:39:09.435 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 24/32 2024-04-20T01:39:09.570 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 14/32 2024-04-20T01:39:09.735 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 15/32 2024-04-20T01:39:09.759 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 25/32 2024-04-20T01:39:09.910 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 16/32 2024-04-20T01:39:10.108 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 17/32 2024-04-20T01:39:10.126 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-04-20T01:39:10.253 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 18/32 2024-04-20T01:39:10.340 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 28/32 2024-04-20T01:39:10.498 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 19/32 2024-04-20T01:39:10.551 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 29/32 2024-04-20T01:39:10.553 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1252.g15eecde2.el8. 27/32 2024-04-20T01:39:10.718 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 20/32 2024-04-20T01:39:10.765 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 28/32 2024-04-20T01:39:10.961 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-04-20T01:39:11.030 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 29/32 2024-04-20T01:39:11.193 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 30/32 2024-04-20T01:39:11.226 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-04-20T01:39:11.407 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 30/32 2024-04-20T01:39:11.508 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-04-20T01:39:11.568 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 31/32 2024-04-20T01:39:11.586 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 31/32 2024-04-20T01:39:11.683 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 32/32 2024-04-20T01:39:11.761 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 24/32 2024-04-20T01:39:12.063 INFO:teuthology.orchestra.run.smithi123.stdout:(6/6): ceph-test-18.2.2-1252.g15eecde2.el8.x86_ 2.3 MB/s | 47 MB 00:20 2024-04-20T01:39:12.064 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:39:12.065 INFO:teuthology.orchestra.run.smithi123.stdout:Total 2.4 MB/s | 48 MB 00:20 2024-04-20T01:39:12.069 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 32/32 2024-04-20T01:39:12.069 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 1/32 2024-04-20T01:39:12.069 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 2/32 2024-04-20T01:39:12.069 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 3/32 2024-04-20T01:39:12.069 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 4/32 2024-04-20T01:39:12.069 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 5/32 2024-04-20T01:39:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1252.g15eecde2.el8. 6/32 2024-04-20T01:39:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/32 2024-04-20T01:39:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/32 2024-04-20T01:39:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/32 2024-04-20T01:39:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/32 2024-04-20T01:39:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/32 2024-04-20T01:39:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/32 2024-04-20T01:39:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/32 2024-04-20T01:39:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/32 2024-04-20T01:39:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/32 2024-04-20T01:39:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/32 2024-04-20T01:39:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/32 2024-04-20T01:39:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/32 2024-04-20T01:39:12.070 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/32 2024-04-20T01:39:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/32 2024-04-20T01:39:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 21/32 2024-04-20T01:39:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/32 2024-04-20T01:39:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/32 2024-04-20T01:39:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/32 2024-04-20T01:39:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 25/32 2024-04-20T01:39:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 26/32 2024-04-20T01:39:12.071 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 27/32 2024-04-20T01:39:12.072 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 28/32 2024-04-20T01:39:12.072 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 29/32 2024-04-20T01:39:12.072 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 30/32 2024-04-20T01:39:12.072 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 31/32 2024-04-20T01:39:12.081 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 25/32 2024-04-20T01:39:12.104 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-20T01:39:12.127 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-20T01:39:12.127 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-20T01:39:12.396 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-20T01:39:12.403 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 28/32 2024-04-20T01:39:12.405 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-20T01:39:12.462 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-04-20T01:39:12.656 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 29/32 2024-04-20T01:39:12.886 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1252.g15eecde2.el8. 27/32 2024-04-20T01:39:12.899 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-20T01:39:13.036 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 1/6 2024-04-20T01:39:13.091 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 28/32 2024-04-20T01:39:13.115 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 1/6 2024-04-20T01:39:13.134 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 29/32 2024-04-20T01:39:13.292 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 30/32 2024-04-20T01:39:13.339 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : jq-1.6-7.el8.x86_64 2/6 2024-04-20T01:39:13.507 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 30/32 2024-04-20T01:39:13.545 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 3/6 2024-04-20T01:39:13.709 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 31/32 2024-04-20T01:39:13.714 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2024-04-20T01:39:13.728 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 31/32 2024-04-20T01:39:13.858 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 32/32 2024-04-20T01:39:13.902 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 5/6 2024-04-20T01:39:14.279 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 32/32 2024-04-20T01:39:14.279 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 1/32 2024-04-20T01:39:14.279 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 2/32 2024-04-20T01:39:14.279 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 3/32 2024-04-20T01:39:14.279 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 4/32 2024-04-20T01:39:14.279 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 5/32 2024-04-20T01:39:14.279 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1252.g15eecde2.el8. 6/32 2024-04-20T01:39:14.279 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/32 2024-04-20T01:39:14.280 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/32 2024-04-20T01:39:14.280 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/32 2024-04-20T01:39:14.280 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/32 2024-04-20T01:39:14.280 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/32 2024-04-20T01:39:14.280 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/32 2024-04-20T01:39:14.280 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/32 2024-04-20T01:39:14.280 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/32 2024-04-20T01:39:14.280 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/32 2024-04-20T01:39:14.280 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/32 2024-04-20T01:39:14.280 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/32 2024-04-20T01:39:14.280 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/32 2024-04-20T01:39:14.280 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/32 2024-04-20T01:39:14.280 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/32 2024-04-20T01:39:14.280 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 21/32 2024-04-20T01:39:14.280 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/32 2024-04-20T01:39:14.281 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/32 2024-04-20T01:39:14.281 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/32 2024-04-20T01:39:14.281 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 25/32 2024-04-20T01:39:14.281 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 26/32 2024-04-20T01:39:14.281 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 27/32 2024-04-20T01:39:14.281 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 28/32 2024-04-20T01:39:14.281 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 29/32 2024-04-20T01:39:14.281 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 30/32 2024-04-20T01:39:14.281 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 31/32 2024-04-20T01:39:14.904 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 28/32 2024-04-20T01:39:15.140 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 29/32 2024-04-20T01:39:15.673 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 29/32 2024-04-20T01:39:15.837 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 30/32 2024-04-20T01:39:16.074 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 30/32 2024-04-20T01:39:16.254 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 31/32 2024-04-20T01:39:16.278 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 31/32 2024-04-20T01:39:16.420 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 32/32 2024-04-20T01:39:16.847 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 32/32 2024-04-20T01:39:16.847 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 1/32 2024-04-20T01:39:16.847 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 2/32 2024-04-20T01:39:16.847 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 3/32 2024-04-20T01:39:16.847 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 4/32 2024-04-20T01:39:16.848 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 5/32 2024-04-20T01:39:16.848 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1252.g15eecde2.el8. 6/32 2024-04-20T01:39:16.848 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/32 2024-04-20T01:39:16.848 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/32 2024-04-20T01:39:16.848 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/32 2024-04-20T01:39:16.848 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/32 2024-04-20T01:39:16.848 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/32 2024-04-20T01:39:16.848 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/32 2024-04-20T01:39:16.849 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/32 2024-04-20T01:39:16.849 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/32 2024-04-20T01:39:16.849 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/32 2024-04-20T01:39:16.849 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/32 2024-04-20T01:39:16.849 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/32 2024-04-20T01:39:16.849 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/32 2024-04-20T01:39:16.849 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/32 2024-04-20T01:39:16.849 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/32 2024-04-20T01:39:16.849 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 21/32 2024-04-20T01:39:16.850 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/32 2024-04-20T01:39:16.850 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/32 2024-04-20T01:39:16.850 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/32 2024-04-20T01:39:16.850 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 25/32 2024-04-20T01:39:16.850 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 26/32 2024-04-20T01:39:16.850 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 27/32 2024-04-20T01:39:16.851 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 28/32 2024-04-20T01:39:16.851 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 29/32 2024-04-20T01:39:16.851 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 30/32 2024-04-20T01:39:16.851 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 31/32 2024-04-20T01:39:18.428 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 5/6 2024-04-20T01:39:18.699 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 6/6 2024-04-20T01:39:18.965 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 6/6 2024-04-20T01:39:18.965 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 1/6 2024-04-20T01:39:18.965 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 2/6 2024-04-20T01:39:18.965 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2024-04-20T01:39:18.965 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/6 2024-04-20T01:39:18.965 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : jq-1.6-7.el8.x86_64 5/6 2024-04-20T01:39:26.401 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 32/32 2024-04-20T01:39:26.402 INFO:teuthology.orchestra.run.smithi165.stdout:Installed products updated. 2024-04-20T01:39:26.954 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:39:26.955 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-20T01:39:26.955 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:39:26.956 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:39:26.956 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:39:26.956 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:39:26.956 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:39:26.956 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:39:26.956 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-04-20T01:39:26.956 INFO:teuthology.orchestra.run.smithi165.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-04-20T01:39:26.956 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-04-20T01:39:26.956 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-04-20T01:39:26.956 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-04-20T01:39:26.956 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-04-20T01:39:26.956 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-6.2-6.el8.noarch 2024-04-20T01:39:26.956 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-04-20T01:39:26.956 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-04-20T01:39:26.957 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako-1.0.6-14.el8.noarch 2024-04-20T01:39:26.957 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-04-20T01:39:26.957 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-04-20T01:39:26.957 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-04-20T01:39:26.957 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend-2.6-1.el8.noarch 2024-04-20T01:39:26.957 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-04-20T01:39:26.957 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser-2.14-14.el8.noarch 2024-04-20T01:39:26.957 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz-2017.2-11.el8.noarch 2024-04-20T01:39:26.957 INFO:teuthology.orchestra.run.smithi165.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-04-20T01:39:26.957 INFO:teuthology.orchestra.run.smithi165.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-04-20T01:39:26.957 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-04-20T01:39:26.957 INFO:teuthology.orchestra.run.smithi165.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-04-20T01:39:26.957 INFO:teuthology.orchestra.run.smithi165.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-04-20T01:39:26.957 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-04-20T01:39:26.958 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-04-20T01:39:26.958 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-04-20T01:39:26.958 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-04-20T01:39:26.958 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:39:26.958 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:39:27.134 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-base 2024-04-20T01:39:27.487 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:39:27.547 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 32/32 2024-04-20T01:39:27.548 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2024-04-20T01:39:28.086 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:39:28.086 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-04-20T01:39:28.086 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:39:28.086 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:39:28.086 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:39:28.086 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:39:28.086 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:39:28.086 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:39:28.086 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-04-20T01:39:28.086 INFO:teuthology.orchestra.run.smithi121.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-04-20T01:39:28.087 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-04-20T01:39:28.087 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-04-20T01:39:28.087 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-04-20T01:39:28.087 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-04-20T01:39:28.087 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-6.2-6.el8.noarch 2024-04-20T01:39:28.087 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-04-20T01:39:28.087 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-04-20T01:39:28.087 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako-1.0.6-14.el8.noarch 2024-04-20T01:39:28.087 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-04-20T01:39:28.087 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-04-20T01:39:28.087 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-04-20T01:39:28.087 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend-2.6-1.el8.noarch 2024-04-20T01:39:28.087 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-04-20T01:39:28.087 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser-2.14-14.el8.noarch 2024-04-20T01:39:28.087 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz-2017.2-11.el8.noarch 2024-04-20T01:39:28.087 INFO:teuthology.orchestra.run.smithi121.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-04-20T01:39:28.088 INFO:teuthology.orchestra.run.smithi121.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-04-20T01:39:28.088 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-04-20T01:39:28.088 INFO:teuthology.orchestra.run.smithi121.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-04-20T01:39:28.088 INFO:teuthology.orchestra.run.smithi121.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-04-20T01:39:28.088 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-04-20T01:39:28.088 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-04-20T01:39:28.088 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-04-20T01:39:28.088 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-04-20T01:39:28.088 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:39:28.088 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:39:28.308 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-base 2024-04-20T01:39:28.658 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:39:29.273 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 32/32 2024-04-20T01:39:29.273 INFO:teuthology.orchestra.run.smithi044.stdout:Installed products updated. 2024-04-20T01:39:29.590 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 29 kB/s | 4.5 kB 00:00 2024-04-20T01:39:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:39:29.900 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-20T01:39:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:39:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:39:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:39:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:39:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:39:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:39:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-04-20T01:39:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-04-20T01:39:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-04-20T01:39:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-04-20T01:39:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-04-20T01:39:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-04-20T01:39:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-6.2-6.el8.noarch 2024-04-20T01:39:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-04-20T01:39:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-04-20T01:39:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako-1.0.6-14.el8.noarch 2024-04-20T01:39:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-04-20T01:39:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-04-20T01:39:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-04-20T01:39:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend-2.6-1.el8.noarch 2024-04-20T01:39:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-04-20T01:39:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser-2.14-14.el8.noarch 2024-04-20T01:39:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz-2017.2-11.el8.noarch 2024-04-20T01:39:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-04-20T01:39:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-04-20T01:39:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-04-20T01:39:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-04-20T01:39:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-04-20T01:39:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-04-20T01:39:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-04-20T01:39:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-04-20T01:39:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-04-20T01:39:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:39:29.902 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:39:30.100 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-base 2024-04-20T01:39:30.251 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 33 kB/s | 4.1 kB 00:00 2024-04-20T01:39:30.505 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:39:30.606 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 22 kB/s | 4.5 kB 00:00 2024-04-20T01:39:31.219 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 42 kB/s | 4.1 kB 00:00 2024-04-20T01:39:31.737 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 6/6 2024-04-20T01:39:31.737 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2024-04-20T01:39:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:39:32.272 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-20T01:39:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:39:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: jq-1.6-7.el8.x86_64 2024-04-20T01:39:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:39:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-04-20T01:39:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-1.el8.x86_64 2024-04-20T01:39:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-04-20T01:39:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:39:32.273 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:39:32.407 INFO:teuthology.orchestra.run.smithi165.stdout:Package ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:39:32.443 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph 2024-04-20T01:39:32.472 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 35 kB/s | 4.1 kB 00:00 2024-04-20T01:39:32.591 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:39:32.592 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-20T01:39:32.592 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:39:32.743 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install cephadm 2024-04-20T01:39:32.783 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:39:32.813 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 kB/s | 4.5 kB 00:00 2024-04-20T01:39:33.091 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:39:33.356 INFO:teuthology.orchestra.run.smithi121.stdout:Package ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:39:33.540 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:39:33.541 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-04-20T01:39:33.541 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:39:33.677 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install cephadm 2024-04-20T01:39:34.025 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:39:34.790 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 30 kB/s | 4.5 kB 00:00 2024-04-20T01:39:34.866 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 kB/s | 4.5 kB 00:00 2024-04-20T01:39:35.349 INFO:teuthology.orchestra.run.smithi044.stdout:Package ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:39:35.397 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 30 kB/s | 4.1 kB 00:00 2024-04-20T01:39:35.514 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 37 kB/s | 4.1 kB 00:00 2024-04-20T01:39:35.555 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:39:35.556 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-20T01:39:35.557 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:39:35.701 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install cephadm 2024-04-20T01:39:35.926 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 28 kB/s | 4.5 kB 00:00 2024-04-20T01:39:36.099 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:39:36.633 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 22 kB/s | 4.1 kB 00:00 2024-04-20T01:39:37.742 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:39:37.745 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================================= 2024-04-20T01:39:37.745 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-04-20T01:39:37.746 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================================= 2024-04-20T01:39:37.746 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-20T01:39:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 6.4 k 2024-04-20T01:39:37.746 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-04-20T01:39:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 2.3 M 2024-04-20T01:39:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 1.6 M 2024-04-20T01:39:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 246 k 2024-04-20T01:39:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 4.8 M 2024-04-20T01:39:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 18 M 2024-04-20T01:39:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-04-20T01:39:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-04-20T01:39:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-04-20T01:39:37.746 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-04-20T01:39:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-04-20T01:39:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-04-20T01:39:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-04-20T01:39:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-04-20T01:39:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-04-20T01:39:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-04-20T01:39:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-04-20T01:39:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-04-20T01:39:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-04-20T01:39:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-04-20T01:39:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-04-20T01:39:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-04-20T01:39:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-04-20T01:39:37.747 INFO:teuthology.orchestra.run.smithi123.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-04-20T01:39:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-04-20T01:39:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-04-20T01:39:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-04-20T01:39:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-04-20T01:39:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-04-20T01:39:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-04-20T01:39:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-04-20T01:39:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-04-20T01:39:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:39:37.748 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-20T01:39:37.748 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================================= 2024-04-20T01:39:37.748 INFO:teuthology.orchestra.run.smithi123.stdout:Install 32 Packages 2024-04-20T01:39:37.748 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:39:37.749 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 31 M 2024-04-20T01:39:37.749 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 124 M 2024-04-20T01:39:37.749 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-20T01:39:38.027 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:39:38.028 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:39:38.028 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-20T01:39:38.028 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:39:38.028 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-20T01:39:38.028 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 173 k 2024-04-20T01:39:38.028 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:39:38.028 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-20T01:39:38.028 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:39:38.028 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-04-20T01:39:38.029 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:39:38.029 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 173 k 2024-04-20T01:39:38.029 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 692 k 2024-04-20T01:39:38.029 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-20T01:39:38.062 INFO:teuthology.orchestra.run.smithi123.stdout:(1/32): ceph-18.2.2-1252.g15eecde2.el8.x86_64.r 38 kB/s | 6.4 kB 00:00 2024-04-20T01:39:38.293 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 43 kB/s | 4.1 kB 00:00 2024-04-20T01:39:38.610 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 53 kB/s | 4.5 kB 00:00 2024-04-20T01:39:38.659 INFO:teuthology.orchestra.run.smithi165.stdout:cephadm-18.2.2-1252.g15eecde2.el8.noarch.rpm 274 kB/s | 173 kB 00:00 2024-04-20T01:39:38.659 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:39:38.660 INFO:teuthology.orchestra.run.smithi165.stdout:Total 273 kB/s | 173 kB 00:00 2024-04-20T01:39:38.660 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-20T01:39:38.669 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-20T01:39:38.669 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-20T01:39:38.672 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-20T01:39:38.681 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-20T01:39:38.736 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-20T01:39:38.948 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:39:38.948 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:39:38.948 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-04-20T01:39:38.948 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:39:38.948 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-04-20T01:39:38.948 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 173 k 2024-04-20T01:39:38.948 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:39:38.949 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-20T01:39:38.949 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:39:38.949 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-04-20T01:39:38.949 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:39:38.949 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 173 k 2024-04-20T01:39:38.949 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 692 k 2024-04-20T01:39:38.949 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-20T01:39:39.154 INFO:teuthology.orchestra.run.smithi123.stdout:(2/32): ceph-mgr-18.2.2-1252.g15eecde2.el8.x86_ 1.2 MB/s | 1.6 MB 00:01 2024-04-20T01:39:39.386 INFO:teuthology.orchestra.run.smithi121.stdout:cephadm-18.2.2-1252.g15eecde2.el8.noarch.rpm 396 kB/s | 173 kB 00:00 2024-04-20T01:39:39.387 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:39:39.387 INFO:teuthology.orchestra.run.smithi121.stdout:Total 394 kB/s | 173 kB 00:00 2024-04-20T01:39:39.387 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-20T01:39:39.395 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-20T01:39:39.395 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-20T01:39:39.397 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-20T01:39:39.406 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-20T01:39:39.457 INFO:teuthology.orchestra.run.smithi123.stdout:(3/32): ceph-mds-18.2.2-1252.g15eecde2.el8.x86_ 1.5 MB/s | 2.3 MB 00:01 2024-04-20T01:39:39.458 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-20T01:39:39.597 INFO:teuthology.orchestra.run.smithi123.stdout:(4/32): ceph-mgr-modules-core-18.2.2-1252.g15ee 1.7 MB/s | 246 kB 00:00 2024-04-20T01:39:39.814 INFO:teuthology.orchestra.run.smithi123.stdout:(5/32): ceph-mon-18.2.2-1252.g15eecde2.el8.x86_ 2.8 MB/s | 4.8 MB 00:01 2024-04-20T01:39:39.955 INFO:teuthology.orchestra.run.smithi123.stdout:(6/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 122 kB/s | 44 kB 00:00 2024-04-20T01:39:40.239 INFO:teuthology.orchestra.run.smithi123.stdout:(7/32): python3-beautifulsoup4-4.6.3-2.el8.1.no 436 kB/s | 185 kB 00:00 2024-04-20T01:39:40.398 INFO:teuthology.orchestra.run.smithi123.stdout:(8/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 390 kB/s | 173 kB 00:00 2024-04-20T01:39:40.606 INFO:teuthology.orchestra.run.smithi123.stdout:(9/32): python3-jaraco-6.2-6.el8.noarch.rpm 53 kB/s | 11 kB 00:00 2024-04-20T01:39:40.875 INFO:teuthology.orchestra.run.smithi123.stdout:(10/32): ceph-osd-18.2.2-1252.g15eecde2.el8.x86 11 MB/s | 18 MB 00:01 2024-04-20T01:39:40.907 INFO:teuthology.orchestra.run.smithi123.stdout:(11/32): python3-cherrypy-18.4.0-1.el8.noarch.r 576 kB/s | 384 kB 00:00 2024-04-20T01:39:41.057 INFO:teuthology.orchestra.run.smithi123.stdout:(12/32): python3-jaraco-functools-2.0-4.el8.noa 39 kB/s | 18 kB 00:00 2024-04-20T01:39:41.195 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T01:39:41.232 INFO:teuthology.orchestra.run.smithi123.stdout:(13/32): python3-logutils-0.3.5-11.el8.noarch.r 136 kB/s | 49 kB 00:00 2024-04-20T01:39:41.258 INFO:teuthology.orchestra.run.smithi123.stdout:(14/32): python3-more-itertools-7.2.0-3.el8.noa 167 kB/s | 59 kB 00:00 2024-04-20T01:39:41.282 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T01:39:41.349 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T01:39:41.361 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:39:41.361 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:39:41.362 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-20T01:39:41.362 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:39:41.362 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-20T01:39:41.362 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 173 k 2024-04-20T01:39:41.362 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:39:41.362 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-20T01:39:41.362 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:39:41.362 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-04-20T01:39:41.362 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:39:41.363 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 173 k 2024-04-20T01:39:41.363 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 692 k 2024-04-20T01:39:41.363 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-20T01:39:41.445 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T01:39:41.508 INFO:teuthology.orchestra.run.smithi123.stdout:(15/32): python3-portend-2.6-1.el8.noarch.rpm 57 kB/s | 16 kB 00:00 2024-04-20T01:39:41.533 INFO:teuthology.orchestra.run.smithi123.stdout:(16/32): python3-simplegeneric-0.8.1-17.el8.noa 68 kB/s | 19 kB 00:00 2024-04-20T01:39:41.550 INFO:teuthology.orchestra.run.smithi044.stdout:cephadm-18.2.2-1252.g15eecde2.el8.noarch.rpm 923 kB/s | 173 kB 00:00 2024-04-20T01:39:41.551 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:39:41.551 INFO:teuthology.orchestra.run.smithi044.stdout:Total 916 kB/s | 173 kB 00:00 2024-04-20T01:39:41.551 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-20T01:39:41.553 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T01:39:41.558 INFO:teuthology.orchestra.run.smithi123.stdout:(17/32): python3-pecan-1.3.2-9.el8.noarch.rpm 564 kB/s | 283 kB 00:00 2024-04-20T01:39:41.561 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-20T01:39:41.561 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-20T01:39:41.564 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-20T01:39:41.574 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-20T01:39:41.626 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-20T01:39:41.645 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T01:39:41.775 INFO:teuthology.orchestra.run.smithi123.stdout:(18/32): python3-singledispatch-3.4.0.3-18.el8. 90 kB/s | 24 kB 00:00 2024-04-20T01:39:41.801 INFO:teuthology.orchestra.run.smithi123.stdout:(19/32): python3-tempora-1.14.1-5.el8.noarch.rp 109 kB/s | 29 kB 00:00 2024-04-20T01:39:41.826 INFO:teuthology.orchestra.run.smithi123.stdout:(20/32): python3-trustme-0.6.0-4.el8.noarch.rpm 103 kB/s | 27 kB 00:00 2024-04-20T01:39:42.168 INFO:teuthology.orchestra.run.smithi123.stdout:(21/32): python3-webtest-2.0.33-1.el8.noarch.rp 250 kB/s | 85 kB 00:00 2024-04-20T01:39:42.311 INFO:teuthology.orchestra.run.smithi123.stdout:(22/32): python3-waitress-1.4.3-1.el8.noarch.rp 477 kB/s | 255 kB 00:00 2024-04-20T01:39:42.343 INFO:teuthology.orchestra.run.smithi123.stdout:(23/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 463 kB/s | 251 kB 00:00 2024-04-20T01:39:42.427 INFO:teuthology.orchestra.run.smithi123.stdout:(24/32): python3-zc-lockfile-2.0-2.el8.noarch.r 89 kB/s | 23 kB 00:00 2024-04-20T01:39:42.510 INFO:teuthology.orchestra.run.smithi123.stdout:(25/32): python3-markupsafe-0.23-19.el8.x86_64. 233 kB/s | 39 kB 00:00 2024-04-20T01:39:42.536 INFO:teuthology.orchestra.run.smithi123.stdout:(26/32): python3-werkzeug-0.12.2-4.el8.noarch.r 2.0 MB/s | 457 kB 00:00 2024-04-20T01:39:42.619 INFO:teuthology.orchestra.run.smithi123.stdout:(27/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 539 kB/s | 103 kB 00:00 2024-04-20T01:39:42.645 INFO:teuthology.orchestra.run.smithi123.stdout:(28/32): python3-mako-1.0.6-14.el8.noarch.rpm 1.1 MB/s | 157 kB 00:00 2024-04-20T01:39:42.670 INFO:teuthology.orchestra.run.smithi123.stdout:(29/32): python3-pytz-2017.2-11.el8.noarch.rpm 405 kB/s | 54 kB 00:00 2024-04-20T01:39:42.745 INFO:teuthology.orchestra.run.smithi123.stdout:(30/32): python3-cryptography-3.2.1-7.el8_9.x86 7.3 MB/s | 559 kB 00:00 2024-04-20T01:39:42.770 INFO:teuthology.orchestra.run.smithi123.stdout:(31/32): python3-pycparser-2.14-14.el8.noarch.r 724 kB/s | 109 kB 00:00 2024-04-20T01:39:42.812 INFO:teuthology.orchestra.run.smithi123.stdout:(32/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.4 MB/s | 238 kB 00:00 2024-04-20T01:39:42.814 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:39:42.815 INFO:teuthology.orchestra.run.smithi123.stdout:Total 6.1 MB/s | 31 MB 00:05 2024-04-20T01:39:42.916 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-20T01:39:42.955 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-20T01:39:42.955 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-20T01:39:43.179 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-20T01:39:43.187 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-20T01:39:43.704 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T01:39:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T01:39:44.152 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T01:39:44.358 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-20T01:39:44.577 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-04-20T01:39:44.792 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-04-20T01:39:45.016 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 3/32 2024-04-20T01:39:45.420 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 4/32 2024-04-20T01:39:45.675 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 5/32 2024-04-20T01:39:45.897 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 6/32 2024-04-20T01:39:46.081 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 7/32 2024-04-20T01:39:46.337 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 8/32 2024-04-20T01:39:47.209 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 9/32 2024-04-20T01:39:48.671 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 10/32 2024-04-20T01:39:48.981 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 11/32 2024-04-20T01:39:49.205 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 12/32 2024-04-20T01:39:49.457 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 13/32 2024-04-20T01:39:49.697 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 14/32 2024-04-20T01:39:49.909 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 15/32 2024-04-20T01:39:50.157 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 16/32 2024-04-20T01:39:50.333 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 17/32 2024-04-20T01:39:50.519 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 18/32 2024-04-20T01:39:50.730 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 19/32 2024-04-20T01:39:50.925 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 20/32 2024-04-20T01:39:51.152 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-04-20T01:39:51.407 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-04-20T01:39:51.659 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-04-20T01:39:51.913 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 24/32 2024-04-20T01:39:52.200 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 25/32 2024-04-20T01:39:52.565 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-04-20T01:39:52.971 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1252.g15eecde2.el8. 27/32 2024-04-20T01:39:53.150 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 28/32 2024-04-20T01:39:54.788 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 28/32 2024-04-20T01:39:55.034 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 29/32 2024-04-20T01:39:55.449 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T01:39:55.449 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2024-04-20T01:39:55.514 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 29/32 2024-04-20T01:39:55.662 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 30/32 2024-04-20T01:39:55.876 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 30/32 2024-04-20T01:39:56.005 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:39:56.005 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-04-20T01:39:56.005 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:39:56.005 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:39:56.005 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:39:56.054 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 31/32 2024-04-20T01:39:56.072 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 31/32 2024-04-20T01:39:56.194 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 32/32 2024-04-20T01:39:56.197 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-immutable-object-cache 2024-04-20T01:39:56.559 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:39:56.579 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 32/32 2024-04-20T01:39:56.579 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 1/32 2024-04-20T01:39:56.579 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 2/32 2024-04-20T01:39:56.579 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 3/32 2024-04-20T01:39:56.579 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 4/32 2024-04-20T01:39:56.579 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 5/32 2024-04-20T01:39:56.580 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1252.g15eecde2.el8. 6/32 2024-04-20T01:39:56.580 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/32 2024-04-20T01:39:56.580 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/32 2024-04-20T01:39:56.580 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/32 2024-04-20T01:39:56.580 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/32 2024-04-20T01:39:56.580 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/32 2024-04-20T01:39:56.580 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/32 2024-04-20T01:39:56.580 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/32 2024-04-20T01:39:56.580 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/32 2024-04-20T01:39:56.580 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/32 2024-04-20T01:39:56.580 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/32 2024-04-20T01:39:56.580 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/32 2024-04-20T01:39:56.580 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/32 2024-04-20T01:39:56.581 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/32 2024-04-20T01:39:56.581 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/32 2024-04-20T01:39:56.581 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 21/32 2024-04-20T01:39:56.581 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/32 2024-04-20T01:39:56.581 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/32 2024-04-20T01:39:56.581 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/32 2024-04-20T01:39:56.581 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 25/32 2024-04-20T01:39:56.581 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 26/32 2024-04-20T01:39:56.581 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 27/32 2024-04-20T01:39:56.581 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 28/32 2024-04-20T01:39:56.581 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 29/32 2024-04-20T01:39:56.581 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 30/32 2024-04-20T01:39:56.581 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 31/32 2024-04-20T01:39:56.947 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T01:39:56.947 INFO:teuthology.orchestra.run.smithi165.stdout:Installed products updated. 2024-04-20T01:39:57.502 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:39:57.503 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-20T01:39:57.503 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:39:57.503 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:39:57.503 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:39:57.662 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-immutable-object-cache 2024-04-20T01:39:58.027 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:39:58.429 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 kB/s | 4.5 kB 00:00 2024-04-20T01:39:59.039 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 40 kB/s | 4.1 kB 00:00 2024-04-20T01:39:59.979 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 kB/s | 4.5 kB 00:00 2024-04-20T01:40:00.659 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 30 kB/s | 4.1 kB 00:00 2024-04-20T01:40:01.363 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:40:01.363 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:40:01.363 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repo Size 2024-04-20T01:40:01.363 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:40:01.363 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-04-20T01:40:01.364 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 166 k 2024-04-20T01:40:01.364 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:40:01.364 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-20T01:40:01.364 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:40:01.364 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-04-20T01:40:01.364 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:40:01.364 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 166 k 2024-04-20T01:40:01.365 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 503 k 2024-04-20T01:40:01.365 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-20T01:40:01.457 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T01:40:01.457 INFO:teuthology.orchestra.run.smithi044.stdout:Installed products updated. 2024-04-20T01:40:02.159 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:40:02.160 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-20T01:40:02.160 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:40:02.160 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:40:02.160 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:40:02.262 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-immutable-object-cache-18.2.2-1252.g15eecd 185 kB/s | 166 kB 00:00 2024-04-20T01:40:02.262 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:40:02.263 INFO:teuthology.orchestra.run.smithi121.stdout:Total 184 kB/s | 166 kB 00:00 2024-04-20T01:40:02.263 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-20T01:40:02.276 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-20T01:40:02.276 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-20T01:40:02.335 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-20T01:40:02.344 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-20T01:40:02.365 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-immutable-object-cache 2024-04-20T01:40:02.467 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-20T01:40:02.664 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2. 1/1 2024-04-20T01:40:02.788 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:40:03.062 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2. 1/1 2024-04-20T01:40:03.063 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:40:03.064 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:40:03.064 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repo Size 2024-04-20T01:40:03.064 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:40:03.064 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-20T01:40:03.064 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 166 k 2024-04-20T01:40:03.064 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:40:03.065 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-20T01:40:03.065 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:40:03.065 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-04-20T01:40:03.065 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:40:03.065 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 166 k 2024-04-20T01:40:03.065 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 503 k 2024-04-20T01:40:03.065 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-20T01:40:03.558 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-immutable-object-cache-18.2.2-1252.g15eecd 336 kB/s | 166 kB 00:00 2024-04-20T01:40:03.559 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:40:03.559 INFO:teuthology.orchestra.run.smithi165.stdout:Total 335 kB/s | 166 kB 00:00 2024-04-20T01:40:03.560 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-20T01:40:03.571 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-20T01:40:03.571 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-20T01:40:03.627 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-20T01:40:03.636 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-20T01:40:03.754 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-20T01:40:03.932 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2. 1/1 2024-04-20T01:40:04.316 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2. 1/1 2024-04-20T01:40:04.980 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 17 kB/s | 4.1 kB 00:00 2024-04-20T01:40:05.336 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 41 kB/s | 4.5 kB 00:00 2024-04-20T01:40:07.915 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 32/32 2024-04-20T01:40:07.916 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2024-04-20T01:40:08.096 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:40:08.097 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:40:08.097 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repo Size 2024-04-20T01:40:08.097 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:40:08.097 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-20T01:40:08.097 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 166 k 2024-04-20T01:40:08.097 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:40:08.097 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-20T01:40:08.097 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:40:08.097 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-04-20T01:40:08.097 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:40:08.098 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 166 k 2024-04-20T01:40:08.098 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 503 k 2024-04-20T01:40:08.098 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-20T01:40:08.307 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-immutable-object-cache-18.2.2-1252.g15eecd 791 kB/s | 166 kB 00:00 2024-04-20T01:40:08.308 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:40:08.308 INFO:teuthology.orchestra.run.smithi044.stdout:Total 786 kB/s | 166 kB 00:00 2024-04-20T01:40:08.309 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-20T01:40:08.322 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-20T01:40:08.323 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-20T01:40:08.388 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-20T01:40:08.399 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-20T01:40:08.446 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:40:08.446 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-20T01:40:08.446 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:40:08.446 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:40:08.446 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:40:08.446 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:40:08.446 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:40:08.446 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:40:08.446 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-04-20T01:40:08.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-04-20T01:40:08.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-04-20T01:40:08.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-04-20T01:40:08.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-04-20T01:40:08.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-04-20T01:40:08.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-6.2-6.el8.noarch 2024-04-20T01:40:08.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-04-20T01:40:08.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-04-20T01:40:08.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.0.6-14.el8.noarch 2024-04-20T01:40:08.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-04-20T01:40:08.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-04-20T01:40:08.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-04-20T01:40:08.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-2.6-1.el8.noarch 2024-04-20T01:40:08.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-04-20T01:40:08.447 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.14-14.el8.noarch 2024-04-20T01:40:08.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2017.2-11.el8.noarch 2024-04-20T01:40:08.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-04-20T01:40:08.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-04-20T01:40:08.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-04-20T01:40:08.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-04-20T01:40:08.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-04-20T01:40:08.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-04-20T01:40:08.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-04-20T01:40:08.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-04-20T01:40:08.448 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-04-20T01:40:08.448 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:40:08.448 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:40:08.529 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-20T01:40:08.669 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-base 2024-04-20T01:40:08.713 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2. 1/1 2024-04-20T01:40:09.022 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:40:09.162 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2. 1/1 2024-04-20T01:40:10.924 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 45 kB/s | 4.5 kB 00:00 2024-04-20T01:40:11.554 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 26 kB/s | 4.1 kB 00:00 2024-04-20T01:40:13.697 INFO:teuthology.orchestra.run.smithi123.stdout:Package ceph-base-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:40:13.882 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:40:13.882 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-04-20T01:40:13.883 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:40:14.045 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install cephadm 2024-04-20T01:40:14.393 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:40:16.733 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 27 kB/s | 4.5 kB 00:00 2024-04-20T01:40:17.323 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 35 kB/s | 4.1 kB 00:00 2024-04-20T01:40:19.624 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:40:19.625 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:40:19.625 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-04-20T01:40:19.625 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:40:19.626 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-20T01:40:19.626 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 173 k 2024-04-20T01:40:19.626 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:40:19.626 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-20T01:40:19.626 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:40:19.626 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-04-20T01:40:19.626 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:40:19.626 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 173 k 2024-04-20T01:40:19.626 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 692 k 2024-04-20T01:40:19.626 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-20T01:40:20.168 INFO:teuthology.orchestra.run.smithi123.stdout:cephadm-18.2.2-1252.g15eecde2.el8.noarch.rpm 319 kB/s | 173 kB 00:00 2024-04-20T01:40:20.168 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:40:20.169 INFO:teuthology.orchestra.run.smithi123.stdout:Total 318 kB/s | 173 kB 00:00 2024-04-20T01:40:20.169 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-20T01:40:20.177 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-20T01:40:20.177 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-20T01:40:20.180 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-20T01:40:20.189 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-20T01:40:20.230 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-20T01:40:21.076 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2. 1/1 2024-04-20T01:40:21.076 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2024-04-20T01:40:21.647 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:40:21.648 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-04-20T01:40:21.648 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:40:21.648 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:40:21.648 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:40:21.819 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr 2024-04-20T01:40:22.030 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T01:40:22.168 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:40:22.216 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T01:40:22.416 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T01:40:22.629 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2. 1/1 2024-04-20T01:40:22.630 INFO:teuthology.orchestra.run.smithi044.stdout:Installed products updated. 2024-04-20T01:40:22.765 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2. 1/1 2024-04-20T01:40:22.765 INFO:teuthology.orchestra.run.smithi165.stdout:Installed products updated. 2024-04-20T01:40:23.258 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:40:23.258 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-20T01:40:23.258 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:40:23.258 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:40:23.258 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:40:23.325 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:40:23.325 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-20T01:40:23.325 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:40:23.325 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:40:23.326 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:40:23.421 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-mgr 2024-04-20T01:40:23.500 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-mgr 2024-04-20T01:40:23.824 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:40:23.852 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:40:24.051 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 40 kB/s | 4.5 kB 00:00 2024-04-20T01:40:24.686 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 33 kB/s | 4.1 kB 00:00 2024-04-20T01:40:26.047 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 23 kB/s | 4.5 kB 00:00 2024-04-20T01:40:26.237 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 28 kB/s | 4.1 kB 00:00 2024-04-20T01:40:26.635 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 29 kB/s | 4.5 kB 00:00 2024-04-20T01:40:26.759 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 24 kB/s | 4.1 kB 00:00 2024-04-20T01:40:26.825 INFO:teuthology.orchestra.run.smithi121.stdout:Package ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:40:27.011 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:40:27.012 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-04-20T01:40:27.012 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:40:27.158 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr-dashboard 2024-04-20T01:40:27.504 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:40:28.862 INFO:teuthology.orchestra.run.smithi165.stdout:Package ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:40:29.050 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:40:29.052 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-20T01:40:29.052 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:40:29.186 INFO:teuthology.orchestra.run.smithi044.stdout:Package ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:40:29.231 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-mgr-dashboard 2024-04-20T01:40:29.401 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:40:29.402 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-20T01:40:29.402 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:40:29.421 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-04-20T01:40:29.545 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-mgr-dashboard 2024-04-20T01:40:29.577 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:40:29.947 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:40:30.074 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 32 kB/s | 4.1 kB 00:00 2024-04-20T01:40:31.686 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-04-20T01:40:32.133 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 34 kB/s | 4.1 kB 00:00 2024-04-20T01:40:32.309 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 56 kB/s | 4.1 kB 00:00 2024-04-20T01:40:32.471 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 45 kB/s | 4.5 kB 00:00 2024-04-20T01:40:32.504 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:40:32.505 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:40:32.506 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-04-20T01:40:32.506 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:40:32.506 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-04-20T01:40:32.506 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 3.5 M 2024-04-20T01:40:32.506 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-04-20T01:40:32.506 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 24 k 2024-04-20T01:40:32.506 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 15 k 2024-04-20T01:40:32.506 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-04-20T01:40:32.506 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-04-20T01:40:32.506 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:40:32.506 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-20T01:40:32.506 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:40:32.507 INFO:teuthology.orchestra.run.smithi121.stdout:Install 5 Packages 2024-04-20T01:40:32.507 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:40:32.507 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 3.8 M 2024-04-20T01:40:32.507 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 87 M 2024-04-20T01:40:32.507 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-20T01:40:32.729 INFO:teuthology.orchestra.run.smithi121.stdout:(1/5): ceph-prometheus-alerts-18.2.2-1252.g15ee 95 kB/s | 15 kB 00:00 2024-04-20T01:40:32.787 INFO:teuthology.orchestra.run.smithi121.stdout:(2/5): python3-repoze-lru-0.7-6.el8.noarch.rpm 577 kB/s | 34 kB 00:00 2024-04-20T01:40:32.821 INFO:teuthology.orchestra.run.smithi121.stdout:(3/5): ceph-grafana-dashboards-18.2.2-1252.g15e 96 kB/s | 24 kB 00:00 2024-04-20T01:40:32.854 INFO:teuthology.orchestra.run.smithi121.stdout:(4/5): python3-routes-2.4.1-12.el8.noarch.rpm 2.9 MB/s | 196 kB 00:00 2024-04-20T01:40:33.330 INFO:teuthology.orchestra.run.smithi121.stdout:(5/5): ceph-mgr-dashboard-18.2.2-1252.g15eecde2 4.6 MB/s | 3.5 MB 00:00 2024-04-20T01:40:33.332 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:40:33.332 INFO:teuthology.orchestra.run.smithi121.stdout:Total 4.6 MB/s | 3.8 MB 00:00 2024-04-20T01:40:33.333 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-20T01:40:33.340 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-20T01:40:33.341 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-20T01:40:33.408 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-20T01:40:33.417 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-20T01:40:33.674 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-20T01:40:33.921 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/5 2024-04-20T01:40:34.160 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/5 2024-04-20T01:40:34.314 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1252.g15eecde2.el8.n 3/5 2024-04-20T01:40:34.625 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:40:34.626 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:40:34.626 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-20T01:40:34.626 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:40:34.626 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-20T01:40:34.626 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 3.5 M 2024-04-20T01:40:34.626 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-20T01:40:34.626 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 24 k 2024-04-20T01:40:34.626 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 15 k 2024-04-20T01:40:34.626 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-04-20T01:40:34.627 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-04-20T01:40:34.627 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:40:34.627 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-20T01:40:34.627 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:40:34.627 INFO:teuthology.orchestra.run.smithi165.stdout:Install 5 Packages 2024-04-20T01:40:34.627 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:40:34.627 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 3.8 M 2024-04-20T01:40:34.627 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 87 M 2024-04-20T01:40:34.628 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-20T01:40:34.910 INFO:teuthology.orchestra.run.smithi165.stdout:(1/5): ceph-grafana-dashboards-18.2.2-1252.g15e 165 kB/s | 24 kB 00:00 2024-04-20T01:40:35.011 INFO:teuthology.orchestra.run.smithi165.stdout:(2/5): ceph-prometheus-alerts-18.2.2-1252.g15ee 61 kB/s | 15 kB 00:00 2024-04-20T01:40:35.077 INFO:teuthology.orchestra.run.smithi165.stdout:(3/5): python3-repoze-lru-0.7-6.el8.noarch.rpm 201 kB/s | 34 kB 00:00 2024-04-20T01:40:35.171 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/1 2024-04-20T01:40:35.171 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2024-04-20T01:40:35.228 INFO:teuthology.orchestra.run.smithi165.stdout:(4/5): python3-routes-2.4.1-12.el8.noarch.rpm 906 kB/s | 196 kB 00:00 2024-04-20T01:40:35.254 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:40:35.255 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:40:35.256 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-20T01:40:35.256 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:40:35.256 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-20T01:40:35.256 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 3.5 M 2024-04-20T01:40:35.256 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-20T01:40:35.256 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 24 k 2024-04-20T01:40:35.256 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 15 k 2024-04-20T01:40:35.256 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-04-20T01:40:35.256 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-04-20T01:40:35.256 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:40:35.256 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-20T01:40:35.256 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:40:35.256 INFO:teuthology.orchestra.run.smithi044.stdout:Install 5 Packages 2024-04-20T01:40:35.256 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:40:35.257 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 3.8 M 2024-04-20T01:40:35.257 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 87 M 2024-04-20T01:40:35.257 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-20T01:40:35.491 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1252.g15eecde2.el8. 4/5 2024-04-20T01:40:35.586 INFO:teuthology.orchestra.run.smithi165.stdout:(5/5): ceph-mgr-dashboard-18.2.2-1252.g15eecde2 4.3 MB/s | 3.5 MB 00:00 2024-04-20T01:40:35.588 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:40:35.588 INFO:teuthology.orchestra.run.smithi165.stdout:Total 3.9 MB/s | 3.8 MB 00:00 2024-04-20T01:40:35.588 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-20T01:40:35.596 INFO:teuthology.orchestra.run.smithi044.stdout:(1/5): ceph-grafana-dashboards-18.2.2-1252.g15e 149 kB/s | 24 kB 00:00 2024-04-20T01:40:35.596 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-20T01:40:35.596 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-20T01:40:35.663 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-20T01:40:35.672 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-20T01:40:35.688 INFO:teuthology.orchestra.run.smithi044.stdout:(2/5): ceph-prometheus-alerts-18.2.2-1252.g15ee 60 kB/s | 15 kB 00:00 2024-04-20T01:40:35.713 INFO:teuthology.orchestra.run.smithi044.stdout:(3/5): python3-repoze-lru-0.7-6.el8.noarch.rpm 288 kB/s | 34 kB 00:00 2024-04-20T01:40:35.730 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:40:35.730 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-20T01:40:35.730 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:40:35.730 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:40:35.730 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:40:35.787 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarc 5/5 2024-04-20T01:40:35.895 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-20T01:40:35.896 INFO:teuthology.orchestra.run.smithi044.stdout:(4/5): python3-routes-2.4.1-12.el8.noarch.rpm 946 kB/s | 196 kB 00:00 2024-04-20T01:40:35.914 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-immutable-object-cache 2024-04-20T01:40:36.041 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarc 5/5 2024-04-20T01:40:36.042 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1252.g15eecde2.el8. 1/5 2024-04-20T01:40:36.042 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarc 2/5 2024-04-20T01:40:36.042 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1252.g15eecde2.el8.n 3/5 2024-04-20T01:40:36.042 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/5 2024-04-20T01:40:36.115 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/5 2024-04-20T01:40:36.238 INFO:teuthology.orchestra.run.smithi044.stdout:(5/5): ceph-mgr-dashboard-18.2.2-1252.g15eecde2 4.4 MB/s | 3.5 MB 00:00 2024-04-20T01:40:36.240 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:40:36.240 INFO:teuthology.orchestra.run.smithi044.stdout:Total 3.9 MB/s | 3.8 MB 00:00 2024-04-20T01:40:36.241 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-20T01:40:36.251 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-20T01:40:36.251 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-20T01:40:36.279 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:40:36.321 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/5 2024-04-20T01:40:36.332 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-20T01:40:36.342 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-20T01:40:36.461 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1252.g15eecde2.el8.n 3/5 2024-04-20T01:40:36.585 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-20T01:40:36.839 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/5 2024-04-20T01:40:37.030 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/5 2024-04-20T01:40:37.196 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1252.g15eecde2.el8.n 3/5 2024-04-20T01:40:37.622 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1252.g15eecde2.el8. 4/5 2024-04-20T01:40:38.126 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarc 5/5 2024-04-20T01:40:38.209 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-04-20T01:40:38.398 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarc 5/5 2024-04-20T01:40:38.399 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1252.g15eecde2.el8. 1/5 2024-04-20T01:40:38.399 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarc 2/5 2024-04-20T01:40:38.399 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1252.g15eecde2.el8.n 3/5 2024-04-20T01:40:38.399 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/5 2024-04-20T01:40:38.448 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1252.g15eecde2.el8. 4/5 2024-04-20T01:40:38.814 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 kB/s | 4.1 kB 00:00 2024-04-20T01:40:38.912 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarc 5/5 2024-04-20T01:40:39.231 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarc 5/5 2024-04-20T01:40:39.231 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1252.g15eecde2.el8. 1/5 2024-04-20T01:40:39.231 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarc 2/5 2024-04-20T01:40:39.231 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1252.g15eecde2.el8.n 3/5 2024-04-20T01:40:39.231 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/5 2024-04-20T01:40:41.135 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:40:41.136 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:40:41.136 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repo Size 2024-04-20T01:40:41.136 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:40:41.136 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-20T01:40:41.136 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 166 k 2024-04-20T01:40:41.136 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:40:41.136 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-20T01:40:41.136 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:40:41.136 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-04-20T01:40:41.137 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:40:41.137 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 166 k 2024-04-20T01:40:41.137 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 503 k 2024-04-20T01:40:41.137 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-20T01:40:41.316 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-immutable-object-cache-18.2.2-1252.g15eecd 927 kB/s | 166 kB 00:00 2024-04-20T01:40:41.317 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:40:41.317 INFO:teuthology.orchestra.run.smithi123.stdout:Total 921 kB/s | 166 kB 00:00 2024-04-20T01:40:41.317 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-20T01:40:41.328 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-20T01:40:41.328 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-20T01:40:41.381 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-20T01:40:41.389 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-20T01:40:41.514 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-20T01:40:41.704 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2. 1/1 2024-04-20T01:40:42.109 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2. 1/1 2024-04-20T01:40:47.813 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/5 2024-04-20T01:40:47.814 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2024-04-20T01:40:48.361 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:40:48.361 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-04-20T01:40:48.361 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:40:48.361 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:40:48.361 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:40:48.361 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-04-20T01:40:48.361 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes-2.4.1-12.el8.noarch 2024-04-20T01:40:48.361 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:40:48.361 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:40:48.548 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-20T01:40:48.900 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:40:50.899 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 kB/s | 4.5 kB 00:00 2024-04-20T01:40:51.500 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/5 2024-04-20T01:40:51.500 INFO:teuthology.orchestra.run.smithi165.stdout:Installed products updated. 2024-04-20T01:40:51.538 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 32 kB/s | 4.1 kB 00:00 2024-04-20T01:40:52.107 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:40:52.107 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-20T01:40:52.107 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:40:52.107 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:40:52.107 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:40:52.107 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-04-20T01:40:52.107 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes-2.4.1-12.el8.noarch 2024-04-20T01:40:52.107 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:40:52.108 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:40:52.283 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-20T01:40:52.636 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:40:53.899 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:40:53.900 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================================= 2024-04-20T01:40:53.901 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-04-20T01:40:53.901 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================================= 2024-04-20T01:40:53.901 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-04-20T01:40:53.901 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 7.4 M 2024-04-20T01:40:53.901 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-04-20T01:40:53.901 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-04-20T01:40:53.901 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-04-20T01:40:53.901 INFO:teuthology.orchestra.run.smithi121.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-04-20T01:40:53.901 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-04-20T01:40:53.901 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-04-20T01:40:53.901 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-04-20T01:40:53.901 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-04-20T01:40:53.901 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:40:53.902 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-20T01:40:53.902 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================================= 2024-04-20T01:40:53.902 INFO:teuthology.orchestra.run.smithi121.stdout:Install 8 Packages 2024-04-20T01:40:53.902 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:40:53.902 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 36 M 2024-04-20T01:40:53.902 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 207 M 2024-04-20T01:40:53.903 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-20T01:40:53.961 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/5 2024-04-20T01:40:53.961 INFO:teuthology.orchestra.run.smithi044.stdout:Installed products updated. 2024-04-20T01:40:54.081 INFO:teuthology.orchestra.run.smithi121.stdout:(1/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 1.2 MB/s | 225 kB 00:00 2024-04-20T01:40:54.144 INFO:teuthology.orchestra.run.smithi121.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 15 MB/s | 3.7 MB 00:00 2024-04-20T01:40:54.303 INFO:teuthology.orchestra.run.smithi121.stdout:(3/8): openblas-0.3.15-6.el8.x86_64.rpm 32 MB/s | 5.0 MB 00:00 2024-04-20T01:40:54.420 INFO:teuthology.orchestra.run.smithi121.stdout:(4/8): openblas-threads-0.3.15-6.el8.x86_64.rpm 15 MB/s | 5.2 MB 00:00 2024-04-20T01:40:54.578 INFO:teuthology.orchestra.run.smithi121.stdout:(5/8): libgfortran-8.5.0-20.el8.x86_64.rpm 4.0 MB/s | 645 kB 00:00 2024-04-20T01:40:54.608 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 kB/s | 4.5 kB 00:00 2024-04-20T01:40:54.615 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:40:54.615 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-20T01:40:54.616 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:40:54.616 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:40:54.616 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:40:54.616 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-04-20T01:40:54.616 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes-2.4.1-12.el8.noarch 2024-04-20T01:40:54.616 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:40:54.616 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:40:54.762 INFO:teuthology.orchestra.run.smithi121.stdout:(6/8): python3-scipy-1.0.0-21.module+el8.9.0+20 30 MB/s | 14 MB 00:00 2024-04-20T01:40:54.808 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-20T01:40:54.845 INFO:teuthology.orchestra.run.smithi121.stdout:(7/8): ceph-mgr-diskprediction-local-18.2.2-125 7.8 MB/s | 7.4 MB 00:00 2024-04-20T01:40:54.904 INFO:teuthology.orchestra.run.smithi121.stdout:(8/8): libquadmath-8.5.0-20.el8.x86_64.rpm 530 kB/s | 172 kB 00:00 2024-04-20T01:40:54.905 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:40:54.905 INFO:teuthology.orchestra.run.smithi121.stdout:Total 36 MB/s | 36 MB 00:01 2024-04-20T01:40:55.155 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-20T01:40:55.174 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-20T01:40:55.174 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-20T01:40:55.211 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:40:55.292 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 30 kB/s | 4.1 kB 00:00 2024-04-20T01:40:55.486 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-20T01:40:55.494 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-20T01:40:56.087 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-20T01:40:56.088 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2. 1/1 2024-04-20T01:40:56.088 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2024-04-20T01:40:56.285 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 1/8 2024-04-20T01:40:56.415 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 1/8 2024-04-20T01:40:56.611 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 2/8 2024-04-20T01:40:56.712 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:40:56.713 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-20T01:40:56.713 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:40:56.713 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:40:56.713 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:40:56.881 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-mgr 2024-04-20T01:40:57.202 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 41 kB/s | 4.1 kB 00:00 2024-04-20T01:40:57.233 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:40:57.236 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 2/8 2024-04-20T01:40:57.443 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 3/8 2024-04-20T01:40:57.532 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 48 kB/s | 4.5 kB 00:00 2024-04-20T01:40:57.592 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:40:57.593 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================================================= 2024-04-20T01:40:57.593 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-20T01:40:57.593 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================================================= 2024-04-20T01:40:57.594 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-20T01:40:57.594 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 7.4 M 2024-04-20T01:40:57.594 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-20T01:40:57.594 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-04-20T01:40:57.594 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-04-20T01:40:57.594 INFO:teuthology.orchestra.run.smithi165.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-04-20T01:40:57.594 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-04-20T01:40:57.594 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-04-20T01:40:57.594 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-04-20T01:40:57.594 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-04-20T01:40:57.595 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:40:57.595 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-20T01:40:57.595 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================================================= 2024-04-20T01:40:57.595 INFO:teuthology.orchestra.run.smithi165.stdout:Install 8 Packages 2024-04-20T01:40:57.595 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:40:57.595 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 36 M 2024-04-20T01:40:57.595 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 207 M 2024-04-20T01:40:57.596 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-20T01:40:57.841 INFO:teuthology.orchestra.run.smithi165.stdout:(1/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 918 kB/s | 225 kB 00:00 2024-04-20T01:40:57.933 INFO:teuthology.orchestra.run.smithi165.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 11 MB/s | 3.7 MB 00:00 2024-04-20T01:40:58.050 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 3/8 2024-04-20T01:40:58.050 INFO:teuthology.orchestra.run.smithi165.stdout:(3/8): openblas-threads-0.3.15-6.el8.x86_64.rpm 25 MB/s | 5.2 MB 00:00 2024-04-20T01:40:58.200 INFO:teuthology.orchestra.run.smithi165.stdout:(4/8): openblas-0.3.15-6.el8.x86_64.rpm 19 MB/s | 5.0 MB 00:00 2024-04-20T01:40:58.292 INFO:teuthology.orchestra.run.smithi165.stdout:(5/8): ceph-mgr-diskprediction-local-18.2.2-125 11 MB/s | 7.4 MB 00:00 2024-04-20T01:40:58.302 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-0.3.15-6.el8.x86_64 4/8 2024-04-20T01:40:58.351 INFO:teuthology.orchestra.run.smithi165.stdout:(6/8): libgfortran-8.5.0-20.el8.x86_64.rpm 4.2 MB/s | 645 kB 00:00 2024-04-20T01:40:58.585 INFO:teuthology.orchestra.run.smithi165.stdout:(7/8): python3-scipy-1.0.0-21.module+el8.9.0+20 26 MB/s | 14 MB 00:00 2024-04-20T01:40:58.609 INFO:teuthology.orchestra.run.smithi165.stdout:(8/8): libquadmath-8.5.0-20.el8.x86_64.rpm 544 kB/s | 172 kB 00:00 2024-04-20T01:40:58.610 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:40:58.611 INFO:teuthology.orchestra.run.smithi165.stdout:Total 36 MB/s | 36 MB 00:01 2024-04-20T01:40:58.859 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-20T01:40:58.878 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-20T01:40:58.878 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-20T01:40:58.970 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 4/8 2024-04-20T01:40:59.137 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 33 kB/s | 4.5 kB 00:00 2024-04-20T01:40:59.189 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-20T01:40:59.198 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-20T01:40:59.339 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-04-20T01:40:59.552 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-20T01:40:59.745 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-20T01:40:59.771 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 27 kB/s | 4.1 kB 00:00 2024-04-20T01:40:59.905 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 1/8 2024-04-20T01:41:00.048 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 1/8 2024-04-20T01:41:00.230 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 2/8 2024-04-20T01:41:00.307 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:41:00.309 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================================================= 2024-04-20T01:41:00.309 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-20T01:41:00.309 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================================================= 2024-04-20T01:41:00.309 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-20T01:41:00.310 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 7.4 M 2024-04-20T01:41:00.310 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-20T01:41:00.310 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-04-20T01:41:00.310 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-04-20T01:41:00.310 INFO:teuthology.orchestra.run.smithi044.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-04-20T01:41:00.310 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-04-20T01:41:00.310 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-04-20T01:41:00.310 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-04-20T01:41:00.310 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-04-20T01:41:00.310 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:41:00.310 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-20T01:41:00.311 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================================================= 2024-04-20T01:41:00.311 INFO:teuthology.orchestra.run.smithi044.stdout:Install 8 Packages 2024-04-20T01:41:00.311 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:41:00.311 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 36 M 2024-04-20T01:41:00.311 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 207 M 2024-04-20T01:41:00.311 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-20T01:41:00.495 INFO:teuthology.orchestra.run.smithi044.stdout:(1/8): libquadmath-8.5.0-20.el8.x86_64.rpm 943 kB/s | 172 kB 00:00 2024-04-20T01:41:00.528 INFO:teuthology.orchestra.run.smithi044.stdout:(2/8): libgfortran-8.5.0-20.el8.x86_64.rpm 2.9 MB/s | 645 kB 00:00 2024-04-20T01:41:00.687 INFO:teuthology.orchestra.run.smithi044.stdout:(3/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 19 MB/s | 3.7 MB 00:00 2024-04-20T01:41:00.721 INFO:teuthology.orchestra.run.smithi044.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 1.1 MB/s | 225 kB 00:00 2024-04-20T01:41:00.870 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 2/8 2024-04-20T01:41:00.938 INFO:teuthology.orchestra.run.smithi044.stdout:(5/8): ceph-mgr-diskprediction-local-18.2.2-125 12 MB/s | 7.4 MB 00:00 2024-04-20T01:41:01.013 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 3/8 2024-04-20T01:41:01.055 INFO:teuthology.orchestra.run.smithi044.stdout:(6/8): openblas-threads-0.3.15-6.el8.x86_64.rpm 14 MB/s | 5.2 MB 00:00 2024-04-20T01:41:01.163 INFO:teuthology.orchestra.run.smithi044.stdout:(7/8): openblas-0.3.15-6.el8.x86_64.rpm 11 MB/s | 5.0 MB 00:00 2024-04-20T01:41:01.281 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-20T01:41:01.397 INFO:teuthology.orchestra.run.smithi044.stdout:(8/8): python3-scipy-1.0.0-21.module+el8.9.0+20 30 MB/s | 14 MB 00:00 2024-04-20T01:41:01.398 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:41:01.398 INFO:teuthology.orchestra.run.smithi044.stdout:Total 33 MB/s | 36 MB 00:01 2024-04-20T01:41:01.631 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 3/8 2024-04-20T01:41:01.669 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-20T01:41:01.690 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-20T01:41:01.690 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-20T01:41:01.863 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-0.3.15-6.el8.x86_64 4/8 2024-04-20T01:41:01.901 INFO:teuthology.orchestra.run.smithi123.stdout:Package ceph-mgr-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:41:02.036 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-20T01:41:02.046 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-20T01:41:02.088 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:41:02.089 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-04-20T01:41:02.089 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:41:02.249 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-mgr-dashboard 2024-04-20T01:41:02.541 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 4/8 2024-04-20T01:41:02.569 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525 7/8 2024-04-20T01:41:02.597 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:41:02.660 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-20T01:41:02.809 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde 8/8 2024-04-20T01:41:02.830 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-04-20T01:41:02.839 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 1/8 2024-04-20T01:41:02.964 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 1/8 2024-04-20T01:41:03.055 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-20T01:41:03.106 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 2/8 2024-04-20T01:41:03.157 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde 8/8 2024-04-20T01:41:03.157 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde 1/8 2024-04-20T01:41:03.157 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 2/8 2024-04-20T01:41:03.157 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 3/8 2024-04-20T01:41:03.157 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 4/8 2024-04-20T01:41:03.157 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 5/8 2024-04-20T01:41:03.157 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525 6/8 2024-04-20T01:41:03.157 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 7/8 2024-04-20T01:41:03.811 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 2/8 2024-04-20T01:41:04.114 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 3/8 2024-04-20T01:41:04.314 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 30 kB/s | 4.5 kB 00:00 2024-04-20T01:41:04.773 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 3/8 2024-04-20T01:41:04.826 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-20T01:41:04.990 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-0.3.15-6.el8.x86_64 4/8 2024-04-20T01:41:04.996 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 19 kB/s | 4.1 kB 00:00 2024-04-20T01:41:05.770 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 4/8 2024-04-20T01:41:06.115 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-04-20T01:41:06.129 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525 7/8 2024-04-20T01:41:06.337 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde 8/8 2024-04-20T01:41:06.339 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-20T01:41:06.639 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde 8/8 2024-04-20T01:41:06.639 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde 1/8 2024-04-20T01:41:06.639 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 2/8 2024-04-20T01:41:06.639 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 3/8 2024-04-20T01:41:06.639 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 4/8 2024-04-20T01:41:06.639 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 5/8 2024-04-20T01:41:06.639 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525 6/8 2024-04-20T01:41:06.640 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 7/8 2024-04-20T01:41:07.315 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:41:07.316 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:41:07.316 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-04-20T01:41:07.316 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:41:07.316 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-20T01:41:07.316 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 3.5 M 2024-04-20T01:41:07.316 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-04-20T01:41:07.316 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 24 k 2024-04-20T01:41:07.316 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 15 k 2024-04-20T01:41:07.316 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-04-20T01:41:07.316 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-04-20T01:41:07.317 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:41:07.317 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-20T01:41:07.317 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:41:07.317 INFO:teuthology.orchestra.run.smithi123.stdout:Install 5 Packages 2024-04-20T01:41:07.317 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:41:07.317 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 3.8 M 2024-04-20T01:41:07.317 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 87 M 2024-04-20T01:41:07.317 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-20T01:41:07.594 INFO:teuthology.orchestra.run.smithi123.stdout:(1/5): ceph-prometheus-alerts-18.2.2-1252.g15ee 107 kB/s | 15 kB 00:00 2024-04-20T01:41:07.711 INFO:teuthology.orchestra.run.smithi123.stdout:(2/5): ceph-grafana-dashboards-18.2.2-1252.g15e 94 kB/s | 24 kB 00:00 2024-04-20T01:41:07.736 INFO:teuthology.orchestra.run.smithi123.stdout:(3/5): python3-repoze-lru-0.7-6.el8.noarch.rpm 237 kB/s | 34 kB 00:00 2024-04-20T01:41:07.920 INFO:teuthology.orchestra.run.smithi123.stdout:(4/5): python3-routes-2.4.1-12.el8.noarch.rpm 940 kB/s | 196 kB 00:00 2024-04-20T01:41:08.162 INFO:teuthology.orchestra.run.smithi123.stdout:(5/5): ceph-mgr-dashboard-18.2.2-1252.g15eecde2 5.0 MB/s | 3.5 MB 00:00 2024-04-20T01:41:08.163 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:41:08.164 INFO:teuthology.orchestra.run.smithi123.stdout:Total 4.5 MB/s | 3.8 MB 00:00 2024-04-20T01:41:08.164 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-20T01:41:08.172 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-20T01:41:08.172 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-20T01:41:08.240 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-20T01:41:08.248 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-20T01:41:08.329 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-20T01:41:08.493 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-20T01:41:08.731 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/5 2024-04-20T01:41:08.953 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/5 2024-04-20T01:41:09.104 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1252.g15eecde2.el8.n 3/5 2024-04-20T01:41:09.857 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525 7/8 2024-04-20T01:41:10.263 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde 8/8 2024-04-20T01:41:10.275 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1252.g15eecde2.el8. 4/5 2024-04-20T01:41:10.554 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarc 5/5 2024-04-20T01:41:10.615 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde 8/8 2024-04-20T01:41:10.615 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde 1/8 2024-04-20T01:41:10.615 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 2/8 2024-04-20T01:41:10.615 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 3/8 2024-04-20T01:41:10.615 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-04-20T01:41:10.615 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-04-20T01:41:10.616 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 6/8 2024-04-20T01:41:10.616 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 7/8 2024-04-20T01:41:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarc 5/5 2024-04-20T01:41:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1252.g15eecde2.el8. 1/5 2024-04-20T01:41:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarc 2/5 2024-04-20T01:41:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1252.g15eecde2.el8.n 3/5 2024-04-20T01:41:10.816 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/5 2024-04-20T01:41:11.927 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 8/8 2024-04-20T01:41:11.928 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2024-04-20T01:41:12.491 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:41:12.491 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-04-20T01:41:12.492 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:41:12.492 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-04-20T01:41:12.492 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-04-20T01:41:12.492 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-0.3.15-6.el8.x86_64 2024-04-20T01:41:12.492 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-04-20T01:41:12.492 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-04-20T01:41:12.492 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-04-20T01:41:12.492 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-04-20T01:41:12.492 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:41:12.492 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:41:12.661 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr-rook 2024-04-20T01:41:13.011 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:41:15.130 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 28 kB/s | 4.5 kB 00:00 2024-04-20T01:41:15.797 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 35 kB/s | 4.1 kB 00:00 2024-04-20T01:41:18.107 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:41:18.109 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================== 2024-04-20T01:41:18.109 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-04-20T01:41:18.109 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================== 2024-04-20T01:41:18.109 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-04-20T01:41:18.109 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 49 k 2024-04-20T01:41:18.109 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-04-20T01:41:18.109 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-04-20T01:41:18.109 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-04-20T01:41:18.109 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-04-20T01:41:18.110 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-04-20T01:41:18.110 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-04-20T01:41:18.110 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-04-20T01:41:18.110 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-04-20T01:41:18.110 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-04-20T01:41:18.110 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-04-20T01:41:18.110 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-04-20T01:41:18.110 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-04-20T01:41:18.110 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-04-20T01:41:18.110 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-04-20T01:41:18.110 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:41:18.110 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-20T01:41:18.111 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================== 2024-04-20T01:41:18.111 INFO:teuthology.orchestra.run.smithi121.stdout:Install 14 Packages 2024-04-20T01:41:18.111 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:41:18.111 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 2.0 M 2024-04-20T01:41:18.111 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 25 M 2024-04-20T01:41:18.111 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-20T01:41:18.512 INFO:teuthology.orchestra.run.smithi121.stdout:(1/14): ceph-mgr-rook-18.2.2-1252.g15eecde2.el8 187 kB/s | 49 kB 00:00 2024-04-20T01:41:18.677 INFO:teuthology.orchestra.run.smithi121.stdout:(2/14): python3-certifi-2018.10.15-7.el8.noarch 38 kB/s | 16 kB 00:00 2024-04-20T01:41:18.854 INFO:teuthology.orchestra.run.smithi121.stdout:(3/14): python3-cachetools-3.1.1-4.el8.noarch.r 55 kB/s | 33 kB 00:00 2024-04-20T01:41:19.079 INFO:teuthology.orchestra.run.smithi121.stdout:(4/14): python3-google-auth-1.1.1-10.el8.noarch 143 kB/s | 81 kB 00:00 2024-04-20T01:41:19.113 INFO:teuthology.orchestra.run.smithi121.stdout:(5/14): python3-rsa-4.9-2.el8.noarch.rpm 248 kB/s | 64 kB 00:00 2024-04-20T01:41:19.296 INFO:teuthology.orchestra.run.smithi121.stdout:(6/14): python3-jsonpointer-1.10-11.el8.noarch. 109 kB/s | 20 kB 00:00 2024-04-20T01:41:19.322 INFO:teuthology.orchestra.run.smithi121.stdout:(7/14): python3-websocket-client-0.56.0-5.el8.n 253 kB/s | 61 kB 00:00 2024-04-20T01:41:19.472 INFO:teuthology.orchestra.run.smithi121.stdout:(8/14): python3-jsonpatch-1.21-2.el8.noarch.rpm 177 kB/s | 27 kB 00:00 2024-04-20T01:41:19.497 INFO:teuthology.orchestra.run.smithi121.stdout:(9/14): python3-pyasn1-modules-0.3.7-6.el8.noar 549 kB/s | 110 kB 00:00 2024-04-20T01:41:19.564 INFO:teuthology.orchestra.run.smithi121.stdout:(10/14): python3-jwt-1.6.1-2.el8.noarch.rpm 638 kB/s | 43 kB 00:00 2024-04-20T01:41:19.614 INFO:teuthology.orchestra.run.smithi121.stdout:(11/14): python3-pyasn1-0.3.7-6.el8.noarch.rpm 890 kB/s | 126 kB 00:00 2024-04-20T01:41:19.681 INFO:teuthology.orchestra.run.smithi121.stdout:(12/14): python3-oauthlib-2.1.0-1.el8.noarch.rp 1.3 MB/s | 155 kB 00:00 2024-04-20T01:41:19.813 INFO:teuthology.orchestra.run.smithi121.stdout:(13/14): python3-requests-oauthlib-1.0.0-1.el8. 215 kB/s | 43 kB 00:00 2024-04-20T01:41:20.948 INFO:teuthology.orchestra.run.smithi121.stdout:(14/14): python3-kubernetes-11.0.0-6.el8.noarch 526 kB/s | 1.2 MB 00:02 2024-04-20T01:41:20.949 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:41:20.949 INFO:teuthology.orchestra.run.smithi121.stdout:Total 712 kB/s | 2.0 MB 00:02 2024-04-20T01:41:21.029 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-20T01:41:21.044 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-20T01:41:21.044 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-20T01:41:21.125 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-20T01:41:21.134 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-20T01:41:21.145 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 8/8 2024-04-20T01:41:21.145 INFO:teuthology.orchestra.run.smithi165.stdout:Installed products updated. 2024-04-20T01:41:21.705 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-20T01:41:21.726 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:41:21.726 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-20T01:41:21.727 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:41:21.727 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-04-20T01:41:21.727 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-04-20T01:41:21.727 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-0.3.15-6.el8.x86_64 2024-04-20T01:41:21.727 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-04-20T01:41:21.727 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-04-20T01:41:21.727 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-04-20T01:41:21.727 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-04-20T01:41:21.727 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:41:21.727 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:41:21.895 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-mgr-rook 2024-04-20T01:41:21.964 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/14 2024-04-20T01:41:22.207 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rsa-4.9-2.el8.noarch 2/14 2024-04-20T01:41:22.247 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:41:22.448 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/14 2024-04-20T01:41:22.716 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 4/14 2024-04-20T01:41:23.009 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 5/14 2024-04-20T01:41:23.252 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 6/14 2024-04-20T01:41:23.486 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 7/14 2024-04-20T01:41:23.658 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 8/14 2024-04-20T01:41:23.904 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 9/14 2024-04-20T01:41:24.125 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 31 kB/s | 4.5 kB 00:00 2024-04-20T01:41:24.136 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 10/14 2024-04-20T01:41:24.445 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 11/14 2024-04-20T01:41:24.779 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 37 kB/s | 4.1 kB 00:00 2024-04-20T01:41:25.394 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/5 2024-04-20T01:41:25.394 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2024-04-20T01:41:25.537 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 12/14 2024-04-20T01:41:25.823 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 13/14 2024-04-20T01:41:25.840 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525 8/8 2024-04-20T01:41:25.840 INFO:teuthology.orchestra.run.smithi044.stdout:Installed products updated. 2024-04-20T01:41:25.952 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:41:25.952 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-20T01:41:25.952 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:41:25.952 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:41:25.953 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:41:25.953 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-04-20T01:41:25.953 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.4.1-12.el8.noarch 2024-04-20T01:41:25.953 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:41:25.953 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:41:26.037 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 14/14 2024-04-20T01:41:26.135 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-20T01:41:26.316 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 14/14 2024-04-20T01:41:26.316 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 1/14 2024-04-20T01:41:26.316 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/14 2024-04-20T01:41:26.316 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/14 2024-04-20T01:41:26.316 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/14 2024-04-20T01:41:26.316 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/14 2024-04-20T01:41:26.316 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 6/14 2024-04-20T01:41:26.316 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/14 2024-04-20T01:41:26.317 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 8/14 2024-04-20T01:41:26.317 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 9/14 2024-04-20T01:41:26.317 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 10/14 2024-04-20T01:41:26.317 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 11/14 2024-04-20T01:41:26.317 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 12/14 2024-04-20T01:41:26.318 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 13/14 2024-04-20T01:41:26.451 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:41:26.451 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-20T01:41:26.451 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:41:26.451 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-04-20T01:41:26.451 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-04-20T01:41:26.452 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-0.3.15-6.el8.x86_64 2024-04-20T01:41:26.452 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-04-20T01:41:26.452 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-04-20T01:41:26.452 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-04-20T01:41:26.452 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-04-20T01:41:26.452 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:41:26.452 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:41:26.486 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:41:26.632 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-mgr-rook 2024-04-20T01:41:27.036 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:41:27.107 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:41:27.108 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================================== 2024-04-20T01:41:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-20T01:41:27.109 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================================== 2024-04-20T01:41:27.109 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-20T01:41:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 49 k 2024-04-20T01:41:27.109 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-20T01:41:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-04-20T01:41:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-04-20T01:41:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-04-20T01:41:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-04-20T01:41:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-04-20T01:41:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-04-20T01:41:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-04-20T01:41:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-04-20T01:41:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-04-20T01:41:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-04-20T01:41:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-04-20T01:41:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-04-20T01:41:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-04-20T01:41:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:41:27.110 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-20T01:41:27.110 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================================== 2024-04-20T01:41:27.110 INFO:teuthology.orchestra.run.smithi165.stdout:Install 14 Packages 2024-04-20T01:41:27.110 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:41:27.111 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 2.0 M 2024-04-20T01:41:27.111 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 25 M 2024-04-20T01:41:27.111 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-20T01:41:27.411 INFO:teuthology.orchestra.run.smithi165.stdout:(1/14): python3-certifi-2018.10.15-7.el8.noarch 94 kB/s | 16 kB 00:00 2024-04-20T01:41:27.437 INFO:teuthology.orchestra.run.smithi165.stdout:(2/14): python3-cachetools-3.1.1-4.el8.noarch.r 166 kB/s | 33 kB 00:00 2024-04-20T01:41:27.487 INFO:teuthology.orchestra.run.smithi165.stdout:(3/14): python3-google-auth-1.1.1-10.el8.noarch 1.1 MB/s | 81 kB 00:00 2024-04-20T01:41:27.537 INFO:teuthology.orchestra.run.smithi165.stdout:(4/14): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2024-04-20T01:41:27.587 INFO:teuthology.orchestra.run.smithi165.stdout:(5/14): python3-websocket-client-0.56.0-5.el8.n 1.2 MB/s | 61 kB 00:00 2024-04-20T01:41:27.696 INFO:teuthology.orchestra.run.smithi165.stdout:(6/14): python3-kubernetes-11.0.0-6.el8.noarch. 4.5 MB/s | 1.2 MB 00:00 2024-04-20T01:41:27.721 INFO:teuthology.orchestra.run.smithi165.stdout:(7/14): ceph-mgr-rook-18.2.2-1252.g15eecde2.el8 102 kB/s | 49 kB 00:00 2024-04-20T01:41:27.755 INFO:teuthology.orchestra.run.smithi165.stdout:(8/14): python3-jsonpointer-1.10-11.el8.noarch. 119 kB/s | 20 kB 00:00 2024-04-20T01:41:27.864 INFO:teuthology.orchestra.run.smithi165.stdout:(9/14): python3-pyasn1-modules-0.3.7-6.el8.noar 658 kB/s | 110 kB 00:00 2024-04-20T01:41:27.889 INFO:teuthology.orchestra.run.smithi165.stdout:(10/14): python3-pyasn1-0.3.7-6.el8.noarch.rpm 986 kB/s | 126 kB 00:00 2024-04-20T01:41:27.914 INFO:teuthology.orchestra.run.smithi165.stdout:(11/14): python3-jsonpatch-1.21-2.el8.noarch.rp 142 kB/s | 27 kB 00:00 2024-04-20T01:41:27.997 INFO:teuthology.orchestra.run.smithi165.stdout:(12/14): python3-jwt-1.6.1-2.el8.noarch.rpm 320 kB/s | 43 kB 00:00 2024-04-20T01:41:28.031 INFO:teuthology.orchestra.run.smithi165.stdout:(13/14): python3-oauthlib-2.1.0-1.el8.noarch.rp 1.1 MB/s | 155 kB 00:00 2024-04-20T01:41:28.081 INFO:teuthology.orchestra.run.smithi165.stdout:(14/14): python3-requests-oauthlib-1.0.0-1.el8. 256 kB/s | 43 kB 00:00 2024-04-20T01:41:28.082 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:41:28.082 INFO:teuthology.orchestra.run.smithi165.stdout:Total 2.0 MB/s | 2.0 MB 00:00 2024-04-20T01:41:28.158 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-20T01:41:28.174 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-20T01:41:28.174 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-20T01:41:28.255 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-20T01:41:28.264 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-20T01:41:28.326 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-04-20T01:41:28.825 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-20T01:41:28.884 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 kB/s | 4.1 kB 00:00 2024-04-20T01:41:29.066 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/14 2024-04-20T01:41:29.179 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 42 kB/s | 4.1 kB 00:00 2024-04-20T01:41:29.285 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rsa-4.9-2.el8.noarch 2/14 2024-04-20T01:41:29.517 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/14 2024-04-20T01:41:29.526 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 41 kB/s | 4.5 kB 00:00 2024-04-20T01:41:29.785 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 4/14 2024-04-20T01:41:30.012 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 5/14 2024-04-20T01:41:30.205 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 6/14 2024-04-20T01:41:30.666 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 7/14 2024-04-20T01:41:30.870 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 8/14 2024-04-20T01:41:31.107 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 9/14 2024-04-20T01:41:31.272 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:41:31.274 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================================================= 2024-04-20T01:41:31.274 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-04-20T01:41:31.274 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================================================= 2024-04-20T01:41:31.274 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-20T01:41:31.274 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 7.4 M 2024-04-20T01:41:31.274 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-04-20T01:41:31.274 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-04-20T01:41:31.274 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-04-20T01:41:31.274 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-04-20T01:41:31.274 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-04-20T01:41:31.274 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-04-20T01:41:31.274 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-04-20T01:41:31.275 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-04-20T01:41:31.275 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:41:31.275 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-20T01:41:31.275 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================================================= 2024-04-20T01:41:31.275 INFO:teuthology.orchestra.run.smithi123.stdout:Install 8 Packages 2024-04-20T01:41:31.275 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:41:31.275 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 36 M 2024-04-20T01:41:31.276 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 207 M 2024-04-20T01:41:31.276 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-20T01:41:31.306 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 10/14 2024-04-20T01:41:31.502 INFO:teuthology.orchestra.run.smithi123.stdout:(1/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 997 kB/s | 225 kB 00:00 2024-04-20T01:41:31.590 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 11/14 2024-04-20T01:41:31.626 INFO:teuthology.orchestra.run.smithi123.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 11 MB/s | 3.7 MB 00:00 2024-04-20T01:41:31.809 INFO:teuthology.orchestra.run.smithi123.stdout:(3/8): openblas-threads-0.3.15-6.el8.x86_64.rpm 17 MB/s | 5.2 MB 00:00 2024-04-20T01:41:31.935 INFO:teuthology.orchestra.run.smithi123.stdout:(4/8): openblas-0.3.15-6.el8.x86_64.rpm 16 MB/s | 5.0 MB 00:00 2024-04-20T01:41:32.118 INFO:teuthology.orchestra.run.smithi123.stdout:(5/8): libgfortran-8.5.0-20.el8.x86_64.rpm 3.4 MB/s | 645 kB 00:00 2024-04-20T01:41:32.294 INFO:teuthology.orchestra.run.smithi123.stdout:(6/8): python3-scipy-1.0.0-21.module+el8.9.0+20 28 MB/s | 14 MB 00:00 2024-04-20T01:41:32.300 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:41:32.302 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================================== 2024-04-20T01:41:32.302 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-20T01:41:32.302 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================================== 2024-04-20T01:41:32.302 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-20T01:41:32.302 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 49 k 2024-04-20T01:41:32.302 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-20T01:41:32.302 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-04-20T01:41:32.302 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-04-20T01:41:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-04-20T01:41:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-04-20T01:41:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-04-20T01:41:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-04-20T01:41:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-04-20T01:41:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-04-20T01:41:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-04-20T01:41:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-04-20T01:41:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-04-20T01:41:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-04-20T01:41:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-04-20T01:41:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:41:32.303 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-20T01:41:32.303 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================================== 2024-04-20T01:41:32.304 INFO:teuthology.orchestra.run.smithi044.stdout:Install 14 Packages 2024-04-20T01:41:32.304 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:41:32.304 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 2.0 M 2024-04-20T01:41:32.304 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 25 M 2024-04-20T01:41:32.304 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-20T01:41:32.319 INFO:teuthology.orchestra.run.smithi123.stdout:(7/8): libquadmath-8.5.0-20.el8.x86_64.rpm 862 kB/s | 172 kB 00:00 2024-04-20T01:41:32.662 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 12/14 2024-04-20T01:41:32.823 INFO:teuthology.orchestra.run.smithi044.stdout:(1/14): python3-certifi-2018.10.15-7.el8.noarch 64 kB/s | 16 kB 00:00 2024-04-20T01:41:32.848 INFO:teuthology.orchestra.run.smithi044.stdout:(2/14): python3-cachetools-3.1.1-4.el8.noarch.r 118 kB/s | 33 kB 00:00 2024-04-20T01:41:32.935 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 13/14 2024-04-20T01:41:32.956 INFO:teuthology.orchestra.run.smithi044.stdout:(3/14): python3-google-auth-1.1.1-10.el8.noarch 608 kB/s | 81 kB 00:00 2024-04-20T01:41:33.032 INFO:teuthology.orchestra.run.smithi044.stdout:(4/14): ceph-mgr-rook-18.2.2-1252.g15eecde2.el8 106 kB/s | 49 kB 00:00 2024-04-20T01:41:33.082 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 14/14 2024-04-20T01:41:33.088 INFO:teuthology.orchestra.run.smithi044.stdout:(5/14): python3-rsa-4.9-2.el8.noarch.rpm 490 kB/s | 64 kB 00:00 2024-04-20T01:41:33.191 INFO:teuthology.orchestra.run.smithi044.stdout:(6/14): python3-kubernetes-11.0.0-6.el8.noarch. 3.4 MB/s | 1.2 MB 00:00 2024-04-20T01:41:33.216 INFO:teuthology.orchestra.run.smithi044.stdout:(7/14): python3-jwt-1.6.1-2.el8.noarch.rpm 333 kB/s | 43 kB 00:00 2024-04-20T01:41:33.319 INFO:teuthology.orchestra.run.smithi123.stdout:(8/8): ceph-mgr-diskprediction-local-18.2.2-125 3.6 MB/s | 7.4 MB 00:02 2024-04-20T01:41:33.320 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:41:33.320 INFO:teuthology.orchestra.run.smithi123.stdout:Total 18 MB/s | 36 MB 00:02 2024-04-20T01:41:33.341 INFO:teuthology.orchestra.run.smithi044.stdout:(8/14): python3-websocket-client-0.56.0-5.el8.n 198 kB/s | 61 kB 00:00 2024-04-20T01:41:33.367 INFO:teuthology.orchestra.run.smithi044.stdout:(9/14): python3-oauthlib-2.1.0-1.el8.noarch.rpm 887 kB/s | 155 kB 00:00 2024-04-20T01:41:33.400 INFO:teuthology.orchestra.run.smithi044.stdout:(10/14): python3-requests-oauthlib-1.0.0-1.el8. 232 kB/s | 43 kB 00:00 2024-04-20T01:41:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 14/14 2024-04-20T01:41:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 1/14 2024-04-20T01:41:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/14 2024-04-20T01:41:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/14 2024-04-20T01:41:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/14 2024-04-20T01:41:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/14 2024-04-20T01:41:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 6/14 2024-04-20T01:41:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/14 2024-04-20T01:41:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 8/14 2024-04-20T01:41:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 9/14 2024-04-20T01:41:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 10/14 2024-04-20T01:41:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 11/14 2024-04-20T01:41:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 12/14 2024-04-20T01:41:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 13/14 2024-04-20T01:41:33.476 INFO:teuthology.orchestra.run.smithi044.stdout:(11/14): python3-pyasn1-modules-0.3.7-6.el8.noa 1.1 MB/s | 110 kB 00:00 2024-04-20T01:41:33.543 INFO:teuthology.orchestra.run.smithi044.stdout:(12/14): python3-jsonpatch-1.21-2.el8.noarch.rp 187 kB/s | 27 kB 00:00 2024-04-20T01:41:33.568 INFO:teuthology.orchestra.run.smithi044.stdout:(13/14): python3-jsonpointer-1.10-11.el8.noarch 88 kB/s | 20 kB 00:00 2024-04-20T01:41:33.569 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-20T01:41:33.588 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-20T01:41:33.588 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-20T01:41:33.643 INFO:teuthology.orchestra.run.smithi044.stdout:(14/14): python3-pyasn1-0.3.7-6.el8.noarch.rpm 757 kB/s | 126 kB 00:00 2024-04-20T01:41:33.646 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:41:33.646 INFO:teuthology.orchestra.run.smithi044.stdout:Total 1.5 MB/s | 2.0 MB 00:01 2024-04-20T01:41:33.733 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-20T01:41:33.750 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-20T01:41:33.751 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-20T01:41:33.846 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-20T01:41:33.856 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-20T01:41:33.900 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-20T01:41:33.908 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-20T01:41:34.414 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-20T01:41:34.484 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-20T01:41:34.642 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/14 2024-04-20T01:41:34.687 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 1/8 2024-04-20T01:41:34.820 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 1/8 2024-04-20T01:41:34.912 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rsa-4.9-2.el8.noarch 2/14 2024-04-20T01:41:35.004 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 2/8 2024-04-20T01:41:35.093 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/14 2024-04-20T01:41:35.286 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 4/14 2024-04-20T01:41:35.599 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 5/14 2024-04-20T01:41:35.610 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 2/8 2024-04-20T01:41:35.804 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 3/8 2024-04-20T01:41:35.877 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 6/14 2024-04-20T01:41:36.399 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 3/8 2024-04-20T01:41:36.620 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.15-6.el8.x86_64 4/8 2024-04-20T01:41:37.310 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 4/8 2024-04-20T01:41:37.324 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 7/14 2024-04-20T01:41:37.616 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 8/14 2024-04-20T01:41:37.666 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-04-20T01:41:37.801 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 9/14 2024-04-20T01:41:37.920 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-20T01:41:38.026 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 10/14 2024-04-20T01:41:38.262 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 11/14 2024-04-20T01:41:39.429 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 14/14 2024-04-20T01:41:39.429 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2024-04-20T01:41:39.467 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 12/14 2024-04-20T01:41:39.668 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-20T01:41:39.750 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 13/14 2024-04-20T01:41:39.926 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 14/14 2024-04-20T01:41:39.995 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:41:39.995 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-04-20T01:41:39.995 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:41:39.995 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-04-20T01:41:39.995 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-04-20T01:41:39.995 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-04-20T01:41:39.996 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-04-20T01:41:39.996 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-04-20T01:41:39.996 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-04-20T01:41:39.996 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-04-20T01:41:39.996 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-04-20T01:41:39.996 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-04-20T01:41:39.996 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-04-20T01:41:39.996 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-04-20T01:41:39.996 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa-4.9-2.el8.noarch 2024-04-20T01:41:39.996 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-04-20T01:41:39.996 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:41:39.996 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:41:40.160 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr-cephadm 2024-04-20T01:41:40.245 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 14/14 2024-04-20T01:41:40.245 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 1/14 2024-04-20T01:41:40.245 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/14 2024-04-20T01:41:40.245 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/14 2024-04-20T01:41:40.245 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/14 2024-04-20T01:41:40.245 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/14 2024-04-20T01:41:40.245 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 6/14 2024-04-20T01:41:40.245 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/14 2024-04-20T01:41:40.245 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 8/14 2024-04-20T01:41:40.245 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 9/14 2024-04-20T01:41:40.245 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 10/14 2024-04-20T01:41:40.245 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 11/14 2024-04-20T01:41:40.246 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 12/14 2024-04-20T01:41:40.246 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 13/14 2024-04-20T01:41:40.515 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:41:40.972 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525 7/8 2024-04-20T01:41:41.169 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde 8/8 2024-04-20T01:41:41.474 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde 8/8 2024-04-20T01:41:41.474 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde 1/8 2024-04-20T01:41:41.474 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 2/8 2024-04-20T01:41:41.474 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 3/8 2024-04-20T01:41:41.474 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 4/8 2024-04-20T01:41:41.474 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 5/8 2024-04-20T01:41:41.475 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525 6/8 2024-04-20T01:41:41.475 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 7/8 2024-04-20T01:41:42.338 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 32 kB/s | 4.5 kB 00:00 2024-04-20T01:41:43.005 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 28 kB/s | 4.1 kB 00:00 2024-04-20T01:41:45.324 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:41:45.325 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================================== 2024-04-20T01:41:45.325 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-04-20T01:41:45.325 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================================== 2024-04-20T01:41:45.326 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-04-20T01:41:45.326 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 138 k 2024-04-20T01:41:45.326 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-04-20T01:41:45.326 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-04-20T01:41:45.326 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-04-20T01:41:45.326 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-04-20T01:41:45.326 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-04-20T01:41:45.326 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:41:45.326 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-20T01:41:45.326 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================================== 2024-04-20T01:41:45.326 INFO:teuthology.orchestra.run.smithi121.stdout:Install 5 Packages 2024-04-20T01:41:45.327 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:41:45.327 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 6.0 M 2024-04-20T01:41:45.327 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 26 M 2024-04-20T01:41:45.327 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-20T01:41:45.617 INFO:teuthology.orchestra.run.smithi121.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 432 kB/s | 60 kB 00:00 2024-04-20T01:41:45.642 INFO:teuthology.orchestra.run.smithi121.stdout:(2/5): ceph-mgr-cephadm-18.2.2-1252.g15eecde2.e 838 kB/s | 138 kB 00:00 2024-04-20T01:41:45.701 INFO:teuthology.orchestra.run.smithi121.stdout:(3/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 2.1 MB/s | 476 kB 00:00 2024-04-20T01:41:45.893 INFO:teuthology.orchestra.run.smithi121.stdout:(4/5): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2024-04-20T01:41:45.918 INFO:teuthology.orchestra.run.smithi121.stdout:(5/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 1.9 MB/s | 538 kB 00:00 2024-04-20T01:41:45.918 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:41:45.919 INFO:teuthology.orchestra.run.smithi121.stdout:Total 10 MB/s | 6.0 MB 00:00 2024-04-20T01:41:45.973 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-20T01:41:45.984 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-20T01:41:45.984 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-20T01:41:46.071 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-20T01:41:46.080 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-20T01:41:47.076 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-20T01:41:47.445 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 1/5 2024-04-20T01:41:47.767 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 2/5 2024-04-20T01:41:48.002 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 14/14 2024-04-20T01:41:48.002 INFO:teuthology.orchestra.run.smithi165.stdout:Installed products updated. 2024-04-20T01:41:48.146 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2024-04-20T01:41:48.465 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-04-20T01:41:48.563 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:41:48.563 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-20T01:41:48.563 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:41:48.563 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-04-20T01:41:48.563 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-04-20T01:41:48.563 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-04-20T01:41:48.563 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-04-20T01:41:48.563 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-04-20T01:41:48.563 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-04-20T01:41:48.563 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-04-20T01:41:48.563 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-04-20T01:41:48.563 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-04-20T01:41:48.564 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-04-20T01:41:48.564 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-04-20T01:41:48.564 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa-4.9-2.el8.noarch 2024-04-20T01:41:48.564 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-04-20T01:41:48.564 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:41:48.564 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:41:48.682 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 5/5 2024-04-20T01:41:48.719 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-mgr-cephadm 2024-04-20T01:41:48.951 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 5/5 2024-04-20T01:41:48.951 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/5 2024-04-20T01:41:48.952 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-04-20T01:41:48.952 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2024-04-20T01:41:48.952 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 4/5 2024-04-20T01:41:49.078 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:41:50.879 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 40 kB/s | 4.5 kB 00:00 2024-04-20T01:41:51.490 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 55 kB/s | 4.1 kB 00:00 2024-04-20T01:41:53.802 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:41:53.803 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================================== 2024-04-20T01:41:53.803 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-20T01:41:53.803 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================================== 2024-04-20T01:41:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-20T01:41:53.804 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 138 k 2024-04-20T01:41:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-20T01:41:53.804 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-04-20T01:41:53.804 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-04-20T01:41:53.804 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-04-20T01:41:53.804 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-04-20T01:41:53.804 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:41:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-20T01:41:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================================== 2024-04-20T01:41:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:Install 5 Packages 2024-04-20T01:41:53.805 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:41:53.805 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 6.0 M 2024-04-20T01:41:53.805 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 26 M 2024-04-20T01:41:53.805 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-20T01:41:54.192 INFO:teuthology.orchestra.run.smithi165.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 272 kB/s | 60 kB 00:00 2024-04-20T01:41:54.274 INFO:teuthology.orchestra.run.smithi165.stdout:(2/5): ceph-mgr-cephadm-18.2.2-1252.g15eecde2.e 447 kB/s | 138 kB 00:00 2024-04-20T01:41:54.324 INFO:teuthology.orchestra.run.smithi165.stdout:(3/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.3 MB/s | 476 kB 00:00 2024-04-20T01:41:54.425 INFO:teuthology.orchestra.run.smithi165.stdout:(4/5): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2024-04-20T01:41:54.458 INFO:teuthology.orchestra.run.smithi165.stdout:(5/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 2.9 MB/s | 538 kB 00:00 2024-04-20T01:41:54.460 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:41:54.460 INFO:teuthology.orchestra.run.smithi165.stdout:Total 9.1 MB/s | 6.0 MB 00:00 2024-04-20T01:41:54.514 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-20T01:41:54.526 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-20T01:41:54.526 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-20T01:41:54.615 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-20T01:41:54.624 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-20T01:41:54.979 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 14/14 2024-04-20T01:41:54.979 INFO:teuthology.orchestra.run.smithi044.stdout:Installed products updated. 2024-04-20T01:41:55.592 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-20T01:41:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:41:55.618 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-20T01:41:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:41:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-04-20T01:41:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-04-20T01:41:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-04-20T01:41:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-04-20T01:41:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-04-20T01:41:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-04-20T01:41:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-04-20T01:41:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-04-20T01:41:55.618 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-04-20T01:41:55.619 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-04-20T01:41:55.619 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-04-20T01:41:55.619 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa-4.9-2.el8.noarch 2024-04-20T01:41:55.619 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-04-20T01:41:55.619 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:41:55.619 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:41:55.801 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-mgr-cephadm 2024-04-20T01:41:55.931 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 1/5 2024-04-20T01:41:56.179 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 2/5 2024-04-20T01:41:56.211 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:41:56.509 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2024-04-20T01:41:56.746 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 8/8 2024-04-20T01:41:56.747 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2024-04-20T01:41:56.777 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-04-20T01:41:56.978 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 5/5 2024-04-20T01:41:57.243 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 5/5 2024-04-20T01:41:57.243 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/5 2024-04-20T01:41:57.243 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-04-20T01:41:57.243 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2024-04-20T01:41:57.243 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 4/5 2024-04-20T01:41:57.332 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:41:57.332 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-20T01:41:57.332 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:41:57.332 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-04-20T01:41:57.332 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-04-20T01:41:57.332 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.15-6.el8.x86_64 2024-04-20T01:41:57.332 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-04-20T01:41:57.332 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-04-20T01:41:57.333 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-04-20T01:41:57.333 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-04-20T01:41:57.333 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:41:57.333 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:41:57.506 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-mgr-rook 2024-04-20T01:41:57.855 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:41:58.355 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 31 kB/s | 4.1 kB 00:00 2024-04-20T01:41:58.816 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 23 kB/s | 4.5 kB 00:00 2024-04-20T01:42:00.291 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 kB/s | 4.5 kB 00:00 2024-04-20T01:42:00.908 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 kB/s | 4.1 kB 00:00 2024-04-20T01:42:01.604 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:42:01.605 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================================== 2024-04-20T01:42:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-20T01:42:01.606 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================================== 2024-04-20T01:42:01.606 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-20T01:42:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 138 k 2024-04-20T01:42:01.606 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-20T01:42:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-04-20T01:42:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-04-20T01:42:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-04-20T01:42:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-04-20T01:42:01.606 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:42:01.606 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-20T01:42:01.606 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================================== 2024-04-20T01:42:01.606 INFO:teuthology.orchestra.run.smithi044.stdout:Install 5 Packages 2024-04-20T01:42:01.607 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:42:01.607 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 6.0 M 2024-04-20T01:42:01.607 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 26 M 2024-04-20T01:42:01.607 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-20T01:42:02.058 INFO:teuthology.orchestra.run.smithi044.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 156 kB/s | 60 kB 00:00 2024-04-20T01:42:02.158 INFO:teuthology.orchestra.run.smithi044.stdout:(2/5): ceph-mgr-cephadm-18.2.2-1252.g15eecde2.e 283 kB/s | 138 kB 00:00 2024-04-20T01:42:02.350 INFO:teuthology.orchestra.run.smithi044.stdout:(3/5): python3-babel-2.5.1-7.el8.noarch.rpm 16 MB/s | 4.8 MB 00:00 2024-04-20T01:42:02.375 INFO:teuthology.orchestra.run.smithi044.stdout:(4/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 675 kB/s | 476 kB 00:00 2024-04-20T01:42:02.425 INFO:teuthology.orchestra.run.smithi044.stdout:(5/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 2.0 MB/s | 538 kB 00:00 2024-04-20T01:42:02.426 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:42:02.426 INFO:teuthology.orchestra.run.smithi044.stdout:Total 7.3 MB/s | 6.0 MB 00:00 2024-04-20T01:42:02.488 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-20T01:42:02.501 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-20T01:42:02.501 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-20T01:42:02.602 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-20T01:42:02.612 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-20T01:42:03.347 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:42:03.348 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================================== 2024-04-20T01:42:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-04-20T01:42:03.349 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================================== 2024-04-20T01:42:03.349 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-20T01:42:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 49 k 2024-04-20T01:42:03.349 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-04-20T01:42:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-04-20T01:42:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-04-20T01:42:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-04-20T01:42:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-04-20T01:42:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-04-20T01:42:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-04-20T01:42:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-04-20T01:42:03.349 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-04-20T01:42:03.350 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-04-20T01:42:03.350 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-04-20T01:42:03.350 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-04-20T01:42:03.350 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-04-20T01:42:03.350 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-04-20T01:42:03.350 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:42:03.350 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-20T01:42:03.350 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================================== 2024-04-20T01:42:03.350 INFO:teuthology.orchestra.run.smithi123.stdout:Install 14 Packages 2024-04-20T01:42:03.350 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:42:03.351 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 2.0 M 2024-04-20T01:42:03.351 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 25 M 2024-04-20T01:42:03.351 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-20T01:42:03.610 INFO:teuthology.orchestra.run.smithi123.stdout:(1/14): python3-certifi-2018.10.15-7.el8.noarch 221 kB/s | 16 kB 00:00 2024-04-20T01:42:03.635 INFO:teuthology.orchestra.run.smithi123.stdout:(2/14): python3-cachetools-3.1.1-4.el8.noarch.r 333 kB/s | 33 kB 00:00 2024-04-20T01:42:03.660 INFO:teuthology.orchestra.run.smithi123.stdout:(3/14): python3-google-auth-1.1.1-10.el8.noarch 1.6 MB/s | 81 kB 00:00 2024-04-20T01:42:03.685 INFO:teuthology.orchestra.run.smithi123.stdout:(4/14): python3-rsa-4.9-2.el8.noarch.rpm 2.5 MB/s | 64 kB 00:00 2024-04-20T01:42:03.702 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-20T01:42:03.744 INFO:teuthology.orchestra.run.smithi123.stdout:(5/14): ceph-mgr-rook-18.2.2-1252.g15eecde2.el8 236 kB/s | 49 kB 00:00 2024-04-20T01:42:03.769 INFO:teuthology.orchestra.run.smithi123.stdout:(6/14): python3-websocket-client-0.56.0-5.el8.n 732 kB/s | 61 kB 00:00 2024-04-20T01:42:03.803 INFO:teuthology.orchestra.run.smithi123.stdout:(7/14): python3-kubernetes-11.0.0-6.el8.noarch. 6.9 MB/s | 1.2 MB 00:00 2024-04-20T01:42:03.945 INFO:teuthology.orchestra.run.smithi123.stdout:(8/14): python3-jsonpointer-1.10-11.el8.noarch. 100 kB/s | 20 kB 00:00 2024-04-20T01:42:03.970 INFO:teuthology.orchestra.run.smithi123.stdout:(9/14): python3-pyasn1-modules-0.3.7-6.el8.noar 549 kB/s | 110 kB 00:00 2024-04-20T01:42:04.022 INFO:teuthology.orchestra.run.smithi123.stdout:(10/14): python3-jsonpatch-1.21-2.el8.noarch.rp 129 kB/s | 27 kB 00:00 2024-04-20T01:42:04.063 INFO:teuthology.orchestra.run.smithi123.stdout:(11/14): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2024-04-20T01:42:04.071 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 1/5 2024-04-20T01:42:04.088 INFO:teuthology.orchestra.run.smithi123.stdout:(12/14): python3-jwt-1.6.1-2.el8.noarch.rpm 363 kB/s | 43 kB 00:00 2024-04-20T01:42:04.180 INFO:teuthology.orchestra.run.smithi123.stdout:(13/14): python3-oauthlib-2.1.0-1.el8.noarch.rp 975 kB/s | 155 kB 00:00 2024-04-20T01:42:04.304 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 2/5 2024-04-20T01:42:04.347 INFO:teuthology.orchestra.run.smithi123.stdout:(14/14): python3-requests-oauthlib-1.0.0-1.el8. 150 kB/s | 43 kB 00:00 2024-04-20T01:42:04.347 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:42:04.347 INFO:teuthology.orchestra.run.smithi123.stdout:Total 2.0 MB/s | 2.0 MB 00:00 2024-04-20T01:42:04.417 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-20T01:42:04.434 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-20T01:42:04.434 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-20T01:42:04.512 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-20T01:42:04.520 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-20T01:42:04.665 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2024-04-20T01:42:04.964 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-04-20T01:42:05.074 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-20T01:42:05.148 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 5/5 2024-04-20T01:42:05.357 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/14 2024-04-20T01:42:05.446 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 5/5 2024-04-20T01:42:05.446 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/5 2024-04-20T01:42:05.446 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-04-20T01:42:05.446 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2024-04-20T01:42:05.446 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 4/5 2024-04-20T01:42:05.601 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el8.noarch 2/14 2024-04-20T01:42:05.841 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/14 2024-04-20T01:42:06.077 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 4/14 2024-04-20T01:42:06.369 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 5/14 2024-04-20T01:42:06.480 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 5/5 2024-04-20T01:42:06.480 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2024-04-20T01:42:06.596 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 6/14 2024-04-20T01:42:06.822 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 7/14 2024-04-20T01:42:07.035 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 8/14 2024-04-20T01:42:07.039 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:42:07.039 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-04-20T01:42:07.039 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:42:07.039 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-04-20T01:42:07.039 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel-2.5.1-7.el8.noarch 2024-04-20T01:42:07.039 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-04-20T01:42:07.039 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-04-20T01:42:07.039 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:42:07.039 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:42:07.194 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-fuse 2024-04-20T01:42:07.289 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 9/14 2024-04-20T01:42:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 10/14 2024-04-20T01:42:07.553 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:42:07.721 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 11/14 2024-04-20T01:42:08.816 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 12/14 2024-04-20T01:42:09.262 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 38 kB/s | 4.5 kB 00:00 2024-04-20T01:42:09.560 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 5/5 2024-04-20T01:42:09.560 INFO:teuthology.orchestra.run.smithi165.stdout:Installed products updated. 2024-04-20T01:42:09.906 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 31 kB/s | 4.1 kB 00:00 2024-04-20T01:42:09.936 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 13/14 2024-04-20T01:42:10.119 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:42:10.119 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-20T01:42:10.119 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:42:10.119 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-04-20T01:42:10.119 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel-2.5.1-7.el8.noarch 2024-04-20T01:42:10.119 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-04-20T01:42:10.119 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-04-20T01:42:10.119 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:42:10.119 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:42:10.279 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-fuse 2024-04-20T01:42:10.635 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:42:11.231 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 14/14 2024-04-20T01:42:11.517 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 14/14 2024-04-20T01:42:11.517 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 1/14 2024-04-20T01:42:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/14 2024-04-20T01:42:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/14 2024-04-20T01:42:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/14 2024-04-20T01:42:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/14 2024-04-20T01:42:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 6/14 2024-04-20T01:42:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/14 2024-04-20T01:42:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 8/14 2024-04-20T01:42:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 9/14 2024-04-20T01:42:11.518 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 10/14 2024-04-20T01:42:11.519 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 11/14 2024-04-20T01:42:11.519 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 12/14 2024-04-20T01:42:11.519 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 13/14 2024-04-20T01:42:12.081 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 28 kB/s | 4.5 kB 00:00 2024-04-20T01:42:12.219 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:42:12.220 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:42:12.220 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-04-20T01:42:12.220 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:42:12.220 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-04-20T01:42:12.220 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse 2024-04-20T01:42:12.220 INFO:teuthology.orchestra.run.smithi121.stdout: x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 859 k 2024-04-20T01:42:12.220 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-04-20T01:42:12.220 INFO:teuthology.orchestra.run.smithi121.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-04-20T01:42:12.220 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:42:12.221 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-20T01:42:12.221 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:42:12.221 INFO:teuthology.orchestra.run.smithi121.stdout:Install 2 Packages 2024-04-20T01:42:12.221 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:42:12.221 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 942 k 2024-04-20T01:42:12.221 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 3.0 M 2024-04-20T01:42:12.221 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-20T01:42:12.451 INFO:teuthology.orchestra.run.smithi121.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 361 kB/s | 83 kB 00:00 2024-04-20T01:42:12.514 INFO:teuthology.orchestra.run.smithi121.stdout:(2/2): ceph-fuse-18.2.2-1252.g15eecde2.el8.x86_ 2.9 MB/s | 859 kB 00:00 2024-04-20T01:42:12.515 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:42:12.515 INFO:teuthology.orchestra.run.smithi121.stdout:Total 3.1 MB/s | 942 kB 00:00 2024-04-20T01:42:12.527 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-20T01:42:12.549 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-20T01:42:12.549 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-20T01:42:12.633 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-20T01:42:12.642 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-20T01:42:12.776 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 kB/s | 4.1 kB 00:00 2024-04-20T01:42:12.809 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-20T01:42:13.051 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-04-20T01:42:13.235 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 2/2 2024-04-20T01:42:13.603 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 2/2 2024-04-20T01:42:13.604 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/2 2024-04-20T01:42:15.159 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:42:15.159 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:42:15.159 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-20T01:42:15.159 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:42:15.160 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-20T01:42:15.160 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse 2024-04-20T01:42:15.160 INFO:teuthology.orchestra.run.smithi165.stdout: x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 859 k 2024-04-20T01:42:15.160 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-20T01:42:15.160 INFO:teuthology.orchestra.run.smithi165.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-04-20T01:42:15.160 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:42:15.160 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-20T01:42:15.160 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:42:15.160 INFO:teuthology.orchestra.run.smithi165.stdout:Install 2 Packages 2024-04-20T01:42:15.160 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:42:15.161 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 942 k 2024-04-20T01:42:15.161 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 3.0 M 2024-04-20T01:42:15.161 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-20T01:42:15.293 INFO:teuthology.orchestra.run.smithi165.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 631 kB/s | 83 kB 00:00 2024-04-20T01:42:15.390 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 5/5 2024-04-20T01:42:15.390 INFO:teuthology.orchestra.run.smithi044.stdout:Installed products updated. 2024-04-20T01:42:15.439 INFO:teuthology.orchestra.run.smithi165.stdout:(2/2): ceph-fuse-18.2.2-1252.g15eecde2.el8.x86_ 3.0 MB/s | 859 kB 00:00 2024-04-20T01:42:15.440 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:42:15.440 INFO:teuthology.orchestra.run.smithi165.stdout:Total 3.3 MB/s | 942 kB 00:00 2024-04-20T01:42:15.452 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-20T01:42:15.473 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-20T01:42:15.474 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-20T01:42:15.561 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-20T01:42:15.569 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-20T01:42:15.743 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-20T01:42:15.998 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-04-20T01:42:16.013 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:42:16.013 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-20T01:42:16.013 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:42:16.013 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-04-20T01:42:16.013 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel-2.5.1-7.el8.noarch 2024-04-20T01:42:16.013 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-04-20T01:42:16.014 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-04-20T01:42:16.014 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:42:16.014 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:42:16.207 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 2/2 2024-04-20T01:42:16.314 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-fuse 2024-04-20T01:42:16.578 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 2/2 2024-04-20T01:42:16.579 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/2 2024-04-20T01:42:16.727 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:42:18.621 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 kB/s | 4.1 kB 00:00 2024-04-20T01:42:19.027 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 32 kB/s | 4.5 kB 00:00 2024-04-20T01:42:21.776 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:42:21.777 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:42:21.777 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-20T01:42:21.777 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:42:21.777 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-20T01:42:21.777 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse 2024-04-20T01:42:21.777 INFO:teuthology.orchestra.run.smithi044.stdout: x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 859 k 2024-04-20T01:42:21.777 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-20T01:42:21.778 INFO:teuthology.orchestra.run.smithi044.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-04-20T01:42:21.778 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:42:21.778 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-20T01:42:21.778 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:42:21.778 INFO:teuthology.orchestra.run.smithi044.stdout:Install 2 Packages 2024-04-20T01:42:21.778 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:42:21.778 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 942 k 2024-04-20T01:42:21.779 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 3.0 M 2024-04-20T01:42:21.779 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-20T01:42:21.987 INFO:teuthology.orchestra.run.smithi044.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 399 kB/s | 83 kB 00:00 2024-04-20T01:42:22.180 INFO:teuthology.orchestra.run.smithi044.stdout:(2/2): ceph-fuse-18.2.2-1252.g15eecde2.el8.x86_ 2.1 MB/s | 859 kB 00:00 2024-04-20T01:42:22.181 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:42:22.181 INFO:teuthology.orchestra.run.smithi044.stdout:Total 2.3 MB/s | 942 kB 00:00 2024-04-20T01:42:22.195 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-20T01:42:22.221 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-20T01:42:22.221 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-20T01:42:22.320 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-20T01:42:22.330 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-20T01:42:22.521 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-20T01:42:22.829 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-04-20T01:42:22.996 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 2/2 2024-04-20T01:42:23.403 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 2/2 2024-04-20T01:42:23.403 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/2 2024-04-20T01:42:25.506 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 14/14 2024-04-20T01:42:25.506 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2024-04-20T01:42:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:42:26.077 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-20T01:42:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:42:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-04-20T01:42:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-04-20T01:42:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-04-20T01:42:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-04-20T01:42:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-04-20T01:42:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-04-20T01:42:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-04-20T01:42:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-04-20T01:42:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-04-20T01:42:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-04-20T01:42:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-04-20T01:42:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el8.noarch 2024-04-20T01:42:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-04-20T01:42:26.078 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:42:26.078 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:42:26.161 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-04-20T01:42:26.161 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2024-04-20T01:42:26.267 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-mgr-cephadm 2024-04-20T01:42:26.623 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:42:26.729 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:42:26.729 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-04-20T01:42:26.729 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-04-20T01:42:26.729 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:42:26.729 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:42:26.936 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-volume 2024-04-20T01:42:27.298 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:42:29.047 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 32 kB/s | 4.5 kB 00:00 2024-04-20T01:42:29.498 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-04-20T01:42:29.498 INFO:teuthology.orchestra.run.smithi165.stdout:Installed products updated. 2024-04-20T01:42:29.594 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 29 kB/s | 4.5 kB 00:00 2024-04-20T01:42:29.663 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 35 kB/s | 4.1 kB 00:00 2024-04-20T01:42:30.072 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:42:30.072 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-20T01:42:30.072 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-04-20T01:42:30.072 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:42:30.073 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:42:30.226 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-volume 2024-04-20T01:42:30.253 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 kB/s | 4.1 kB 00:00 2024-04-20T01:42:30.589 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:42:31.989 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:42:32.013 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================================== 2024-04-20T01:42:32.013 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-04-20T01:42:32.014 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================================== 2024-04-20T01:42:32.014 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-20T01:42:32.014 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 138 k 2024-04-20T01:42:32.014 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-04-20T01:42:32.014 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-04-20T01:42:32.014 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-04-20T01:42:32.014 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-04-20T01:42:32.014 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-04-20T01:42:32.014 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:42:32.014 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-20T01:42:32.014 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================================== 2024-04-20T01:42:32.014 INFO:teuthology.orchestra.run.smithi123.stdout:Install 5 Packages 2024-04-20T01:42:32.015 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:42:32.015 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 6.0 M 2024-04-20T01:42:32.015 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 26 M 2024-04-20T01:42:32.015 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-20T01:42:32.378 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 39 kB/s | 4.5 kB 00:00 2024-04-20T01:42:32.678 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:42:32.678 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================== 2024-04-20T01:42:32.679 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-04-20T01:42:32.679 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================== 2024-04-20T01:42:32.679 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-04-20T01:42:32.679 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 263 k 2024-04-20T01:42:32.679 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-04-20T01:42:32.679 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-04-20T01:42:32.679 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:42:32.679 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-20T01:42:32.679 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================== 2024-04-20T01:42:32.679 INFO:teuthology.orchestra.run.smithi121.stdout:Install 2 Packages 2024-04-20T01:42:32.679 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:42:32.680 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 311 k 2024-04-20T01:42:32.680 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 1.4 M 2024-04-20T01:42:32.680 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-20T01:42:32.681 INFO:teuthology.orchestra.run.smithi123.stdout:(1/5): ceph-mgr-cephadm-18.2.2-1252.g15eecde2.e 381 kB/s | 138 kB 00:00 2024-04-20T01:42:32.846 INFO:teuthology.orchestra.run.smithi121.stdout:(1/2): python3-packaging-16.8-9.el8.noarch.rpm 290 kB/s | 48 kB 00:00 2024-04-20T01:42:32.913 INFO:teuthology.orchestra.run.smithi121.stdout:(2/2): ceph-volume-18.2.2-1252.g15eecde2.el8.no 1.1 MB/s | 263 kB 00:00 2024-04-20T01:42:32.914 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:42:32.914 INFO:teuthology.orchestra.run.smithi121.stdout:Total 1.3 MB/s | 311 kB 00:00 2024-04-20T01:42:32.915 INFO:teuthology.orchestra.run.smithi123.stdout:(2/5): python3-natsort-7.1.1-2.el8.noarch.rpm 101 kB/s | 60 kB 00:00 2024-04-20T01:42:32.925 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-20T01:42:32.938 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-20T01:42:32.938 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-20T01:42:32.976 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-20T01:42:32.984 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-20T01:42:32.990 INFO:teuthology.orchestra.run.smithi123.stdout:(3/5): python3-babel-2.5.1-7.el8.noarch.rpm 16 MB/s | 4.8 MB 00:00 2024-04-20T01:42:33.041 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 37 kB/s | 4.1 kB 00:00 2024-04-20T01:42:33.132 INFO:teuthology.orchestra.run.smithi123.stdout:(4/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 2.4 MB/s | 538 kB 00:00 2024-04-20T01:42:33.201 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-20T01:42:33.549 INFO:teuthology.orchestra.run.smithi123.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 387 kB/s | 476 kB 00:01 2024-04-20T01:42:33.551 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:42:33.551 INFO:teuthology.orchestra.run.smithi123.stdout:Total 3.8 MB/s | 6.0 MB 00:01 2024-04-20T01:42:33.589 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-packaging-16.8-9.el8.noarch 1/2 2024-04-20T01:42:33.608 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-20T01:42:33.620 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-20T01:42:33.620 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-20T01:42:33.710 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-20T01:42:33.719 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-20T01:42:33.872 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 2/2 2024-04-20T01:42:34.270 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 2/2 2024-04-20T01:42:34.271 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 1/2 2024-04-20T01:42:34.690 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-20T01:42:35.073 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 1/5 2024-04-20T01:42:35.350 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:42:35.350 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================================== 2024-04-20T01:42:35.350 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-20T01:42:35.350 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================================== 2024-04-20T01:42:35.350 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-20T01:42:35.350 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 263 k 2024-04-20T01:42:35.350 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-20T01:42:35.351 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-04-20T01:42:35.351 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:42:35.351 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-20T01:42:35.351 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================================== 2024-04-20T01:42:35.351 INFO:teuthology.orchestra.run.smithi165.stdout:Install 2 Packages 2024-04-20T01:42:35.351 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:42:35.351 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 311 k 2024-04-20T01:42:35.352 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 1.4 M 2024-04-20T01:42:35.352 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-20T01:42:35.371 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 2/5 2024-04-20T01:42:35.513 INFO:teuthology.orchestra.run.smithi165.stdout:(1/2): python3-packaging-16.8-9.el8.noarch.rpm 300 kB/s | 48 kB 00:00 2024-04-20T01:42:35.700 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2024-04-20T01:42:35.730 INFO:teuthology.orchestra.run.smithi165.stdout:(2/2): ceph-volume-18.2.2-1252.g15eecde2.el8.no 696 kB/s | 263 kB 00:00 2024-04-20T01:42:35.730 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:42:35.730 INFO:teuthology.orchestra.run.smithi165.stdout:Total 820 kB/s | 311 kB 00:00 2024-04-20T01:42:35.743 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-20T01:42:35.756 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-20T01:42:35.756 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-20T01:42:35.794 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-20T01:42:35.803 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-20T01:42:35.965 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-20T01:42:35.993 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-04-20T01:42:36.202 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 5/5 2024-04-20T01:42:36.302 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-packaging-16.8-9.el8.noarch 1/2 2024-04-20T01:42:36.509 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 5/5 2024-04-20T01:42:36.509 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 1/5 2024-04-20T01:42:36.509 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-04-20T01:42:36.509 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2024-04-20T01:42:36.509 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 4/5 2024-04-20T01:42:36.554 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 2/2 2024-04-20T01:42:36.951 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 2/2 2024-04-20T01:42:36.951 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 1/2 2024-04-20T01:42:37.242 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-04-20T01:42:37.242 INFO:teuthology.orchestra.run.smithi044.stdout:Installed products updated. 2024-04-20T01:42:37.866 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:42:37.867 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-20T01:42:37.867 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-04-20T01:42:37.867 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:42:37.867 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:42:38.034 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-volume 2024-04-20T01:42:38.451 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:42:40.408 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 39 kB/s | 4.1 kB 00:00 2024-04-20T01:42:40.784 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 31 kB/s | 4.5 kB 00:00 2024-04-20T01:42:43.566 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:42:43.566 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================================== 2024-04-20T01:42:43.566 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-20T01:42:43.566 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================================== 2024-04-20T01:42:43.566 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-20T01:42:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 263 k 2024-04-20T01:42:43.567 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-20T01:42:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-04-20T01:42:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:42:43.567 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-20T01:42:43.567 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================================== 2024-04-20T01:42:43.567 INFO:teuthology.orchestra.run.smithi044.stdout:Install 2 Packages 2024-04-20T01:42:43.567 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:42:43.568 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 311 k 2024-04-20T01:42:43.568 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 1.4 M 2024-04-20T01:42:43.568 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-20T01:42:43.736 INFO:teuthology.orchestra.run.smithi044.stdout:(1/2): python3-packaging-16.8-9.el8.noarch.rpm 288 kB/s | 48 kB 00:00 2024-04-20T01:42:43.903 INFO:teuthology.orchestra.run.smithi044.stdout:(2/2): ceph-volume-18.2.2-1252.g15eecde2.el8.no 786 kB/s | 263 kB 00:00 2024-04-20T01:42:43.904 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:42:43.904 INFO:teuthology.orchestra.run.smithi044.stdout:Total 924 kB/s | 311 kB 00:00 2024-04-20T01:42:43.919 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-20T01:42:43.933 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-20T01:42:43.934 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-20T01:42:43.980 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-20T01:42:43.990 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-20T01:42:44.169 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-20T01:42:44.508 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-packaging-16.8-9.el8.noarch 1/2 2024-04-20T01:42:44.751 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 2/2 2024-04-20T01:42:45.340 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 2/2 2024-04-20T01:42:45.340 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 1/2 2024-04-20T01:42:46.137 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 2/2 2024-04-20T01:42:46.137 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2024-04-20T01:42:46.692 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:42:46.692 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-04-20T01:42:46.692 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:42:46.692 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging-16.8-9.el8.noarch 2024-04-20T01:42:46.693 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:42:46.693 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:42:46.833 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install librados-devel 2024-04-20T01:42:47.193 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:42:49.050 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 kB/s | 4.5 kB 00:00 2024-04-20T01:42:49.661 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 40 kB/s | 4.1 kB 00:00 2024-04-20T01:42:51.133 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 5/5 2024-04-20T01:42:51.133 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2024-04-20T01:42:51.271 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 2/2 2024-04-20T01:42:51.271 INFO:teuthology.orchestra.run.smithi165.stdout:Installed products updated. 2024-04-20T01:42:51.694 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:42:51.694 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-20T01:42:51.694 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:42:51.695 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-04-20T01:42:51.695 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.5.1-7.el8.noarch 2024-04-20T01:42:51.695 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-04-20T01:42:51.695 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-04-20T01:42:51.695 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:42:51.695 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:42:51.816 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:42:51.816 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-20T01:42:51.816 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:42:51.816 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging-16.8-9.el8.noarch 2024-04-20T01:42:51.817 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:42:51.817 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:42:51.858 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-fuse 2024-04-20T01:42:52.000 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:42:52.001 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install librados-devel 2024-04-20T01:42:52.001 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:42:52.002 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repo Size 2024-04-20T01:42:52.002 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:42:52.002 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-04-20T01:42:52.002 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 139 k 2024-04-20T01:42:52.002 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:42:52.002 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-20T01:42:52.002 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:42:52.002 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-04-20T01:42:52.002 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:42:52.003 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 139 k 2024-04-20T01:42:52.003 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 513 k 2024-04-20T01:42:52.003 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-20T01:42:52.214 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:42:52.334 INFO:teuthology.orchestra.run.smithi121.stdout:librados-devel-18.2.2-1252.g15eecde2.el8.x86_64 418 kB/s | 139 kB 00:00 2024-04-20T01:42:52.335 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:42:52.335 INFO:teuthology.orchestra.run.smithi121.stdout:Total 416 kB/s | 139 kB 00:00 2024-04-20T01:42:52.335 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-20T01:42:52.341 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-20T01:42:52.341 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-20T01:42:52.362 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:42:52.398 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-20T01:42:52.406 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-20T01:42:52.539 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-20T01:42:52.808 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:42:53.122 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:42:54.466 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 30 kB/s | 4.5 kB 00:00 2024-04-20T01:42:54.495 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 kB/s | 4.5 kB 00:00 2024-04-20T01:42:55.039 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 56 kB/s | 4.1 kB 00:00 2024-04-20T01:42:55.132 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 45 kB/s | 4.1 kB 00:00 2024-04-20T01:42:57.358 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:42:57.359 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:42:57.359 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-04-20T01:42:57.359 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:42:57.359 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-20T01:42:57.359 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse 2024-04-20T01:42:57.359 INFO:teuthology.orchestra.run.smithi123.stdout: x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 859 k 2024-04-20T01:42:57.359 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-04-20T01:42:57.359 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-04-20T01:42:57.359 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:42:57.359 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-20T01:42:57.359 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:42:57.359 INFO:teuthology.orchestra.run.smithi123.stdout:Install 2 Packages 2024-04-20T01:42:57.360 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:42:57.360 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 942 k 2024-04-20T01:42:57.360 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 3.0 M 2024-04-20T01:42:57.360 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-20T01:42:57.480 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:42:57.481 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:42:57.481 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repo Size 2024-04-20T01:42:57.481 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:42:57.481 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-20T01:42:57.481 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 139 k 2024-04-20T01:42:57.481 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:42:57.481 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-20T01:42:57.482 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:42:57.482 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-04-20T01:42:57.482 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:42:57.482 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 139 k 2024-04-20T01:42:57.482 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 513 k 2024-04-20T01:42:57.482 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-20T01:42:57.525 INFO:teuthology.orchestra.run.smithi123.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 504 kB/s | 83 kB 00:00 2024-04-20T01:42:57.609 INFO:teuthology.orchestra.run.smithi123.stdout:(2/2): ceph-fuse-18.2.2-1252.g15eecde2.el8.x86_ 3.4 MB/s | 859 kB 00:00 2024-04-20T01:42:57.609 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:42:57.609 INFO:teuthology.orchestra.run.smithi123.stdout:Total 3.7 MB/s | 942 kB 00:00 2024-04-20T01:42:57.621 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-20T01:42:57.643 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-20T01:42:57.643 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-20T01:42:57.666 INFO:teuthology.orchestra.run.smithi165.stdout:librados-devel-18.2.2-1252.g15eecde2.el8.x86_64 757 kB/s | 139 kB 00:00 2024-04-20T01:42:57.667 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:42:57.667 INFO:teuthology.orchestra.run.smithi165.stdout:Total 752 kB/s | 139 kB 00:00 2024-04-20T01:42:57.667 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-20T01:42:57.673 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-20T01:42:57.673 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-20T01:42:57.728 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-20T01:42:57.730 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-20T01:42:57.737 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-20T01:42:57.739 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-20T01:42:57.877 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-20T01:42:57.905 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-20T01:42:58.116 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:42:58.199 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-04-20T01:42:58.382 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:42:58.406 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 2/2 2024-04-20T01:42:58.775 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 2/2 2024-04-20T01:42:58.775 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/2 2024-04-20T01:42:59.331 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 2/2 2024-04-20T01:42:59.331 INFO:teuthology.orchestra.run.smithi044.stdout:Installed products updated. 2024-04-20T01:42:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:42:59.968 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-20T01:42:59.968 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:42:59.969 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging-16.8-9.el8.noarch 2024-04-20T01:42:59.969 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:42:59.969 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:43:00.123 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install librados-devel 2024-04-20T01:43:00.539 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:02.400 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 20 kB/s | 4.1 kB 00:00 2024-04-20T01:43:02.769 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 kB/s | 4.5 kB 00:00 2024-04-20T01:43:05.500 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:43:05.501 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:43:05.501 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repo Size 2024-04-20T01:43:05.501 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:43:05.501 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-20T01:43:05.501 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 139 k 2024-04-20T01:43:05.501 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:43:05.501 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-20T01:43:05.501 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:43:05.501 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-04-20T01:43:05.501 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:43:05.502 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 139 k 2024-04-20T01:43:05.502 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 513 k 2024-04-20T01:43:05.502 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-20T01:43:05.786 INFO:teuthology.orchestra.run.smithi044.stdout:librados-devel-18.2.2-1252.g15eecde2.el8.x86_64 490 kB/s | 139 kB 00:00 2024-04-20T01:43:05.787 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:43:05.787 INFO:teuthology.orchestra.run.smithi044.stdout:Total 488 kB/s | 139 kB 00:00 2024-04-20T01:43:05.787 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-20T01:43:05.794 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-20T01:43:05.794 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-20T01:43:05.864 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-20T01:43:05.873 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-20T01:43:06.017 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-20T01:43:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:43:06.542 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:43:07.859 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:43:07.859 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2024-04-20T01:43:08.425 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:43:08.425 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-04-20T01:43:08.425 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:43:08.425 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:43:08.425 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:43:08.580 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install libcephfs2 2024-04-20T01:43:08.939 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:11.083 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 kB/s | 4.5 kB 00:00 2024-04-20T01:43:11.750 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 28 kB/s | 4.1 kB 00:00 2024-04-20T01:43:11.953 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:43:11.953 INFO:teuthology.orchestra.run.smithi165.stdout:Installed products updated. 2024-04-20T01:43:12.522 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:43:12.522 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-20T01:43:12.522 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:43:12.522 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:43:12.523 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:43:12.678 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install libcephfs2 2024-04-20T01:43:13.039 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:13.387 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-04-20T01:43:13.387 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2024-04-20T01:43:13.878 INFO:teuthology.orchestra.run.smithi121.stdout:Package libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:43:13.939 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:43:13.939 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-20T01:43:13.939 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-04-20T01:43:13.939 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:43:13.939 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:43:14.067 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:43:14.068 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-04-20T01:43:14.068 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:43:14.116 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-volume 2024-04-20T01:43:14.200 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install libcephfs-devel 2024-04-20T01:43:14.477 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:14.557 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:15.494 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 32 kB/s | 4.5 kB 00:00 2024-04-20T01:43:16.154 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 37 kB/s | 4.1 kB 00:00 2024-04-20T01:43:16.952 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 kB/s | 4.5 kB 00:00 2024-04-20T01:43:17.093 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 52 kB/s | 4.5 kB 00:00 2024-04-20T01:43:17.594 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 25 kB/s | 4.1 kB 00:00 2024-04-20T01:43:17.862 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 16 kB/s | 4.1 kB 00:00 2024-04-20T01:43:18.266 INFO:teuthology.orchestra.run.smithi165.stdout:Package libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:43:18.456 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:43:18.457 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-20T01:43:18.458 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:43:18.597 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install libcephfs-devel 2024-04-20T01:43:18.954 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:19.906 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:43:19.907 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================================== 2024-04-20T01:43:19.907 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-04-20T01:43:19.907 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================================== 2024-04-20T01:43:19.907 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-20T01:43:19.907 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:18.2.2-1252.g15eecde2.el8 ceph-noarch 263 k 2024-04-20T01:43:19.907 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-04-20T01:43:19.907 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-04-20T01:43:19.907 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:43:19.907 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-20T01:43:19.907 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================================== 2024-04-20T01:43:19.908 INFO:teuthology.orchestra.run.smithi123.stdout:Install 2 Packages 2024-04-20T01:43:19.908 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:43:19.908 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 311 k 2024-04-20T01:43:19.908 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 1.4 M 2024-04-20T01:43:19.908 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-20T01:43:20.047 INFO:teuthology.orchestra.run.smithi123.stdout:(1/2): python3-packaging-16.8-9.el8.noarch.rpm 348 kB/s | 48 kB 00:00 2024-04-20T01:43:20.105 INFO:teuthology.orchestra.run.smithi123.stdout:(2/2): ceph-volume-18.2.2-1252.g15eecde2.el8.no 1.3 MB/s | 263 kB 00:00 2024-04-20T01:43:20.106 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:43:20.106 INFO:teuthology.orchestra.run.smithi123.stdout:Total 1.5 MB/s | 311 kB 00:00 2024-04-20T01:43:20.117 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-20T01:43:20.129 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-20T01:43:20.129 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-20T01:43:20.166 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-20T01:43:20.175 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-20T01:43:20.279 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:43:20.280 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:43:20.280 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repo Size 2024-04-20T01:43:20.280 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:43:20.280 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-04-20T01:43:20.280 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 31 k 2024-04-20T01:43:20.280 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:43:20.280 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-20T01:43:20.280 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:43:20.280 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-04-20T01:43:20.280 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:43:20.281 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 31 k 2024-04-20T01:43:20.281 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 139 k 2024-04-20T01:43:20.281 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-20T01:43:20.341 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-20T01:43:20.559 INFO:teuthology.orchestra.run.smithi121.stdout:libcephfs-devel-18.2.2-1252.g15eecde2.el8.x86_6 111 kB/s | 31 kB 00:00 2024-04-20T01:43:20.559 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:43:20.559 INFO:teuthology.orchestra.run.smithi121.stdout:Total 111 kB/s | 31 kB 00:00 2024-04-20T01:43:20.560 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-20T01:43:20.563 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-20T01:43:20.563 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-20T01:43:20.576 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-20T01:43:20.584 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-20T01:43:20.652 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-20T01:43:20.660 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-packaging-16.8-9.el8.noarch 1/2 2024-04-20T01:43:20.885 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:43:20.953 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 2/2 2024-04-20T01:43:21.034 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 kB/s | 4.5 kB 00:00 2024-04-20T01:43:21.118 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:43:21.349 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 2/2 2024-04-20T01:43:21.349 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 1/2 2024-04-20T01:43:21.732 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:43:21.732 INFO:teuthology.orchestra.run.smithi044.stdout:Installed products updated. 2024-04-20T01:43:21.793 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 19 kB/s | 4.1 kB 00:00 2024-04-20T01:43:22.348 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:43:22.349 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-20T01:43:22.349 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:43:22.349 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:43:22.349 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:43:22.512 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install libcephfs2 2024-04-20T01:43:22.928 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:24.200 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:43:24.201 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:43:24.201 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repo Size 2024-04-20T01:43:24.201 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:43:24.201 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-20T01:43:24.201 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 31 k 2024-04-20T01:43:24.201 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:43:24.201 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-20T01:43:24.201 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:43:24.201 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-04-20T01:43:24.201 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:43:24.201 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 31 k 2024-04-20T01:43:24.201 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 139 k 2024-04-20T01:43:24.202 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-20T01:43:24.343 INFO:teuthology.orchestra.run.smithi165.stdout:libcephfs-devel-18.2.2-1252.g15eecde2.el8.x86_6 219 kB/s | 31 kB 00:00 2024-04-20T01:43:24.343 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:43:24.343 INFO:teuthology.orchestra.run.smithi165.stdout:Total 217 kB/s | 31 kB 00:00 2024-04-20T01:43:24.344 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-20T01:43:24.347 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-20T01:43:24.348 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-20T01:43:24.359 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-20T01:43:24.368 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-20T01:43:24.429 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-20T01:43:24.659 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:43:24.912 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:43:25.134 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 40 kB/s | 4.1 kB 00:00 2024-04-20T01:43:25.557 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 24 kB/s | 4.5 kB 00:00 2024-04-20T01:43:28.038 INFO:teuthology.orchestra.run.smithi044.stdout:Package libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:43:28.250 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:43:28.251 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-20T01:43:28.251 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:43:28.391 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install libcephfs-devel 2024-04-20T01:43:28.804 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:31.277 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 30 kB/s | 4.1 kB 00:00 2024-04-20T01:43:31.624 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 kB/s | 4.5 kB 00:00 2024-04-20T01:43:33.932 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 2/2 2024-04-20T01:43:33.932 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2024-04-20T01:43:34.364 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:43:34.365 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:43:34.365 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repo Size 2024-04-20T01:43:34.365 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:43:34.365 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-20T01:43:34.365 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 31 k 2024-04-20T01:43:34.365 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:43:34.365 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-20T01:43:34.365 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:43:34.365 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-04-20T01:43:34.365 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:43:34.366 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 31 k 2024-04-20T01:43:34.366 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 139 k 2024-04-20T01:43:34.366 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-20T01:43:34.484 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:43:34.484 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-20T01:43:34.484 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:18.2.2-1252.g15eecde2.el8.noarch 2024-04-20T01:43:34.484 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging-16.8-9.el8.noarch 2024-04-20T01:43:34.484 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:43:34.484 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:43:34.507 INFO:teuthology.orchestra.run.smithi044.stdout:libcephfs-devel-18.2.2-1252.g15eecde2.el8.x86_6 220 kB/s | 31 kB 00:00 2024-04-20T01:43:34.507 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:43:34.508 INFO:teuthology.orchestra.run.smithi044.stdout:Total 218 kB/s | 31 kB 00:00 2024-04-20T01:43:34.508 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-20T01:43:34.512 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-20T01:43:34.513 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-20T01:43:34.528 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-20T01:43:34.537 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-20T01:43:34.598 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-20T01:43:34.629 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install librados-devel 2024-04-20T01:43:34.843 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:43:34.990 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:35.124 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:43:35.948 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:43:35.949 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2024-04-20T01:43:36.563 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:43:36.563 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-04-20T01:43:36.563 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:43:36.563 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:43:36.563 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:43:36.715 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install librados2 2024-04-20T01:43:36.903 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:43:36.903 INFO:teuthology.orchestra.run.smithi165.stdout:Installed products updated. 2024-04-20T01:43:37.072 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:37.215 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 40 kB/s | 4.5 kB 00:00 2024-04-20T01:43:37.461 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:43:37.461 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-20T01:43:37.461 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:43:37.461 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:43:37.461 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:43:37.614 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install librados2 2024-04-20T01:43:37.780 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 43 kB/s | 4.1 kB 00:00 2024-04-20T01:43:37.976 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:38.882 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 26 kB/s | 4.5 kB 00:00 2024-04-20T01:43:39.506 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 36 kB/s | 4.1 kB 00:00 2024-04-20T01:43:40.019 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-04-20T01:43:40.078 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:43:40.079 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:43:40.079 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repo Size 2024-04-20T01:43:40.079 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:43:40.079 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-20T01:43:40.079 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 139 k 2024-04-20T01:43:40.079 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:43:40.079 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-20T01:43:40.079 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:43:40.079 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-04-20T01:43:40.079 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:43:40.080 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 139 k 2024-04-20T01:43:40.080 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 513 k 2024-04-20T01:43:40.080 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-20T01:43:40.400 INFO:teuthology.orchestra.run.smithi123.stdout:librados-devel-18.2.2-1252.g15eecde2.el8.x86_64 434 kB/s | 139 kB 00:00 2024-04-20T01:43:40.401 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:43:40.401 INFO:teuthology.orchestra.run.smithi123.stdout:Total 432 kB/s | 139 kB 00:00 2024-04-20T01:43:40.402 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-20T01:43:40.407 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-20T01:43:40.407 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-20T01:43:40.462 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-20T01:43:40.471 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-20T01:43:40.606 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-20T01:43:40.726 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 27 kB/s | 4.1 kB 00:00 2024-04-20T01:43:40.844 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:43:41.106 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:43:41.642 INFO:teuthology.orchestra.run.smithi121.stdout:Package librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:43:41.830 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:43:41.831 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-04-20T01:43:41.831 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:43:41.967 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install librbd1 2024-04-20T01:43:42.325 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:42.970 INFO:teuthology.orchestra.run.smithi165.stdout:Package librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:43:43.159 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:43:43.160 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-20T01:43:43.160 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:43:43.301 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install librbd1 2024-04-20T01:43:43.660 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:44.070 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-04-20T01:43:44.703 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 35 kB/s | 4.1 kB 00:00 2024-04-20T01:43:45.467 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 29 kB/s | 4.5 kB 00:00 2024-04-20T01:43:46.243 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 18 kB/s | 4.1 kB 00:00 2024-04-20T01:43:46.822 INFO:teuthology.orchestra.run.smithi121.stdout:Package librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:43:47.015 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:43:47.016 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-04-20T01:43:47.017 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:43:47.157 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install python3-rados 2024-04-20T01:43:47.515 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:48.367 INFO:teuthology.orchestra.run.smithi165.stdout:Package librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:43:48.559 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:43:48.561 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-20T01:43:48.561 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:43:48.704 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install python3-rados 2024-04-20T01:43:49.063 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:49.229 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 53 kB/s | 4.5 kB 00:00 2024-04-20T01:43:49.866 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 33 kB/s | 4.1 kB 00:00 2024-04-20T01:43:50.926 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 20 kB/s | 4.5 kB 00:00 2024-04-20T01:43:51.049 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:43:51.049 INFO:teuthology.orchestra.run.smithi044.stdout:Installed products updated. 2024-04-20T01:43:51.623 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 28 kB/s | 4.1 kB 00:00 2024-04-20T01:43:51.671 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:43:51.671 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-20T01:43:51.672 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:43:51.672 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:43:51.672 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:43:51.834 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install librados2 2024-04-20T01:43:52.001 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:43:52.191 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:43:52.192 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-04-20T01:43:52.192 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:43:52.246 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:52.326 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install python3-rgw 2024-04-20T01:43:52.684 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:53.728 INFO:teuthology.orchestra.run.smithi165.stdout:Package python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:43:53.919 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:43:53.920 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-20T01:43:53.920 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:43:54.067 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install python3-rgw 2024-04-20T01:43:54.244 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 24 kB/s | 4.1 kB 00:00 2024-04-20T01:43:54.320 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:43:54.320 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2024-04-20T01:43:54.427 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:54.452 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 32 kB/s | 4.5 kB 00:00 2024-04-20T01:43:54.606 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 kB/s | 4.5 kB 00:00 2024-04-20T01:43:54.874 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:43:54.875 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-20T01:43:54.875 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:43:54.875 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:43:54.875 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:43:55.041 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install libcephfs2 2024-04-20T01:43:55.120 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 27 kB/s | 4.1 kB 00:00 2024-04-20T01:43:55.398 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:56.253 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 50 kB/s | 4.5 kB 00:00 2024-04-20T01:43:56.969 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 23 kB/s | 4.1 kB 00:00 2024-04-20T01:43:57.159 INFO:teuthology.orchestra.run.smithi044.stdout:Package librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:43:57.262 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:43:57.373 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:43:57.375 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-20T01:43:57.375 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:43:57.378 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 kB/s | 4.5 kB 00:00 2024-04-20T01:43:57.452 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:43:57.453 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-04-20T01:43:57.453 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:43:57.518 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install librbd1 2024-04-20T01:43:57.583 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install python3-cephfs 2024-04-20T01:43:57.930 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:57.941 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:43:57.947 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 41 kB/s | 4.1 kB 00:00 2024-04-20T01:43:59.172 INFO:teuthology.orchestra.run.smithi165.stdout:Package python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:43:59.363 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:43:59.364 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-20T01:43:59.364 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:43:59.501 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install python3-cephfs 2024-04-20T01:43:59.723 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 28 kB/s | 4.5 kB 00:00 2024-04-20T01:43:59.862 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:00.060 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 26 kB/s | 4.1 kB 00:00 2024-04-20T01:44:00.160 INFO:teuthology.orchestra.run.smithi123.stdout:Package libcephfs2-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:44:00.349 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:44:00.350 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-04-20T01:44:00.350 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:44:00.447 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 24 kB/s | 4.1 kB 00:00 2024-04-20T01:44:00.484 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 27 kB/s | 4.5 kB 00:00 2024-04-20T01:44:00.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install libcephfs-devel 2024-04-20T01:44:00.851 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:01.800 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 25 kB/s | 4.5 kB 00:00 2024-04-20T01:44:02.460 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 38 kB/s | 4.1 kB 00:00 2024-04-20T01:44:02.692 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:44:02.880 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:44:02.881 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-04-20T01:44:02.881 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:44:02.894 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 54 kB/s | 4.5 kB 00:00 2024-04-20T01:44:02.992 INFO:teuthology.orchestra.run.smithi044.stdout:Package librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:44:03.022 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install python3-rbd 2024-04-20T01:44:03.209 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:44:03.211 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-20T01:44:03.211 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:44:03.361 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install python3-rados 2024-04-20T01:44:03.380 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:03.512 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 33 kB/s | 4.1 kB 00:00 2024-04-20T01:44:03.775 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:04.586 INFO:teuthology.orchestra.run.smithi165.stdout:Package python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:44:04.776 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:44:04.778 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-20T01:44:04.778 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:44:04.925 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install python3-rbd 2024-04-20T01:44:05.094 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 kB/s | 4.5 kB 00:00 2024-04-20T01:44:05.286 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:05.768 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 26 kB/s | 4.1 kB 00:00 2024-04-20T01:44:05.815 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 25 kB/s | 4.1 kB 00:00 2024-04-20T01:44:05.818 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:44:05.819 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:44:05.819 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repo Size 2024-04-20T01:44:05.819 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:44:05.819 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-20T01:44:05.819 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 31 k 2024-04-20T01:44:05.819 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:44:05.819 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-20T01:44:05.819 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:44:05.819 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-04-20T01:44:05.820 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:44:05.820 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 31 k 2024-04-20T01:44:05.820 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 139 k 2024-04-20T01:44:05.820 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-20T01:44:06.047 INFO:teuthology.orchestra.run.smithi123.stdout:libcephfs-devel-18.2.2-1252.g15eecde2.el8.x86_6 136 kB/s | 31 kB 00:00 2024-04-20T01:44:06.048 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:44:06.048 INFO:teuthology.orchestra.run.smithi123.stdout:Total 135 kB/s | 31 kB 00:00 2024-04-20T01:44:06.048 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-20T01:44:06.052 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-20T01:44:06.052 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-20T01:44:06.064 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-20T01:44:06.073 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-20T01:44:06.130 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-20T01:44:06.195 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 30 kB/s | 4.5 kB 00:00 2024-04-20T01:44:06.361 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:06.591 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:07.220 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 kB/s | 4.5 kB 00:00 2024-04-20T01:44:07.917 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 28 kB/s | 4.1 kB 00:00 2024-04-20T01:44:07.930 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:44:08.118 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:44:08.119 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-04-20T01:44:08.119 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:44:08.288 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install rbd-fuse 2024-04-20T01:44:08.645 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:08.812 INFO:teuthology.orchestra.run.smithi044.stdout:Package python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:44:09.025 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:44:09.026 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-20T01:44:09.027 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:44:09.176 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install python3-rgw 2024-04-20T01:44:09.591 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:10.061 INFO:teuthology.orchestra.run.smithi165.stdout:Package python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:44:10.250 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:44:10.251 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-20T01:44:10.252 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:44:10.390 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install rbd-fuse 2024-04-20T01:44:10.400 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 kB/s | 4.5 kB 00:00 2024-04-20T01:44:10.748 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:11.105 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 22 kB/s | 4.1 kB 00:00 2024-04-20T01:44:11.483 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 42 kB/s | 4.1 kB 00:00 2024-04-20T01:44:11.831 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 41 kB/s | 4.5 kB 00:00 2024-04-20T01:44:12.576 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 40 kB/s | 4.5 kB 00:00 2024-04-20T01:44:13.244 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 35 kB/s | 4.1 kB 00:00 2024-04-20T01:44:13.468 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:44:13.468 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:44:13.468 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2024-04-20T01:44:13.469 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:44:13.469 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-04-20T01:44:13.469 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 95 k 2024-04-20T01:44:13.469 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:44:13.469 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-20T01:44:13.469 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:44:13.469 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-04-20T01:44:13.469 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:44:13.470 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 95 k 2024-04-20T01:44:13.470 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 280 k 2024-04-20T01:44:13.470 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-20T01:44:13.761 INFO:teuthology.orchestra.run.smithi121.stdout:rbd-fuse-18.2.2-1252.g15eecde2.el8.x86_64.rpm 327 kB/s | 95 kB 00:00 2024-04-20T01:44:13.761 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:44:13.761 INFO:teuthology.orchestra.run.smithi121.stdout:Total 326 kB/s | 95 kB 00:00 2024-04-20T01:44:13.762 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-20T01:44:13.772 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-20T01:44:13.772 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-20T01:44:13.828 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-20T01:44:13.837 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-20T01:44:13.969 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-20T01:44:14.206 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:14.370 INFO:teuthology.orchestra.run.smithi044.stdout:Package python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:44:14.477 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:14.580 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:44:14.581 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-20T01:44:14.581 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:44:14.723 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install python3-cephfs 2024-04-20T01:44:15.135 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:15.544 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:44:15.545 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:44:15.545 INFO:teuthology.orchestra.run.smithi165.stdout: Package Architecture Version Repository Size 2024-04-20T01:44:15.545 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:44:15.545 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-20T01:44:15.545 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 95 k 2024-04-20T01:44:15.545 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:44:15.545 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-20T01:44:15.545 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:44:15.545 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-04-20T01:44:15.545 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:44:15.546 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 95 k 2024-04-20T01:44:15.546 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 280 k 2024-04-20T01:44:15.546 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-20T01:44:15.737 INFO:teuthology.orchestra.run.smithi165.stdout:rbd-fuse-18.2.2-1252.g15eecde2.el8.x86_64.rpm 498 kB/s | 95 kB 00:00 2024-04-20T01:44:15.738 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:44:15.738 INFO:teuthology.orchestra.run.smithi165.stdout:Total 495 kB/s | 95 kB 00:00 2024-04-20T01:44:15.738 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-20T01:44:15.748 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-20T01:44:15.748 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-20T01:44:15.805 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-20T01:44:15.813 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-20T01:44:15.926 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-20T01:44:16.083 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:16.374 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:17.035 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 42 kB/s | 4.1 kB 00:00 2024-04-20T01:44:17.426 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 kB/s | 4.5 kB 00:00 2024-04-20T01:44:19.153 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:19.153 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2024-04-20T01:44:19.714 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:44:19.714 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-20T01:44:19.714 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:44:19.714 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:44:19.714 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:44:19.892 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install librados2 2024-04-20T01:44:19.973 INFO:teuthology.orchestra.run.smithi044.stdout:Package python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:44:20.186 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:44:20.188 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-20T01:44:20.188 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:44:20.252 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:20.342 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install python3-rbd 2024-04-20T01:44:20.756 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:22.298 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 55 kB/s | 4.5 kB 00:00 2024-04-20T01:44:22.914 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 31 kB/s | 4.1 kB 00:00 2024-04-20T01:44:23.001 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 25 kB/s | 4.1 kB 00:00 2024-04-20T01:44:23.381 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-04-20T01:44:25.041 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:44:25.060 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:25.060 INFO:teuthology.orchestra.run.smithi165.stdout:Installed products updated. 2024-04-20T01:44:25.231 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:44:25.232 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-04-20T01:44:25.232 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:44:25.382 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install librbd1 2024-04-20T01:44:25.621 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:44:25.621 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-20T01:44:25.621 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:44:25.622 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:44:25.622 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:44:25.741 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:25.781 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install rbd-mirror 2024-04-20T01:44:25.865 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:25.865 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2024-04-20T01:44:25.978 INFO:teuthology.orchestra.run.smithi044.stdout:Package python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:44:26.143 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:26.191 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:44:26.192 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-20T01:44:26.193 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:44:26.369 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install rbd-fuse 2024-04-20T01:44:26.436 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:44:26.436 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-04-20T01:44:26.436 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:44:26.436 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:44:26.436 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:44:26.582 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install rbd-mirror 2024-04-20T01:44:26.785 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:26.941 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:29.889 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 kB/s | 4.5 kB 00:00 2024-04-20T01:44:29.989 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 kB/s | 4.5 kB 00:00 2024-04-20T01:44:30.111 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 29 kB/s | 4.5 kB 00:00 2024-04-20T01:44:30.307 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 32 kB/s | 4.1 kB 00:00 2024-04-20T01:44:30.562 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 26 kB/s | 4.1 kB 00:00 2024-04-20T01:44:30.580 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 36 kB/s | 4.1 kB 00:00 2024-04-20T01:44:30.730 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 24 kB/s | 4.5 kB 00:00 2024-04-20T01:44:30.800 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 30 kB/s | 4.1 kB 00:00 2024-04-20T01:44:32.806 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:44:32.923 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:44:32.923 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:44:32.923 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repo Size 2024-04-20T01:44:32.924 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:44:32.924 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-04-20T01:44:32.924 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 3.5 M 2024-04-20T01:44:32.924 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:44:32.924 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-20T01:44:32.924 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:44:32.924 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-04-20T01:44:32.924 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:44:32.925 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 3.5 M 2024-04-20T01:44:32.925 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 16 M 2024-04-20T01:44:32.925 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-20T01:44:32.995 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:44:32.996 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-04-20T01:44:32.996 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:44:33.109 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:44:33.110 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:44:33.110 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repo Size 2024-04-20T01:44:33.110 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:44:33.110 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-20T01:44:33.110 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 3.5 M 2024-04-20T01:44:33.110 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:44:33.110 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-20T01:44:33.111 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:44:33.111 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-04-20T01:44:33.111 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:44:33.111 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 3.5 M 2024-04-20T01:44:33.111 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 16 M 2024-04-20T01:44:33.111 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-20T01:44:33.141 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install python3-rados 2024-04-20T01:44:33.497 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:33.522 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:44:33.522 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:44:33.522 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-04-20T01:44:33.522 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:44:33.522 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-20T01:44:33.523 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 95 k 2024-04-20T01:44:33.523 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:44:33.523 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-20T01:44:33.523 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:44:33.523 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-04-20T01:44:33.523 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:44:33.524 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 95 k 2024-04-20T01:44:33.524 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 280 k 2024-04-20T01:44:33.524 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-20T01:44:33.928 INFO:teuthology.orchestra.run.smithi121.stdout:rbd-mirror-18.2.2-1252.g15eecde2.el8.x86_64.rpm 3.4 MB/s | 3.5 MB 00:01 2024-04-20T01:44:33.928 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:44:33.928 INFO:teuthology.orchestra.run.smithi121.stdout:Total 3.4 MB/s | 3.5 MB 00:01 2024-04-20T01:44:33.929 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-20T01:44:33.941 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-20T01:44:33.941 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-20T01:44:34.009 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-20T01:44:34.018 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-20T01:44:34.080 INFO:teuthology.orchestra.run.smithi165.stdout:rbd-mirror-18.2.2-1252.g15eecde2.el8.x86_64.rpm 3.6 MB/s | 3.5 MB 00:00 2024-04-20T01:44:34.080 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:44:34.080 INFO:teuthology.orchestra.run.smithi165.stdout:Total 3.6 MB/s | 3.5 MB 00:00 2024-04-20T01:44:34.081 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-20T01:44:34.093 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-20T01:44:34.093 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-20T01:44:34.163 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-20T01:44:34.171 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-20T01:44:34.472 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-20T01:44:34.620 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-20T01:44:34.662 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:34.814 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:35.047 INFO:teuthology.orchestra.run.smithi044.stdout:rbd-fuse-18.2.2-1252.g15eecde2.el8.x86_64.rpm 62 kB/s | 95 kB 00:01 2024-04-20T01:44:35.047 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:44:35.047 INFO:teuthology.orchestra.run.smithi044.stdout:Total 62 kB/s | 95 kB 00:01 2024-04-20T01:44:35.048 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-20T01:44:35.061 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-20T01:44:35.061 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-20T01:44:35.068 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:35.129 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-20T01:44:35.139 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-20T01:44:35.230 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:35.265 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-20T01:44:35.276 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 22 kB/s | 4.5 kB 00:00 2024-04-20T01:44:35.418 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:35.716 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:35.941 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 22 kB/s | 4.1 kB 00:00 2024-04-20T01:44:38.048 INFO:teuthology.orchestra.run.smithi123.stdout:Package python3-rados-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:44:38.238 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:44:38.239 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-04-20T01:44:38.239 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:44:38.384 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install python3-rgw 2024-04-20T01:44:38.742 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:40.760 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 38 kB/s | 4.5 kB 00:00 2024-04-20T01:44:41.350 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 35 kB/s | 4.1 kB 00:00 2024-04-20T01:44:43.465 INFO:teuthology.orchestra.run.smithi123.stdout:Package python3-rgw-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:44:43.654 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:44:43.655 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-04-20T01:44:43.655 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:44:43.805 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install python3-cephfs 2024-04-20T01:44:44.165 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:46.200 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-04-20T01:44:46.806 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 31 kB/s | 4.1 kB 00:00 2024-04-20T01:44:46.896 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:46.897 INFO:teuthology.orchestra.run.smithi044.stdout:Installed products updated. 2024-04-20T01:44:47.300 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:47.300 INFO:teuthology.orchestra.run.smithi165.stdout:Installed products updated. 2024-04-20T01:44:47.526 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:44:47.527 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-20T01:44:47.527 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:44:47.527 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:44:47.527 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:44:47.703 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install rbd-mirror 2024-04-20T01:44:47.866 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:44:47.866 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-20T01:44:47.867 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:44:47.867 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:44:47.867 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:44:48.093 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install rbd-nbd 2024-04-20T01:44:48.118 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:48.455 INFO:teuthology.orchestra.run.smithi165.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:48.504 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:48.504 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2024-04-20T01:44:48.945 INFO:teuthology.orchestra.run.smithi123.stdout:Package python3-cephfs-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:44:49.106 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:44:49.106 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-04-20T01:44:49.107 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:44:49.107 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:44:49.107 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:44:49.137 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:44:49.138 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-04-20T01:44:49.138 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:44:49.269 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install rbd-nbd 2024-04-20T01:44:49.296 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install python3-rbd 2024-04-20T01:44:49.629 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:49.653 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:50.146 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 kB/s | 4.1 kB 00:00 2024-04-20T01:44:50.346 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 31 kB/s | 4.5 kB 00:00 2024-04-20T01:44:50.507 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 kB/s | 4.5 kB 00:00 2024-04-20T01:44:50.941 INFO:teuthology.orchestra.run.smithi165.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 80 kB/s | 4.1 kB 00:00 2024-04-20T01:44:51.517 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 72 kB/s | 4.5 kB 00:00 2024-04-20T01:44:51.575 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 27 kB/s | 4.5 kB 00:00 2024-04-20T01:44:52.224 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 28 kB/s | 4.1 kB 00:00 2024-04-20T01:44:52.226 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 24 kB/s | 4.1 kB 00:00 2024-04-20T01:44:53.263 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-20T01:44:53.263 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:44:53.264 INFO:teuthology.orchestra.run.smithi165.stdout: Package Architecture Version Repository Size 2024-04-20T01:44:53.264 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:44:53.264 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-20T01:44:53.264 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 182 k 2024-04-20T01:44:53.264 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:44:53.264 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-20T01:44:53.264 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-20T01:44:53.264 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-04-20T01:44:53.264 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:44:53.265 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 182 k 2024-04-20T01:44:53.265 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 572 k 2024-04-20T01:44:53.265 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-20T01:44:53.268 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:44:53.269 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:44:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repo Size 2024-04-20T01:44:53.269 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:44:53.269 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-20T01:44:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 3.5 M 2024-04-20T01:44:53.269 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:44:53.270 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-20T01:44:53.270 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:44:53.270 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-04-20T01:44:53.270 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:44:53.270 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 3.5 M 2024-04-20T01:44:53.270 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 16 M 2024-04-20T01:44:53.271 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-20T01:44:53.565 INFO:teuthology.orchestra.run.smithi165.stdout:rbd-nbd-18.2.2-1252.g15eecde2.el8.x86_64.rpm 606 kB/s | 182 kB 00:00 2024-04-20T01:44:53.565 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:44:53.566 INFO:teuthology.orchestra.run.smithi165.stdout:Total 603 kB/s | 182 kB 00:00 2024-04-20T01:44:53.566 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-20T01:44:53.577 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-20T01:44:53.577 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-20T01:44:53.633 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-20T01:44:53.641 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-20T01:44:53.770 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-20T01:44:53.917 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:54.025 INFO:teuthology.orchestra.run.smithi044.stdout:rbd-mirror-18.2.2-1252.g15eecde2.el8.x86_64.rpm 4.6 MB/s | 3.5 MB 00:00 2024-04-20T01:44:54.026 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:44:54.026 INFO:teuthology.orchestra.run.smithi044.stdout:Total 4.6 MB/s | 3.5 MB 00:00 2024-04-20T01:44:54.027 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-20T01:44:54.042 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-20T01:44:54.042 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-20T01:44:54.124 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-20T01:44:54.133 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-20T01:44:54.228 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:54.330 INFO:teuthology.orchestra.run.smithi123.stdout:Package python3-rbd-2:18.2.2-1252.g15eecde2.el8.x86_64 is already installed. 2024-04-20T01:44:54.523 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:44:54.525 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-04-20T01:44:54.525 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:44:54.622 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-20T01:44:54.645 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T01:44:54.645 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:44:54.646 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2024-04-20T01:44:54.646 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:44:54.646 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-04-20T01:44:54.646 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 182 k 2024-04-20T01:44:54.646 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:44:54.646 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-20T01:44:54.646 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T01:44:54.646 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2024-04-20T01:44:54.646 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:44:54.647 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 182 k 2024-04-20T01:44:54.647 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 572 k 2024-04-20T01:44:54.647 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-20T01:44:54.677 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install rbd-fuse 2024-04-20T01:44:54.792 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:54.845 INFO:teuthology.orchestra.run.smithi121.stdout:rbd-nbd-18.2.2-1252.g15eecde2.el8.x86_64.rpm 917 kB/s | 182 kB 00:00 2024-04-20T01:44:54.846 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:44:54.846 INFO:teuthology.orchestra.run.smithi121.stdout:Total 912 kB/s | 182 kB 00:00 2024-04-20T01:44:54.846 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-20T01:44:54.857 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-20T01:44:54.857 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-20T01:44:54.913 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-20T01:44:54.922 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-20T01:44:55.032 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:44:55.045 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-20T01:44:55.214 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:55.262 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:55.516 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:44:57.000 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 38 kB/s | 4.5 kB 00:00 2024-04-20T01:44:57.617 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 27 kB/s | 4.1 kB 00:00 2024-04-20T01:45:00.024 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:45:00.025 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:45:00.025 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2024-04-20T01:45:00.025 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:45:00.025 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-20T01:45:00.025 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 95 k 2024-04-20T01:45:00.025 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:00.026 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-20T01:45:00.026 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:45:00.026 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-04-20T01:45:00.026 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:00.026 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 95 k 2024-04-20T01:45:00.026 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 280 k 2024-04-20T01:45:00.026 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-20T01:45:00.209 INFO:teuthology.orchestra.run.smithi123.stdout:rbd-fuse-18.2.2-1252.g15eecde2.el8.x86_64.rpm 518 kB/s | 95 kB 00:00 2024-04-20T01:45:00.210 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:45:00.210 INFO:teuthology.orchestra.run.smithi123.stdout:Total 515 kB/s | 95 kB 00:00 2024-04-20T01:45:00.210 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-20T01:45:00.221 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-20T01:45:00.221 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-20T01:45:00.276 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-20T01:45:00.285 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-20T01:45:00.399 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-20T01:45:00.610 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:45:00.881 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:45:07.906 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:45:07.907 INFO:teuthology.orchestra.run.smithi165.stdout:Installed products updated. 2024-04-20T01:45:08.473 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:08.473 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-20T01:45:08.473 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:45:08.473 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:08.474 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-20T01:45:08.781 DEBUG:teuthology.parallel:result is None 2024-04-20T01:45:09.352 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:45:09.352 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2024-04-20T01:45:09.445 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:45:09.445 INFO:teuthology.orchestra.run.smithi044.stdout:Installed products updated. 2024-04-20T01:45:09.916 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:09.916 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-04-20T01:45:09.916 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:45:09.916 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:09.917 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T01:45:10.073 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:10.073 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-20T01:45:10.073 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:45:10.073 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:10.073 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:45:10.088 DEBUG:teuthology.parallel:result is None 2024-04-20T01:45:10.312 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install rbd-nbd 2024-04-20T01:45:10.743 INFO:teuthology.orchestra.run.smithi044.stdout:Updating Subscription Management repositories. 2024-04-20T01:45:12.856 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 40 kB/s | 4.1 kB 00:00 2024-04-20T01:45:12.924 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:45:12.925 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2024-04-20T01:45:13.212 INFO:teuthology.orchestra.run.smithi044.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-04-20T01:45:13.484 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:13.484 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-20T01:45:13.484 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:45:13.484 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:13.484 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:45:13.678 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install rbd-mirror 2024-04-20T01:45:14.040 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:45:15.969 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-20T01:45:15.970 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:45:15.970 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-04-20T01:45:15.970 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:45:15.970 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-20T01:45:15.970 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 182 k 2024-04-20T01:45:15.970 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:15.970 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-20T01:45:15.970 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-20T01:45:15.970 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-04-20T01:45:15.971 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:15.971 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 182 k 2024-04-20T01:45:15.971 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 572 k 2024-04-20T01:45:15.971 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-20T01:45:16.154 INFO:teuthology.orchestra.run.smithi044.stdout:rbd-nbd-18.2.2-1252.g15eecde2.el8.x86_64.rpm 997 kB/s | 182 kB 00:00 2024-04-20T01:45:16.154 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:45:16.155 INFO:teuthology.orchestra.run.smithi044.stdout:Total 989 kB/s | 182 kB 00:00 2024-04-20T01:45:16.155 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-20T01:45:16.169 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-20T01:45:16.169 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-20T01:45:16.238 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-20T01:45:16.247 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-20T01:45:16.329 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 kB/s | 4.5 kB 00:00 2024-04-20T01:45:16.388 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-20T01:45:16.566 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:45:16.877 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:45:16.963 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 26 kB/s | 4.1 kB 00:00 2024-04-20T01:45:19.284 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:45:19.285 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:45:19.285 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repo Size 2024-04-20T01:45:19.286 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:45:19.286 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-20T01:45:19.286 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 3.5 M 2024-04-20T01:45:19.286 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:19.286 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-20T01:45:19.287 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:45:19.287 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-04-20T01:45:19.287 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:19.287 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 3.5 M 2024-04-20T01:45:19.287 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 16 M 2024-04-20T01:45:19.287 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-20T01:45:19.870 INFO:teuthology.orchestra.run.smithi123.stdout:rbd-mirror-18.2.2-1252.g15eecde2.el8.x86_64.rpm 5.9 MB/s | 3.5 MB 00:00 2024-04-20T01:45:19.871 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:45:19.871 INFO:teuthology.orchestra.run.smithi123.stdout:Total 5.9 MB/s | 3.5 MB 00:00 2024-04-20T01:45:19.871 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-20T01:45:19.884 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-20T01:45:19.884 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-20T01:45:19.952 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-20T01:45:19.960 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-20T01:45:20.439 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-20T01:45:20.633 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:45:21.046 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:45:30.330 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:45:30.331 INFO:teuthology.orchestra.run.smithi044.stdout:Installed products updated. 2024-04-20T01:45:30.960 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:30.960 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-20T01:45:30.960 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:45:30.960 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:30.961 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-20T01:45:31.139 DEBUG:teuthology.parallel:result is None 2024-04-20T01:45:33.560 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:45:33.560 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2024-04-20T01:45:34.109 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:34.109 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-20T01:45:34.109 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:45:34.109 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:34.109 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:45:34.285 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install rbd-nbd 2024-04-20T01:45:34.648 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2024-04-20T01:45:36.455 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 30 kB/s | 4.5 kB 00:00 2024-04-20T01:45:37.089 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 26 kB/s | 4.1 kB 00:00 2024-04-20T01:45:39.508 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T01:45:39.508 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:45:39.509 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2024-04-20T01:45:39.509 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:45:39.509 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-20T01:45:39.509 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:18.2.2-1252.g15eecde2.el8 ceph 182 k 2024-04-20T01:45:39.509 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:39.509 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-20T01:45:39.509 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-04-20T01:45:39.509 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-04-20T01:45:39.509 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:39.510 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 182 k 2024-04-20T01:45:39.510 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 572 k 2024-04-20T01:45:39.510 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-20T01:45:39.807 INFO:teuthology.orchestra.run.smithi123.stdout:rbd-nbd-18.2.2-1252.g15eecde2.el8.x86_64.rpm 613 kB/s | 182 kB 00:00 2024-04-20T01:45:39.807 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-20T01:45:39.808 INFO:teuthology.orchestra.run.smithi123.stdout:Total 610 kB/s | 182 kB 00:00 2024-04-20T01:45:39.808 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-20T01:45:39.819 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-20T01:45:39.819 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-20T01:45:39.874 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-20T01:45:39.883 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-20T01:45:40.010 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-20T01:45:40.212 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:45:40.475 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:45:51.495 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 1/1 2024-04-20T01:45:51.495 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2024-04-20T01:45:52.059 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:52.060 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-20T01:45:52.060 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:18.2.2-1252.g15eecde2.el8.x86_64 2024-04-20T01:45:52.060 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:52.060 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T01:45:52.223 DEBUG:teuthology.parallel:result is None 2024-04-20T01:45:52.223 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=15eecde2bab9803889d929c7e423d07970d8cbed 2024-04-20T01:45:52.443 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T01:45:52.471 INFO:teuthology.orchestra.run.smithi044.stdout:18.2.2-1252.g15eecde2.el8 2024-04-20T01:45:52.471 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1252.g15eecde2.el8 2024-04-20T01:45:52.472 INFO:teuthology.task.install:The correct ceph version 18.2.2-1252.g15eecde2 is installed. 2024-04-20T01:45:52.473 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=15eecde2bab9803889d929c7e423d07970d8cbed 2024-04-20T01:45:52.780 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T01:45:52.808 INFO:teuthology.orchestra.run.smithi121.stdout:18.2.2-1252.g15eecde2.el8 2024-04-20T01:45:52.808 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1252.g15eecde2.el8 2024-04-20T01:45:52.808 INFO:teuthology.task.install:The correct ceph version 18.2.2-1252.g15eecde2 is installed. 2024-04-20T01:45:52.810 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=15eecde2bab9803889d929c7e423d07970d8cbed 2024-04-20T01:45:52.947 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T01:45:52.972 INFO:teuthology.orchestra.run.smithi123.stdout:18.2.2-1252.g15eecde2.el8 2024-04-20T01:45:52.972 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1252.g15eecde2.el8 2024-04-20T01:45:52.972 INFO:teuthology.task.install:The correct ceph version 18.2.2-1252.g15eecde2 is installed. 2024-04-20T01:45:52.974 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=15eecde2bab9803889d929c7e423d07970d8cbed 2024-04-20T01:45:53.112 DEBUG:teuthology.orchestra.run.smithi165:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T01:45:53.141 INFO:teuthology.orchestra.run.smithi165.stdout:18.2.2-1252.g15eecde2.el8 2024-04-20T01:45:53.141 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1252.g15eecde2.el8 2024-04-20T01:45:53.141 INFO:teuthology.task.install:The correct ceph version 18.2.2-1252.g15eecde2 is installed. 2024-04-20T01:45:53.143 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-20T01:45:53.143 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:45:53.143 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T01:45:53.180 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T01:45:53.180 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T01:45:53.227 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T01:45:53.228 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T01:45:53.272 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T01:45:53.272 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T01:45:53.319 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-20T01:45:53.319 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:45:53.319 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T01:45:53.353 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T01:45:53.428 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T01:45:53.428 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T01:45:53.461 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T01:45:53.532 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T01:45:53.532 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T01:45:53.564 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T01:45:53.634 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T01:45:53.634 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T01:45:53.667 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T01:45:53.737 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-20T01:45:53.737 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:45:53.737 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T01:45:53.772 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T01:45:53.846 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T01:45:53.846 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T01:45:53.878 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T01:45:53.948 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T01:45:53.948 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T01:45:53.981 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T01:45:54.051 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T01:45:54.051 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T01:45:54.082 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T01:45:54.153 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-20T01:45:54.154 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:45:54.154 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T01:45:54.188 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T01:45:54.262 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T01:45:54.262 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T01:45:54.294 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T01:45:54.363 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T01:45:54.364 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T01:45:54.396 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T01:45:54.465 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T01:45:54.465 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T01:45:54.497 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T01:45:54.569 INFO:teuthology.run_tasks:Running task ceph... 2024-04-20T01:45:54.646 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-20T01:45:54.646 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 777 /var/log/ceph 2024-04-20T01:45:54.649 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 777 /var/log/ceph 2024-04-20T01:45:54.651 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /var/log/ceph 2024-04-20T01:45:54.653 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 777 /var/log/ceph 2024-04-20T01:45:54.683 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-20T01:45:54.683 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-20T01:45:54.722 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-20T01:45:54.725 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-20T01:45:54.727 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-20T01:45:54.757 INFO:tasks.ceph:Creating extra log directories... 2024-04-20T01:45:54.757 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-20T01:45:54.796 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-20T01:45:54.798 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-20T01:45:54.800 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-20T01:45:54.841 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-20T01:45:54.842 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-20T01:45:54.842 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102', 'branch': 'wip-yuri7-testing-2024-04-18-1351-reef', 'description': 'rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7664102', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-yuri7-testing-2024-04-18-1351-reef'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri7-testing-2024-04-18-1351-reef', 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'], ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'], ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'], ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0']], 'seed': 6867, 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-yuri7-testing-2024-04-18-1351-reef', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_15eecde2bab9803889d929c7e423d07970d8cbed/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '15eecde2bab9803889d929c7e423d07970d8cbed', 'targets': {'smithi044.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMIgcP+N0MinUz7O0/WVH1eLrCamp4Ij8HNr9X+vM9WdQmAJ8bkMbCgenNT1rvXHF0KHA6Xxv4NfoM242eSHH8w=', 'smithi121.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOLY3BspgQuGnvmUB46pety3e+CT7KjzA4Ny5DYo9lunqK1EvskSncHv7IU5e1ykYNkoPQYRgG6GyTwKkyvQ/h8=', 'smithi123.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEkjsevvi9EwRZSOFfgnl7OgcuAhGvgOA5spMkNiPuteKF0rrQG98huMZbkMCR1fGcHkctosA9IRsGL4E7YQA+k=', 'smithi165.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIoo/dIkI4rR3vYVmBf83hvaLdIycYt2SJgvAkGKlLeTPh0uuczc7xa9bfxlij62cdXlNmHyaZiYn4kWMdU3GGQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed', 'cluster': 'ceph'}}, {'thrashosds': {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200}}, {'rados': {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'c': 2, 'crush-failure-domain': 'osd', 'k': 4, 'm': 3, 'name': 'shecprofile', 'plugin': 'shec'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-20_01:10:46', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-20T01:45:54.842 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-20T01:45:54.872 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-20T01:45:54.874 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-20T01:45:54.876 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-20T01:45:54.894 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m0777 -- /var/run/ceph 2024-04-20T01:45:54.933 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/run/ceph 2024-04-20T01:45:54.935 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/run/ceph 2024-04-20T01:45:54.937 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m0777 -- /var/run/ceph 2024-04-20T01:45:54.966 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:45:54.966 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T01:45:55.026 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T01:45:55.027 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_1 2024-04-20T01:45:55.088 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T01:45:55.088 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T01:45:55.088 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 181489 Links: 1 2024-04-20T01:45:55.088 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T01:45:55.089 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T01:45:55.089 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-20 01:44:55.008157725 +0000 2024-04-20T01:45:55.089 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-20 01:35:48.601784167 +0000 2024-04-20T01:45:55.089 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-20 01:35:48.601784167 +0000 2024-04-20T01:45:55.089 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: - 2024-04-20T01:45:55.089 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T01:45:55.161 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-20T01:45:55.161 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-20T01:45:55.161 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000268632 s, 1.9 MB/s 2024-04-20T01:45:55.162 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T01:45:55.226 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_2 2024-04-20T01:45:55.288 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T01:45:55.288 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T01:45:55.288 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 175061 Links: 1 2024-04-20T01:45:55.288 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T01:45:55.288 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T01:45:55.288 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-20 01:44:55.008157725 +0000 2024-04-20T01:45:55.288 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-20 01:35:48.964778827 +0000 2024-04-20T01:45:55.288 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-20 01:35:48.964778827 +0000 2024-04-20T01:45:55.288 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: - 2024-04-20T01:45:55.289 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T01:45:55.361 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-20T01:45:55.361 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-20T01:45:55.361 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000235863 s, 2.2 MB/s 2024-04-20T01:45:55.363 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T01:45:55.426 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_3 2024-04-20T01:45:55.488 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T01:45:55.488 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T01:45:55.488 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 175096 Links: 1 2024-04-20T01:45:55.488 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T01:45:55.488 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T01:45:55.488 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-20 01:44:55.009157710 +0000 2024-04-20T01:45:55.488 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-20 01:35:49.343773251 +0000 2024-04-20T01:45:55.489 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-20 01:35:49.343773251 +0000 2024-04-20T01:45:55.489 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: - 2024-04-20T01:45:55.489 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T01:45:55.561 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-20T01:45:55.561 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-20T01:45:55.561 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000208735 s, 2.5 MB/s 2024-04-20T01:45:55.562 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T01:45:55.625 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_4 2024-04-20T01:45:55.687 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T01:45:55.687 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T01:45:55.688 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 6h/6d Inode: 182745 Links: 1 2024-04-20T01:45:55.688 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T01:45:55.688 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T01:45:55.688 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-20 01:44:55.009157710 +0000 2024-04-20T01:45:55.688 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-20 01:35:49.726767617 +0000 2024-04-20T01:45:55.688 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-20 01:35:49.726767617 +0000 2024-04-20T01:45:55.688 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: - 2024-04-20T01:45:55.688 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T01:45:55.760 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-20T01:45:55.760 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-20T01:45:55.760 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.00024459 s, 2.1 MB/s 2024-04-20T01:45:55.761 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T01:45:55.824 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-04-20T01:45:55.824 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T01:45:55.824 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T01:45:55.845 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T01:45:55.845 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_1 2024-04-20T01:45:55.905 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T01:45:55.905 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T01:45:55.905 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 201709 Links: 1 2024-04-20T01:45:55.905 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T01:45:55.905 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T01:45:55.905 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-04-20 01:44:34.852615511 +0000 2024-04-20T01:45:55.906 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-04-20 01:35:28.305933538 +0000 2024-04-20T01:45:55.906 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-04-20 01:35:28.305933538 +0000 2024-04-20T01:45:55.906 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2024-04-20T01:45:55.906 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T01:45:55.974 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-04-20T01:45:55.974 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-04-20T01:45:55.974 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000223603 s, 2.3 MB/s 2024-04-20T01:45:55.975 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T01:45:56.035 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_2 2024-04-20T01:45:56.093 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T01:45:56.094 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T01:45:56.094 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 210771 Links: 1 2024-04-20T01:45:56.094 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T01:45:56.094 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T01:45:56.094 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-04-20 01:44:34.852615511 +0000 2024-04-20T01:45:56.094 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-04-20 01:35:28.657929358 +0000 2024-04-20T01:45:56.094 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-04-20 01:35:28.657929358 +0000 2024-04-20T01:45:56.094 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2024-04-20T01:45:56.094 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T01:45:56.162 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-04-20T01:45:56.162 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-04-20T01:45:56.162 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000304801 s, 1.7 MB/s 2024-04-20T01:45:56.163 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T01:45:56.223 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_3 2024-04-20T01:45:56.282 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T01:45:56.282 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T01:45:56.282 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 214112 Links: 1 2024-04-20T01:45:56.282 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T01:45:56.282 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T01:45:56.282 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-04-20 01:44:34.852615511 +0000 2024-04-20T01:45:56.282 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-04-20 01:35:29.048924713 +0000 2024-04-20T01:45:56.282 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-04-20 01:35:29.048924713 +0000 2024-04-20T01:45:56.282 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2024-04-20T01:45:56.282 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T01:45:56.349 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-04-20T01:45:56.350 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-04-20T01:45:56.350 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000199119 s, 2.6 MB/s 2024-04-20T01:45:56.351 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T01:45:56.410 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_4 2024-04-20T01:45:56.468 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T01:45:56.468 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T01:45:56.468 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 212578 Links: 1 2024-04-20T01:45:56.468 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T01:45:56.468 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T01:45:56.468 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-04-20 01:44:34.852615511 +0000 2024-04-20T01:45:56.468 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-04-20 01:35:29.389920662 +0000 2024-04-20T01:45:56.468 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-04-20 01:35:29.389920662 +0000 2024-04-20T01:45:56.468 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2024-04-20T01:45:56.469 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T01:45:56.534 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-04-20T01:45:56.534 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-04-20T01:45:56.534 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000214777 s, 2.4 MB/s 2024-04-20T01:45:56.535 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T01:45:56.593 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-04-20T01:45:56.593 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T01:45:56.594 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T01:45:56.615 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T01:45:56.615 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-04-20T01:45:56.675 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T01:45:56.675 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T01:45:56.675 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 180556 Links: 1 2024-04-20T01:45:56.675 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T01:45:56.675 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T01:45:56.675 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-20 01:45:20.813551098 +0000 2024-04-20T01:45:56.675 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-20 01:35:10.416278144 +0000 2024-04-20T01:45:56.675 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-20 01:35:10.416278144 +0000 2024-04-20T01:45:56.675 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2024-04-20T01:45:56.675 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T01:45:56.742 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-20T01:45:56.743 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-20T01:45:56.743 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000275765 s, 1.9 MB/s 2024-04-20T01:45:56.744 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T01:45:56.804 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-04-20T01:45:56.863 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T01:45:56.863 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T01:45:56.863 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 182535 Links: 1 2024-04-20T01:45:56.863 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T01:45:56.863 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T01:45:56.863 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-20 01:45:20.813551098 +0000 2024-04-20T01:45:56.863 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-20 01:35:10.767270241 +0000 2024-04-20T01:45:56.864 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-20 01:35:10.767270241 +0000 2024-04-20T01:45:56.864 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2024-04-20T01:45:56.864 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T01:45:56.931 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-20T01:45:56.931 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-20T01:45:56.931 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000245493 s, 2.1 MB/s 2024-04-20T01:45:56.932 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T01:45:56.991 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-04-20T01:45:57.049 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T01:45:57.049 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T01:45:57.049 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 182597 Links: 1 2024-04-20T01:45:57.049 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T01:45:57.049 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T01:45:57.049 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-20 01:45:20.813551098 +0000 2024-04-20T01:45:57.049 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-20 01:35:11.097262811 +0000 2024-04-20T01:45:57.050 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-20 01:35:11.097262811 +0000 2024-04-20T01:45:57.050 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2024-04-20T01:45:57.050 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T01:45:57.115 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-20T01:45:57.115 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-20T01:45:57.116 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000242688 s, 2.1 MB/s 2024-04-20T01:45:57.117 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T01:45:57.174 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-04-20T01:45:57.231 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T01:45:57.231 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T01:45:57.231 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 180646 Links: 1 2024-04-20T01:45:57.231 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T01:45:57.232 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T01:45:57.232 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-20 01:45:20.813551098 +0000 2024-04-20T01:45:57.232 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-20 01:35:11.476254277 +0000 2024-04-20T01:45:57.232 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-20 01:35:11.476254277 +0000 2024-04-20T01:45:57.232 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2024-04-20T01:45:57.232 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T01:45:57.296 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-20T01:45:57.297 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-20T01:45:57.297 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000236541 s, 2.2 MB/s 2024-04-20T01:45:57.298 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T01:45:57.355 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-04-20T01:45:57.355 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T01:45:57.355 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T01:45:57.376 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T01:45:57.376 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_1 2024-04-20T01:45:57.438 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T01:45:57.438 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T01:45:57.438 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 6h/6d Inode: 212262 Links: 1 2024-04-20T01:45:57.438 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T01:45:57.438 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T01:45:57.438 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-20 01:44:34.995010827 +0000 2024-04-20T01:45:57.438 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-20 01:35:21.770874932 +0000 2024-04-20T01:45:57.438 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-20 01:35:21.770874932 +0000 2024-04-20T01:45:57.438 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: - 2024-04-20T01:45:57.438 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T01:45:57.506 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-20T01:45:57.506 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-20T01:45:57.507 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000229685 s, 2.2 MB/s 2024-04-20T01:45:57.508 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T01:45:57.568 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_2 2024-04-20T01:45:57.628 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T01:45:57.628 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T01:45:57.628 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 6h/6d Inode: 212300 Links: 1 2024-04-20T01:45:57.628 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T01:45:57.628 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T01:45:57.628 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-20 01:44:34.995010827 +0000 2024-04-20T01:45:57.628 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-20 01:35:22.123870050 +0000 2024-04-20T01:45:57.628 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-20 01:35:22.123870050 +0000 2024-04-20T01:45:57.628 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: - 2024-04-20T01:45:57.629 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T01:45:57.696 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-20T01:45:57.696 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-20T01:45:57.696 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000225055 s, 2.3 MB/s 2024-04-20T01:45:57.697 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T01:45:57.756 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_3 2024-04-20T01:45:57.816 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T01:45:57.816 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T01:45:57.816 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 6h/6d Inode: 215166 Links: 1 2024-04-20T01:45:57.816 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T01:45:57.816 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T01:45:57.816 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-20 01:44:34.995010827 +0000 2024-04-20T01:45:57.816 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-20 01:35:22.471865235 +0000 2024-04-20T01:45:57.816 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-20 01:35:22.471865235 +0000 2024-04-20T01:45:57.816 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: - 2024-04-20T01:45:57.817 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T01:45:57.885 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-20T01:45:57.885 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-20T01:45:57.885 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000192324 s, 2.7 MB/s 2024-04-20T01:45:57.886 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T01:45:57.946 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_4 2024-04-20T01:45:58.005 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T01:45:58.006 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T01:45:58.006 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 6h/6d Inode: 212373 Links: 1 2024-04-20T01:45:58.006 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T01:45:58.006 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T01:45:58.006 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-20 01:44:34.995010827 +0000 2024-04-20T01:45:58.006 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-20 01:35:22.806860601 +0000 2024-04-20T01:45:58.006 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-20 01:35:22.806860601 +0000 2024-04-20T01:45:58.006 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: - 2024-04-20T01:45:58.006 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T01:45:58.072 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-20T01:45:58.073 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-20T01:45:58.073 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000215527 s, 2.4 MB/s 2024-04-20T01:45:58.074 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T01:45:58.132 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-04-20T01:45:58.132 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi044.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi121.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi165.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2024-04-20T01:45:58.132 INFO:tasks.ceph:Generating config... 2024-04-20T01:45:58.134 INFO:tasks.ceph:[osd] debug monc = 20 2024-04-20T01:45:58.134 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-20T01:45:58.134 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-20T01:45:58.134 INFO:tasks.ceph:[osd] bluestore allocator = stupid 2024-04-20T01:45:58.134 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-20T01:45:58.134 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-20T01:45:58.134 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-04-20T01:45:58.134 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-20T01:45:58.134 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-20T01:45:58.134 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-20T01:45:58.134 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-20T01:45:58.134 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-20T01:45:58.134 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-20T01:45:58.135 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-20T01:45:58.135 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-20T01:45:58.135 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-20T01:45:58.135 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.3 2024-04-20T01:45:58.135 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-04-20T01:45:58.135 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-04-20T01:45:58.135 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-20T01:45:58.135 INFO:tasks.ceph:[osd] osd heartbeat use min delay socket = True 2024-04-20T01:45:58.135 INFO:tasks.ceph:[osd] osd max backfills = 3 2024-04-20T01:45:58.135 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-20T01:45:58.135 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-04-20T01:45:58.135 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-04-20T01:45:58.135 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-20T01:45:58.135 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-04-20T01:45:58.135 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-04-20T01:45:58.135 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2024-04-20T01:45:58.136 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2024-04-20T01:45:58.136 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-04-20T01:45:58.136 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-04-20T01:45:58.136 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-04-20T01:45:58.136 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-04-20T01:45:58.136 INFO:tasks.ceph:[global] osd_object_clean_region_max_num_intervals = 1000 2024-04-20T01:45:58.136 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-20T01:45:58.136 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-20T01:45:58.136 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-20T01:45:58.136 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-20T01:45:58.136 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-20T01:45:58.136 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-04-20T01:45:58.136 INFO:tasks.ceph:Setting up mon.a... 2024-04-20T01:45:58.136 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-20T01:45:58.205 INFO:teuthology.orchestra.run.smithi044.stdout:creating /etc/ceph/ceph.keyring 2024-04-20T01:45:58.209 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-20T01:45:58.263 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T01:45:58.295 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.44'), ('mon.b', '172.21.15.121'), ('mon.c', '172.21.15.123')] 2024-04-20T01:45:58.295 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.44,172.21.15.121,172.21.15.123', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'osd_object_clean_region_max_num_intervals': 1000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-04-20T01:45:58.296 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:45:58.296 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-20T01:45:58.357 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.44 --add b 172.21.15.121 --add c 172.21.15.123 --print /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:45:58.456 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:45:58.456 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: generated fsid 9c2f64c1-aa35-4337-9f5c-081b9e3a2938 2024-04-20T01:45:58.456 INFO:teuthology.orchestra.run.smithi044.stdout:setting min_mon_release = pacific 2024-04-20T01:45:58.456 INFO:teuthology.orchestra.run.smithi044.stdout:epoch 0 2024-04-20T01:45:58.456 INFO:teuthology.orchestra.run.smithi044.stdout:fsid 9c2f64c1-aa35-4337-9f5c-081b9e3a2938 2024-04-20T01:45:58.456 INFO:teuthology.orchestra.run.smithi044.stdout:last_changed 2024-04-20T01:45:58.470962+0000 2024-04-20T01:45:58.457 INFO:teuthology.orchestra.run.smithi044.stdout:created 2024-04-20T01:45:58.470962+0000 2024-04-20T01:45:58.457 INFO:teuthology.orchestra.run.smithi044.stdout:min_mon_release 16 (pacific) 2024-04-20T01:45:58.457 INFO:teuthology.orchestra.run.smithi044.stdout:election_strategy: 1 2024-04-20T01:45:58.457 INFO:teuthology.orchestra.run.smithi044.stdout:0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-04-20T01:45:58.457 INFO:teuthology.orchestra.run.smithi044.stdout:1: [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon.b 2024-04-20T01:45:58.457 INFO:teuthology.orchestra.run.smithi044.stdout:2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.c 2024-04-20T01:45:58.457 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-20T01:45:58.458 DEBUG:teuthology.orchestra.run.smithi044:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-20T01:45:58.478 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9c2f64c1-aa35-4337-9f5c-081b9e3a2938... 2024-04-20T01:45:58.478 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T01:45:58.520 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T01:45:58.523 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T01:45:58.525 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T01:45:58.575 INFO:teuthology.orchestra.run.smithi121.stdout:[global] 2024-04-20T01:45:58.575 INFO:teuthology.orchestra.run.smithi121.stdout: chdir = "" 2024-04-20T01:45:58.575 INFO:teuthology.orchestra.run.smithi121.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T01:45:58.575 INFO:teuthology.orchestra.run.smithi121.stdout: auth supported = cephx 2024-04-20T01:45:58.575 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.575 INFO:teuthology.orchestra.run.smithi121.stdout: filestore xattr use omap = true 2024-04-20T01:45:58.575 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.575 INFO:teuthology.orchestra.run.smithi121.stdout: mon clock drift allowed = 1.000 2024-04-20T01:45:58.575 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.575 INFO:teuthology.orchestra.run.smithi121.stdout: osd crush chooseleaf type = 0 2024-04-20T01:45:58.576 INFO:teuthology.orchestra.run.smithi121.stdout: auth debug = true 2024-04-20T01:45:58.576 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.576 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on old message = true 2024-04-20T01:45:58.576 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on bug = true 2024-04-20T01:45:58.576 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.576 INFO:teuthology.orchestra.run.smithi121.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T01:45:58.576 INFO:teuthology.orchestra.run.smithi121.stdout: mon pg warn max object skew = 0 2024-04-20T01:45:58.576 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.576 INFO:teuthology.orchestra.run.smithi121.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T01:45:58.576 INFO:teuthology.orchestra.run.smithi121.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T01:45:58.577 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.577 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default size = 2 2024-04-20T01:45:58.577 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.577 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow primary affinity = true 2024-04-20T01:45:58.577 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow pg remap = true 2024-04-20T01:45:58.577 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on legacy crush tunables = false 2024-04-20T01:45:58.577 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on crush straw calc version zero = false 2024-04-20T01:45:58.577 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on no sortbitwise = false 2024-04-20T01:45:58.577 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on osd down out interval zero = false 2024-04-20T01:45:58.577 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on too few osds = false 2024-04-20T01:45:58.577 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T01:45:58.577 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T01:45:58.577 INFO:teuthology.orchestra.run.smithi121.stdout: mon_allow_pool_size_one = true 2024-04-20T01:45:58.577 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.577 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T01:45:58.578 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.578 INFO:teuthology.orchestra.run.smithi121.stdout: osd default data pool replay window = 5 2024-04-20T01:45:58.578 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.578 INFO:teuthology.orchestra.run.smithi121.stdout: mon allow pool delete = true 2024-04-20T01:45:58.578 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.578 INFO:teuthology.orchestra.run.smithi121.stdout: mon cluster log file level = debug 2024-04-20T01:45:58.578 INFO:teuthology.orchestra.run.smithi121.stdout: debug asserts on shutdown = true 2024-04-20T01:45:58.578 INFO:teuthology.orchestra.run.smithi121.stdout: mon health detail to clog = false 2024-04-20T01:45:58.578 INFO:teuthology.orchestra.run.smithi121.stdout: mon host = "172.21.15.44,172.21.15.121,172.21.15.123" 2024-04-20T01:45:58.578 INFO:teuthology.orchestra.run.smithi121.stdout: mon client directed command retry = 5 2024-04-20T01:45:58.578 INFO:teuthology.orchestra.run.smithi121.stdout: mon election default strategy = 1 2024-04-20T01:45:58.578 INFO:teuthology.orchestra.run.smithi121.stdout: ms inject socket failures = 5000 2024-04-20T01:45:58.578 INFO:teuthology.orchestra.run.smithi121.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-04-20T01:45:58.578 INFO:teuthology.orchestra.run.smithi121.stdout: fsid = 9c2f64c1-aa35-4337-9f5c-081b9e3a2938 2024-04-20T01:45:58.579 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.579 INFO:teuthology.orchestra.run.smithi121.stdout:[osd] 2024-04-20T01:45:58.579 INFO:teuthology.orchestra.run.smithi121.stdout: osd journal size = 100 2024-04-20T01:45:58.579 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.579 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub load threshold = 5.0 2024-04-20T01:45:58.579 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub max interval = 120 2024-04-20T01:45:58.579 INFO:teuthology.orchestra.run.smithi121.stdout: osd mclock profile = high_recovery_ops 2024-04-20T01:45:58.579 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.579 INFO:teuthology.orchestra.run.smithi121.stdout: osd recover clone overlap = true 2024-04-20T01:45:58.579 INFO:teuthology.orchestra.run.smithi121.stdout: osd recovery max chunk = 1048576 2024-04-20T01:45:58.579 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.579 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug shutdown = true 2024-04-20T01:45:58.579 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug op order = true 2024-04-20T01:45:58.579 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug verify stray on activate = true 2024-04-20T01:45:58.580 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.580 INFO:teuthology.orchestra.run.smithi121.stdout: osd open classes on start = true 2024-04-20T01:45:58.580 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug pg log writeout = true 2024-04-20T01:45:58.580 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.580 INFO:teuthology.orchestra.run.smithi121.stdout: osd deep scrub update digest min age = 30 2024-04-20T01:45:58.580 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.580 INFO:teuthology.orchestra.run.smithi121.stdout: osd map max advance = 10 2024-04-20T01:45:58.580 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.580 INFO:teuthology.orchestra.run.smithi121.stdout: journal zero on create = true 2024-04-20T01:45:58.580 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.580 INFO:teuthology.orchestra.run.smithi121.stdout: filestore ondisk finisher threads = 3 2024-04-20T01:45:58.580 INFO:teuthology.orchestra.run.smithi121.stdout: filestore apply finisher threads = 3 2024-04-20T01:45:58.580 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.580 INFO:teuthology.orchestra.run.smithi121.stdout: bdev debug aio = true 2024-04-20T01:45:58.580 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug misdirected ops = true 2024-04-20T01:45:58.581 INFO:teuthology.orchestra.run.smithi121.stdout: debug monc = 20 2024-04-20T01:45:58.581 INFO:teuthology.orchestra.run.smithi121.stdout: bdev async discard = True 2024-04-20T01:45:58.581 INFO:teuthology.orchestra.run.smithi121.stdout: bdev enable discard = True 2024-04-20T01:45:58.581 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore allocator = stupid 2024-04-20T01:45:58.581 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore block size = 96636764160 2024-04-20T01:45:58.581 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore fsck on mount = True 2024-04-20T01:45:58.581 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore zero block detection = True 2024-04-20T01:45:58.581 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluefs = 20 2024-04-20T01:45:58.581 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluestore = 20 2024-04-20T01:45:58.581 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-04-20T01:45:58.581 INFO:teuthology.orchestra.run.smithi121.stdout: debug osd = 20 2024-04-20T01:45:58.581 INFO:teuthology.orchestra.run.smithi121.stdout: debug rocksdb = 10 2024-04-20T01:45:58.581 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T01:45:58.582 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd full ratio = 0.9 2024-04-20T01:45:58.582 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd nearfull ratio = 0.8 2024-04-20T01:45:58.582 INFO:teuthology.orchestra.run.smithi121.stdout: osd blocked scrub grace period = 3600 2024-04-20T01:45:58.582 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug reject backfill probability = 0.3 2024-04-20T01:45:58.582 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug verify cached snaps = True 2024-04-20T01:45:58.582 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug verify missing on start = True 2024-04-20T01:45:58.582 INFO:teuthology.orchestra.run.smithi121.stdout: osd failsafe full ratio = 0.95 2024-04-20T01:45:58.582 INFO:teuthology.orchestra.run.smithi121.stdout: osd heartbeat use min delay socket = True 2024-04-20T01:45:58.582 INFO:teuthology.orchestra.run.smithi121.stdout: osd max backfills = 3 2024-04-20T01:45:58.582 INFO:teuthology.orchestra.run.smithi121.stdout: osd max markdown count = 1000 2024-04-20T01:45:58.582 INFO:teuthology.orchestra.run.smithi121.stdout: osd mclock override recovery settings = True 2024-04-20T01:45:58.583 INFO:teuthology.orchestra.run.smithi121.stdout: osd objectstore = bluestore 2024-04-20T01:45:58.583 INFO:teuthology.orchestra.run.smithi121.stdout: osd op queue = debug_random 2024-04-20T01:45:58.583 INFO:teuthology.orchestra.run.smithi121.stdout: osd op queue cut off = debug_random 2024-04-20T01:45:58.583 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub min interval = 60 2024-04-20T01:45:58.583 INFO:teuthology.orchestra.run.smithi121.stdout: osd shutdown pgref assert = True 2024-04-20T01:45:58.583 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.583 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr] 2024-04-20T01:45:58.583 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-04-20T01:45:58.583 INFO:teuthology.orchestra.run.smithi121.stdout: debug mgr = 20 2024-04-20T01:45:58.583 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2024-04-20T01:45:58.583 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2024-04-20T01:45:58.584 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2024-04-20T01:45:58.584 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2024-04-20T01:45:58.584 INFO:teuthology.orchestra.run.smithi121.stdout: mgr/telemetry/nag = false 2024-04-20T01:45:58.584 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.584 INFO:teuthology.orchestra.run.smithi121.stdout:[mon] 2024-04-20T01:45:58.584 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-04-20T01:45:58.584 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2024-04-20T01:45:58.584 INFO:teuthology.orchestra.run.smithi121.stdout: debug paxos = 20 2024-04-20T01:45:58.584 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2024-04-20T01:45:58.584 INFO:teuthology.orchestra.run.smithi121.stdout: mon data avail warn = 5 2024-04-20T01:45:58.584 INFO:teuthology.orchestra.run.smithi121.stdout: mon mgr mkfs grace = 240 2024-04-20T01:45:58.584 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2024-04-20T01:45:58.584 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd reporter subtree level = osd 2024-04-20T01:45:58.584 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd prime pg temp = true 2024-04-20T01:45:58.585 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2024-04-20T01:45:58.585 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.585 INFO:teuthology.orchestra.run.smithi121.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T01:45:58.585 INFO:teuthology.orchestra.run.smithi121.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T01:45:58.585 INFO:teuthology.orchestra.run.smithi121.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T01:45:58.585 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.585 INFO:teuthology.orchestra.run.smithi121.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T01:45:58.585 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T01:45:58.585 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T01:45:58.586 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.586 INFO:teuthology.orchestra.run.smithi121.stdout: # 1m isn't quite enough 2024-04-20T01:45:58.586 INFO:teuthology.orchestra.run.smithi121.stdout: mon_down_mkfs_grace = 2m 2024-04-20T01:45:58.586 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.586 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_filestore_osds = false 2024-04-20T01:45:58.586 INFO:teuthology.orchestra.run.smithi121.stdout: mon scrub interval = 300 2024-04-20T01:45:58.586 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T01:45:58.586 INFO:teuthology.orchestra.run.smithi121.stdout:[client] 2024-04-20T01:45:58.586 INFO:teuthology.orchestra.run.smithi121.stdout: rgw cache enabled = true 2024-04-20T01:45:58.586 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable ops log = true 2024-04-20T01:45:58.586 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable usage log = true 2024-04-20T01:45:58.586 INFO:teuthology.orchestra.run.smithi121.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T01:45:58.586 INFO:teuthology.orchestra.run.smithi121.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T01:45:58.586 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.a] 2024-04-20T01:45:58.587 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.b] 2024-04-20T01:45:58.587 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.c] 2024-04-20T01:45:58.587 INFO:teuthology.orchestra.run.smithi123.stdout:[global] 2024-04-20T01:45:58.588 INFO:teuthology.orchestra.run.smithi123.stdout: chdir = "" 2024-04-20T01:45:58.588 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T01:45:58.588 INFO:teuthology.orchestra.run.smithi123.stdout: auth supported = cephx 2024-04-20T01:45:58.588 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.588 INFO:teuthology.orchestra.run.smithi123.stdout: filestore xattr use omap = true 2024-04-20T01:45:58.588 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.588 INFO:teuthology.orchestra.run.smithi123.stdout: mon clock drift allowed = 1.000 2024-04-20T01:45:58.588 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.588 INFO:teuthology.orchestra.run.smithi123.stdout: osd crush chooseleaf type = 0 2024-04-20T01:45:58.588 INFO:teuthology.orchestra.run.smithi123.stdout: auth debug = true 2024-04-20T01:45:58.588 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.589 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on old message = true 2024-04-20T01:45:58.589 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on bug = true 2024-04-20T01:45:58.589 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.589 INFO:teuthology.orchestra.run.smithi123.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T01:45:58.589 INFO:teuthology.orchestra.run.smithi123.stdout: mon pg warn max object skew = 0 2024-04-20T01:45:58.589 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.589 INFO:teuthology.orchestra.run.smithi123.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T01:45:58.589 INFO:teuthology.orchestra.run.smithi123.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T01:45:58.589 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.589 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default size = 2 2024-04-20T01:45:58.589 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.589 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow primary affinity = true 2024-04-20T01:45:58.589 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow pg remap = true 2024-04-20T01:45:58.589 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on legacy crush tunables = false 2024-04-20T01:45:58.590 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on crush straw calc version zero = false 2024-04-20T01:45:58.590 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on no sortbitwise = false 2024-04-20T01:45:58.590 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on osd down out interval zero = false 2024-04-20T01:45:58.590 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on too few osds = false 2024-04-20T01:45:58.590 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T01:45:58.590 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T01:45:58.590 INFO:teuthology.orchestra.run.smithi123.stdout: mon_allow_pool_size_one = true 2024-04-20T01:45:58.590 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.590 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T01:45:58.590 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.590 INFO:teuthology.orchestra.run.smithi123.stdout: osd default data pool replay window = 5 2024-04-20T01:45:58.590 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.591 INFO:teuthology.orchestra.run.smithi123.stdout: mon allow pool delete = true 2024-04-20T01:45:58.591 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.591 INFO:teuthology.orchestra.run.smithi123.stdout: mon cluster log file level = debug 2024-04-20T01:45:58.591 INFO:teuthology.orchestra.run.smithi123.stdout: debug asserts on shutdown = true 2024-04-20T01:45:58.591 INFO:teuthology.orchestra.run.smithi123.stdout: mon health detail to clog = false 2024-04-20T01:45:58.591 INFO:teuthology.orchestra.run.smithi123.stdout: mon host = "172.21.15.44,172.21.15.121,172.21.15.123" 2024-04-20T01:45:58.591 INFO:teuthology.orchestra.run.smithi123.stdout: mon client directed command retry = 5 2024-04-20T01:45:58.591 INFO:teuthology.orchestra.run.smithi123.stdout: mon election default strategy = 1 2024-04-20T01:45:58.591 INFO:teuthology.orchestra.run.smithi123.stdout: ms inject socket failures = 5000 2024-04-20T01:45:58.591 INFO:teuthology.orchestra.run.smithi123.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-04-20T01:45:58.591 INFO:teuthology.orchestra.run.smithi123.stdout: fsid = 9c2f64c1-aa35-4337-9f5c-081b9e3a2938 2024-04-20T01:45:58.591 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.591 INFO:teuthology.orchestra.run.smithi123.stdout:[osd] 2024-04-20T01:45:58.591 INFO:teuthology.orchestra.run.smithi123.stdout: osd journal size = 100 2024-04-20T01:45:58.592 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.592 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub load threshold = 5.0 2024-04-20T01:45:58.592 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub max interval = 120 2024-04-20T01:45:58.592 INFO:teuthology.orchestra.run.smithi123.stdout: osd mclock profile = high_recovery_ops 2024-04-20T01:45:58.592 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.592 INFO:teuthology.orchestra.run.smithi123.stdout: osd recover clone overlap = true 2024-04-20T01:45:58.592 INFO:teuthology.orchestra.run.smithi123.stdout: osd recovery max chunk = 1048576 2024-04-20T01:45:58.592 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.592 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug shutdown = true 2024-04-20T01:45:58.592 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug op order = true 2024-04-20T01:45:58.592 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug verify stray on activate = true 2024-04-20T01:45:58.592 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.592 INFO:teuthology.orchestra.run.smithi123.stdout: osd open classes on start = true 2024-04-20T01:45:58.592 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug pg log writeout = true 2024-04-20T01:45:58.593 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.593 INFO:teuthology.orchestra.run.smithi123.stdout: osd deep scrub update digest min age = 30 2024-04-20T01:45:58.593 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.593 INFO:teuthology.orchestra.run.smithi123.stdout: osd map max advance = 10 2024-04-20T01:45:58.593 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.593 INFO:teuthology.orchestra.run.smithi123.stdout: journal zero on create = true 2024-04-20T01:45:58.593 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.593 INFO:teuthology.orchestra.run.smithi123.stdout: filestore ondisk finisher threads = 3 2024-04-20T01:45:58.593 INFO:teuthology.orchestra.run.smithi123.stdout: filestore apply finisher threads = 3 2024-04-20T01:45:58.593 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.594 INFO:teuthology.orchestra.run.smithi123.stdout: bdev debug aio = true 2024-04-20T01:45:58.594 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug misdirected ops = true 2024-04-20T01:45:58.594 INFO:teuthology.orchestra.run.smithi123.stdout: debug monc = 20 2024-04-20T01:45:58.594 INFO:teuthology.orchestra.run.smithi123.stdout: bdev async discard = True 2024-04-20T01:45:58.594 INFO:teuthology.orchestra.run.smithi123.stdout: bdev enable discard = True 2024-04-20T01:45:58.594 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore allocator = stupid 2024-04-20T01:45:58.594 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore block size = 96636764160 2024-04-20T01:45:58.594 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore fsck on mount = True 2024-04-20T01:45:58.594 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore zero block detection = True 2024-04-20T01:45:58.594 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluefs = 20 2024-04-20T01:45:58.594 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluestore = 20 2024-04-20T01:45:58.594 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-20T01:45:58.595 INFO:teuthology.orchestra.run.smithi123.stdout: debug osd = 20 2024-04-20T01:45:58.595 INFO:teuthology.orchestra.run.smithi123.stdout: debug rocksdb = 10 2024-04-20T01:45:58.595 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T01:45:58.595 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd full ratio = 0.9 2024-04-20T01:45:58.595 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd nearfull ratio = 0.8 2024-04-20T01:45:58.595 INFO:teuthology.orchestra.run.smithi123.stdout: osd blocked scrub grace period = 3600 2024-04-20T01:45:58.595 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug reject backfill probability = 0.3 2024-04-20T01:45:58.595 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug verify cached snaps = True 2024-04-20T01:45:58.595 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug verify missing on start = True 2024-04-20T01:45:58.595 INFO:teuthology.orchestra.run.smithi123.stdout: osd failsafe full ratio = 0.95 2024-04-20T01:45:58.595 INFO:teuthology.orchestra.run.smithi123.stdout: osd heartbeat use min delay socket = True 2024-04-20T01:45:58.595 INFO:teuthology.orchestra.run.smithi123.stdout: osd max backfills = 3 2024-04-20T01:45:58.595 INFO:teuthology.orchestra.run.smithi123.stdout: osd max markdown count = 1000 2024-04-20T01:45:58.595 INFO:teuthology.orchestra.run.smithi123.stdout: osd mclock override recovery settings = True 2024-04-20T01:45:58.595 INFO:teuthology.orchestra.run.smithi123.stdout: osd objectstore = bluestore 2024-04-20T01:45:58.595 INFO:teuthology.orchestra.run.smithi123.stdout: osd op queue = debug_random 2024-04-20T01:45:58.596 INFO:teuthology.orchestra.run.smithi123.stdout: osd op queue cut off = debug_random 2024-04-20T01:45:58.596 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub min interval = 60 2024-04-20T01:45:58.596 INFO:teuthology.orchestra.run.smithi123.stdout: osd shutdown pgref assert = True 2024-04-20T01:45:58.596 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.596 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr] 2024-04-20T01:45:58.596 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-20T01:45:58.596 INFO:teuthology.orchestra.run.smithi123.stdout: debug mgr = 20 2024-04-20T01:45:58.596 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2024-04-20T01:45:58.596 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2024-04-20T01:45:58.596 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2024-04-20T01:45:58.596 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2024-04-20T01:45:58.596 INFO:teuthology.orchestra.run.smithi123.stdout: mgr/telemetry/nag = false 2024-04-20T01:45:58.596 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.596 INFO:teuthology.orchestra.run.smithi123.stdout:[mon] 2024-04-20T01:45:58.596 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-20T01:45:58.597 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2024-04-20T01:45:58.597 INFO:teuthology.orchestra.run.smithi123.stdout: debug paxos = 20 2024-04-20T01:45:58.597 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2024-04-20T01:45:58.597 INFO:teuthology.orchestra.run.smithi123.stdout: mon data avail warn = 5 2024-04-20T01:45:58.597 INFO:teuthology.orchestra.run.smithi123.stdout: mon mgr mkfs grace = 240 2024-04-20T01:45:58.597 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2024-04-20T01:45:58.597 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd reporter subtree level = osd 2024-04-20T01:45:58.597 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd prime pg temp = true 2024-04-20T01:45:58.597 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2024-04-20T01:45:58.597 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.597 INFO:teuthology.orchestra.run.smithi123.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T01:45:58.597 INFO:teuthology.orchestra.run.smithi123.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T01:45:58.597 INFO:teuthology.orchestra.run.smithi123.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T01:45:58.597 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.598 INFO:teuthology.orchestra.run.smithi123.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T01:45:58.598 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T01:45:58.598 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T01:45:58.598 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.598 INFO:teuthology.orchestra.run.smithi123.stdout: # 1m isn't quite enough 2024-04-20T01:45:58.598 INFO:teuthology.orchestra.run.smithi123.stdout: mon_down_mkfs_grace = 2m 2024-04-20T01:45:58.598 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.598 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_filestore_osds = false 2024-04-20T01:45:58.598 INFO:teuthology.orchestra.run.smithi123.stdout: mon scrub interval = 300 2024-04-20T01:45:58.598 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T01:45:58.598 INFO:teuthology.orchestra.run.smithi123.stdout:[client] 2024-04-20T01:45:58.598 INFO:teuthology.orchestra.run.smithi123.stdout: rgw cache enabled = true 2024-04-20T01:45:58.598 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable ops log = true 2024-04-20T01:45:58.598 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable usage log = true 2024-04-20T01:45:58.598 INFO:teuthology.orchestra.run.smithi123.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T01:45:58.599 INFO:teuthology.orchestra.run.smithi123.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T01:45:58.599 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.a] 2024-04-20T01:45:58.599 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.b] 2024-04-20T01:45:58.599 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.c] 2024-04-20T01:45:58.599 INFO:teuthology.orchestra.run.smithi165.stdout:[global] 2024-04-20T01:45:58.599 INFO:teuthology.orchestra.run.smithi165.stdout: chdir = "" 2024-04-20T01:45:58.599 INFO:teuthology.orchestra.run.smithi165.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T01:45:58.599 INFO:teuthology.orchestra.run.smithi165.stdout: auth supported = cephx 2024-04-20T01:45:58.599 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.599 INFO:teuthology.orchestra.run.smithi165.stdout: filestore xattr use omap = true 2024-04-20T01:45:58.599 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.599 INFO:teuthology.orchestra.run.smithi165.stdout: mon clock drift allowed = 1.000 2024-04-20T01:45:58.599 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.600 INFO:teuthology.orchestra.run.smithi165.stdout: osd crush chooseleaf type = 0 2024-04-20T01:45:58.600 INFO:teuthology.orchestra.run.smithi165.stdout: auth debug = true 2024-04-20T01:45:58.600 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.600 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on old message = true 2024-04-20T01:45:58.600 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on bug = true 2024-04-20T01:45:58.600 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.600 INFO:teuthology.orchestra.run.smithi165.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T01:45:58.600 INFO:teuthology.orchestra.run.smithi165.stdout: mon pg warn max object skew = 0 2024-04-20T01:45:58.600 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.600 INFO:teuthology.orchestra.run.smithi165.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T01:45:58.600 INFO:teuthology.orchestra.run.smithi165.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T01:45:58.600 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.600 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default size = 2 2024-04-20T01:45:58.600 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.600 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow primary affinity = true 2024-04-20T01:45:58.601 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow pg remap = true 2024-04-20T01:45:58.601 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on legacy crush tunables = false 2024-04-20T01:45:58.601 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on crush straw calc version zero = false 2024-04-20T01:45:58.601 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on no sortbitwise = false 2024-04-20T01:45:58.601 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on osd down out interval zero = false 2024-04-20T01:45:58.601 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on too few osds = false 2024-04-20T01:45:58.601 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T01:45:58.601 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T01:45:58.601 INFO:teuthology.orchestra.run.smithi165.stdout: mon_allow_pool_size_one = true 2024-04-20T01:45:58.601 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.601 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T01:45:58.601 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.601 INFO:teuthology.orchestra.run.smithi165.stdout: osd default data pool replay window = 5 2024-04-20T01:45:58.601 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.601 INFO:teuthology.orchestra.run.smithi165.stdout: mon allow pool delete = true 2024-04-20T01:45:58.602 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.602 INFO:teuthology.orchestra.run.smithi165.stdout: mon cluster log file level = debug 2024-04-20T01:45:58.602 INFO:teuthology.orchestra.run.smithi165.stdout: debug asserts on shutdown = true 2024-04-20T01:45:58.602 INFO:teuthology.orchestra.run.smithi165.stdout: mon health detail to clog = false 2024-04-20T01:45:58.602 INFO:teuthology.orchestra.run.smithi165.stdout: mon host = "172.21.15.44,172.21.15.121,172.21.15.123" 2024-04-20T01:45:58.602 INFO:teuthology.orchestra.run.smithi165.stdout: mon client directed command retry = 5 2024-04-20T01:45:58.602 INFO:teuthology.orchestra.run.smithi165.stdout: mon election default strategy = 1 2024-04-20T01:45:58.602 INFO:teuthology.orchestra.run.smithi165.stdout: ms inject socket failures = 5000 2024-04-20T01:45:58.602 INFO:teuthology.orchestra.run.smithi165.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-04-20T01:45:58.602 INFO:teuthology.orchestra.run.smithi165.stdout: fsid = 9c2f64c1-aa35-4337-9f5c-081b9e3a2938 2024-04-20T01:45:58.602 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.602 INFO:teuthology.orchestra.run.smithi165.stdout:[osd] 2024-04-20T01:45:58.602 INFO:teuthology.orchestra.run.smithi165.stdout: osd journal size = 100 2024-04-20T01:45:58.602 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.602 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub load threshold = 5.0 2024-04-20T01:45:58.603 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub max interval = 120 2024-04-20T01:45:58.603 INFO:teuthology.orchestra.run.smithi165.stdout: osd mclock profile = high_recovery_ops 2024-04-20T01:45:58.603 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.603 INFO:teuthology.orchestra.run.smithi165.stdout: osd recover clone overlap = true 2024-04-20T01:45:58.603 INFO:teuthology.orchestra.run.smithi165.stdout: osd recovery max chunk = 1048576 2024-04-20T01:45:58.603 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.603 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug shutdown = true 2024-04-20T01:45:58.603 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug op order = true 2024-04-20T01:45:58.603 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug verify stray on activate = true 2024-04-20T01:45:58.603 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.603 INFO:teuthology.orchestra.run.smithi165.stdout: osd open classes on start = true 2024-04-20T01:45:58.603 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug pg log writeout = true 2024-04-20T01:45:58.603 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.603 INFO:teuthology.orchestra.run.smithi165.stdout: osd deep scrub update digest min age = 30 2024-04-20T01:45:58.603 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.604 INFO:teuthology.orchestra.run.smithi165.stdout: osd map max advance = 10 2024-04-20T01:45:58.604 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.604 INFO:teuthology.orchestra.run.smithi165.stdout: journal zero on create = true 2024-04-20T01:45:58.604 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.604 INFO:teuthology.orchestra.run.smithi165.stdout: filestore ondisk finisher threads = 3 2024-04-20T01:45:58.604 INFO:teuthology.orchestra.run.smithi165.stdout: filestore apply finisher threads = 3 2024-04-20T01:45:58.604 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.604 INFO:teuthology.orchestra.run.smithi165.stdout: bdev debug aio = true 2024-04-20T01:45:58.604 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug misdirected ops = true 2024-04-20T01:45:58.604 INFO:teuthology.orchestra.run.smithi165.stdout: debug monc = 20 2024-04-20T01:45:58.604 INFO:teuthology.orchestra.run.smithi165.stdout: bdev async discard = True 2024-04-20T01:45:58.604 INFO:teuthology.orchestra.run.smithi165.stdout: bdev enable discard = True 2024-04-20T01:45:58.604 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore allocator = stupid 2024-04-20T01:45:58.604 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore block size = 96636764160 2024-04-20T01:45:58.604 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore fsck on mount = True 2024-04-20T01:45:58.605 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore zero block detection = True 2024-04-20T01:45:58.605 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluefs = 20 2024-04-20T01:45:58.605 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluestore = 20 2024-04-20T01:45:58.605 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-20T01:45:58.605 INFO:teuthology.orchestra.run.smithi165.stdout: debug osd = 20 2024-04-20T01:45:58.605 INFO:teuthology.orchestra.run.smithi165.stdout: debug rocksdb = 10 2024-04-20T01:45:58.605 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T01:45:58.605 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd full ratio = 0.9 2024-04-20T01:45:58.605 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd nearfull ratio = 0.8 2024-04-20T01:45:58.605 INFO:teuthology.orchestra.run.smithi165.stdout: osd blocked scrub grace period = 3600 2024-04-20T01:45:58.605 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug reject backfill probability = 0.3 2024-04-20T01:45:58.605 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug verify cached snaps = True 2024-04-20T01:45:58.605 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug verify missing on start = True 2024-04-20T01:45:58.605 INFO:teuthology.orchestra.run.smithi165.stdout: osd failsafe full ratio = 0.95 2024-04-20T01:45:58.605 INFO:teuthology.orchestra.run.smithi165.stdout: osd heartbeat use min delay socket = True 2024-04-20T01:45:58.605 INFO:teuthology.orchestra.run.smithi165.stdout: osd max backfills = 3 2024-04-20T01:45:58.606 INFO:teuthology.orchestra.run.smithi165.stdout: osd max markdown count = 1000 2024-04-20T01:45:58.606 INFO:teuthology.orchestra.run.smithi165.stdout: osd mclock override recovery settings = True 2024-04-20T01:45:58.606 INFO:teuthology.orchestra.run.smithi165.stdout: osd objectstore = bluestore 2024-04-20T01:45:58.606 INFO:teuthology.orchestra.run.smithi165.stdout: osd op queue = debug_random 2024-04-20T01:45:58.606 INFO:teuthology.orchestra.run.smithi165.stdout: osd op queue cut off = debug_random 2024-04-20T01:45:58.606 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub min interval = 60 2024-04-20T01:45:58.606 INFO:teuthology.orchestra.run.smithi165.stdout: osd shutdown pgref assert = True 2024-04-20T01:45:58.606 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.606 INFO:teuthology.orchestra.run.smithi165.stdout:[mgr] 2024-04-20T01:45:58.606 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-20T01:45:58.606 INFO:teuthology.orchestra.run.smithi165.stdout: debug mgr = 20 2024-04-20T01:45:58.606 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2024-04-20T01:45:58.606 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2024-04-20T01:45:58.606 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2024-04-20T01:45:58.607 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2024-04-20T01:45:58.607 INFO:teuthology.orchestra.run.smithi165.stdout: mgr/telemetry/nag = false 2024-04-20T01:45:58.607 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.607 INFO:teuthology.orchestra.run.smithi165.stdout:[mon] 2024-04-20T01:45:58.607 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-20T01:45:58.607 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2024-04-20T01:45:58.607 INFO:teuthology.orchestra.run.smithi165.stdout: debug paxos = 20 2024-04-20T01:45:58.607 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2024-04-20T01:45:58.607 INFO:teuthology.orchestra.run.smithi165.stdout: mon data avail warn = 5 2024-04-20T01:45:58.607 INFO:teuthology.orchestra.run.smithi165.stdout: mon mgr mkfs grace = 240 2024-04-20T01:45:58.607 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2024-04-20T01:45:58.607 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd reporter subtree level = osd 2024-04-20T01:45:58.607 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd prime pg temp = true 2024-04-20T01:45:58.607 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2024-04-20T01:45:58.607 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.607 INFO:teuthology.orchestra.run.smithi165.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T01:45:58.608 INFO:teuthology.orchestra.run.smithi165.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T01:45:58.608 INFO:teuthology.orchestra.run.smithi165.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T01:45:58.608 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.608 INFO:teuthology.orchestra.run.smithi165.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T01:45:58.608 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T01:45:58.608 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T01:45:58.608 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.608 INFO:teuthology.orchestra.run.smithi165.stdout: # 1m isn't quite enough 2024-04-20T01:45:58.608 INFO:teuthology.orchestra.run.smithi165.stdout: mon_down_mkfs_grace = 2m 2024-04-20T01:45:58.608 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.608 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_filestore_osds = false 2024-04-20T01:45:58.608 INFO:teuthology.orchestra.run.smithi165.stdout: mon scrub interval = 300 2024-04-20T01:45:58.608 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-20T01:45:58.608 INFO:teuthology.orchestra.run.smithi165.stdout:[client] 2024-04-20T01:45:58.608 INFO:teuthology.orchestra.run.smithi165.stdout: rgw cache enabled = true 2024-04-20T01:45:58.609 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable ops log = true 2024-04-20T01:45:58.609 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable usage log = true 2024-04-20T01:45:58.609 INFO:teuthology.orchestra.run.smithi165.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T01:45:58.609 INFO:teuthology.orchestra.run.smithi165.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T01:45:58.609 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.a] 2024-04-20T01:45:58.609 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.b] 2024-04-20T01:45:58.609 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.c] 2024-04-20T01:45:58.609 INFO:teuthology.orchestra.run.smithi044.stdout:[global] 2024-04-20T01:45:58.609 INFO:teuthology.orchestra.run.smithi044.stdout: chdir = "" 2024-04-20T01:45:58.609 INFO:teuthology.orchestra.run.smithi044.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T01:45:58.609 INFO:teuthology.orchestra.run.smithi044.stdout: auth supported = cephx 2024-04-20T01:45:58.609 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.609 INFO:teuthology.orchestra.run.smithi044.stdout: filestore xattr use omap = true 2024-04-20T01:45:58.609 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.610 INFO:teuthology.orchestra.run.smithi044.stdout: mon clock drift allowed = 1.000 2024-04-20T01:45:58.610 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.610 INFO:teuthology.orchestra.run.smithi044.stdout: osd crush chooseleaf type = 0 2024-04-20T01:45:58.610 INFO:teuthology.orchestra.run.smithi044.stdout: auth debug = true 2024-04-20T01:45:58.610 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.610 INFO:teuthology.orchestra.run.smithi044.stdout: ms die on old message = true 2024-04-20T01:45:58.610 INFO:teuthology.orchestra.run.smithi044.stdout: ms die on bug = true 2024-04-20T01:45:58.610 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.610 INFO:teuthology.orchestra.run.smithi044.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T01:45:58.610 INFO:teuthology.orchestra.run.smithi044.stdout: mon pg warn max object skew = 0 2024-04-20T01:45:58.610 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.610 INFO:teuthology.orchestra.run.smithi044.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T01:45:58.610 INFO:teuthology.orchestra.run.smithi044.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T01:45:58.610 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.611 INFO:teuthology.orchestra.run.smithi044.stdout: osd pool default size = 2 2024-04-20T01:45:58.611 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.611 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd allow primary affinity = true 2024-04-20T01:45:58.611 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd allow pg remap = true 2024-04-20T01:45:58.611 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on legacy crush tunables = false 2024-04-20T01:45:58.611 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on crush straw calc version zero = false 2024-04-20T01:45:58.611 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on no sortbitwise = false 2024-04-20T01:45:58.611 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on osd down out interval zero = false 2024-04-20T01:45:58.611 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on too few osds = false 2024-04-20T01:45:58.611 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T01:45:58.611 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T01:45:58.611 INFO:teuthology.orchestra.run.smithi044.stdout: mon_allow_pool_size_one = true 2024-04-20T01:45:58.611 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.611 INFO:teuthology.orchestra.run.smithi044.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T01:45:58.611 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.612 INFO:teuthology.orchestra.run.smithi044.stdout: osd default data pool replay window = 5 2024-04-20T01:45:58.612 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.612 INFO:teuthology.orchestra.run.smithi044.stdout: mon allow pool delete = true 2024-04-20T01:45:58.612 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.612 INFO:teuthology.orchestra.run.smithi044.stdout: mon cluster log file level = debug 2024-04-20T01:45:58.612 INFO:teuthology.orchestra.run.smithi044.stdout: debug asserts on shutdown = true 2024-04-20T01:45:58.612 INFO:teuthology.orchestra.run.smithi044.stdout: mon health detail to clog = false 2024-04-20T01:45:58.612 INFO:teuthology.orchestra.run.smithi044.stdout: mon host = "172.21.15.44,172.21.15.121,172.21.15.123" 2024-04-20T01:45:58.612 INFO:teuthology.orchestra.run.smithi044.stdout: mon client directed command retry = 5 2024-04-20T01:45:58.612 INFO:teuthology.orchestra.run.smithi044.stdout: mon election default strategy = 1 2024-04-20T01:45:58.612 INFO:teuthology.orchestra.run.smithi044.stdout: ms inject socket failures = 5000 2024-04-20T01:45:58.612 INFO:teuthology.orchestra.run.smithi044.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-04-20T01:45:58.612 INFO:teuthology.orchestra.run.smithi044.stdout: fsid = 9c2f64c1-aa35-4337-9f5c-081b9e3a2938 2024-04-20T01:45:58.612 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.612 INFO:teuthology.orchestra.run.smithi044.stdout:[osd] 2024-04-20T01:45:58.613 INFO:teuthology.orchestra.run.smithi044.stdout: osd journal size = 100 2024-04-20T01:45:58.613 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.613 INFO:teuthology.orchestra.run.smithi044.stdout: osd scrub load threshold = 5.0 2024-04-20T01:45:58.613 INFO:teuthology.orchestra.run.smithi044.stdout: osd scrub max interval = 120 2024-04-20T01:45:58.613 INFO:teuthology.orchestra.run.smithi044.stdout: osd mclock profile = high_recovery_ops 2024-04-20T01:45:58.613 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.613 INFO:teuthology.orchestra.run.smithi044.stdout: osd recover clone overlap = true 2024-04-20T01:45:58.613 INFO:teuthology.orchestra.run.smithi044.stdout: osd recovery max chunk = 1048576 2024-04-20T01:45:58.613 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.613 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug shutdown = true 2024-04-20T01:45:58.613 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug op order = true 2024-04-20T01:45:58.613 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug verify stray on activate = true 2024-04-20T01:45:58.613 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.613 INFO:teuthology.orchestra.run.smithi044.stdout: osd open classes on start = true 2024-04-20T01:45:58.613 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug pg log writeout = true 2024-04-20T01:45:58.614 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.614 INFO:teuthology.orchestra.run.smithi044.stdout: osd deep scrub update digest min age = 30 2024-04-20T01:45:58.614 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.614 INFO:teuthology.orchestra.run.smithi044.stdout: osd map max advance = 10 2024-04-20T01:45:58.614 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.614 INFO:teuthology.orchestra.run.smithi044.stdout: journal zero on create = true 2024-04-20T01:45:58.614 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.614 INFO:teuthology.orchestra.run.smithi044.stdout: filestore ondisk finisher threads = 3 2024-04-20T01:45:58.614 INFO:teuthology.orchestra.run.smithi044.stdout: filestore apply finisher threads = 3 2024-04-20T01:45:58.614 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.614 INFO:teuthology.orchestra.run.smithi044.stdout: bdev debug aio = true 2024-04-20T01:45:58.614 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug misdirected ops = true 2024-04-20T01:45:58.614 INFO:teuthology.orchestra.run.smithi044.stdout: debug monc = 20 2024-04-20T01:45:58.614 INFO:teuthology.orchestra.run.smithi044.stdout: bdev async discard = True 2024-04-20T01:45:58.615 INFO:teuthology.orchestra.run.smithi044.stdout: bdev enable discard = True 2024-04-20T01:45:58.615 INFO:teuthology.orchestra.run.smithi044.stdout: bluestore allocator = stupid 2024-04-20T01:45:58.615 INFO:teuthology.orchestra.run.smithi044.stdout: bluestore block size = 96636764160 2024-04-20T01:45:58.615 INFO:teuthology.orchestra.run.smithi044.stdout: bluestore fsck on mount = True 2024-04-20T01:45:58.615 INFO:teuthology.orchestra.run.smithi044.stdout: bluestore zero block detection = True 2024-04-20T01:45:58.615 INFO:teuthology.orchestra.run.smithi044.stdout: debug bluefs = 20 2024-04-20T01:45:58.615 INFO:teuthology.orchestra.run.smithi044.stdout: debug bluestore = 20 2024-04-20T01:45:58.615 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-04-20T01:45:58.615 INFO:teuthology.orchestra.run.smithi044.stdout: debug osd = 20 2024-04-20T01:45:58.615 INFO:teuthology.orchestra.run.smithi044.stdout: debug rocksdb = 10 2024-04-20T01:45:58.615 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T01:45:58.615 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd full ratio = 0.9 2024-04-20T01:45:58.615 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd nearfull ratio = 0.8 2024-04-20T01:45:58.615 INFO:teuthology.orchestra.run.smithi044.stdout: osd blocked scrub grace period = 3600 2024-04-20T01:45:58.615 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug reject backfill probability = 0.3 2024-04-20T01:45:58.615 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug verify cached snaps = True 2024-04-20T01:45:58.615 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug verify missing on start = True 2024-04-20T01:45:58.616 INFO:teuthology.orchestra.run.smithi044.stdout: osd failsafe full ratio = 0.95 2024-04-20T01:45:58.616 INFO:teuthology.orchestra.run.smithi044.stdout: osd heartbeat use min delay socket = True 2024-04-20T01:45:58.616 INFO:teuthology.orchestra.run.smithi044.stdout: osd max backfills = 3 2024-04-20T01:45:58.616 INFO:teuthology.orchestra.run.smithi044.stdout: osd max markdown count = 1000 2024-04-20T01:45:58.616 INFO:teuthology.orchestra.run.smithi044.stdout: osd mclock override recovery settings = True 2024-04-20T01:45:58.616 INFO:teuthology.orchestra.run.smithi044.stdout: osd objectstore = bluestore 2024-04-20T01:45:58.616 INFO:teuthology.orchestra.run.smithi044.stdout: osd op queue = debug_random 2024-04-20T01:45:58.616 INFO:teuthology.orchestra.run.smithi044.stdout: osd op queue cut off = debug_random 2024-04-20T01:45:58.616 INFO:teuthology.orchestra.run.smithi044.stdout: osd scrub min interval = 60 2024-04-20T01:45:58.616 INFO:teuthology.orchestra.run.smithi044.stdout: osd shutdown pgref assert = True 2024-04-20T01:45:58.616 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.616 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr] 2024-04-20T01:45:58.616 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-04-20T01:45:58.616 INFO:teuthology.orchestra.run.smithi044.stdout: debug mgr = 20 2024-04-20T01:45:58.616 INFO:teuthology.orchestra.run.smithi044.stdout: debug mon = 20 2024-04-20T01:45:58.616 INFO:teuthology.orchestra.run.smithi044.stdout: debug auth = 20 2024-04-20T01:45:58.617 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min pgs per osd = 4 2024-04-20T01:45:58.617 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min bytes per osd = 10 2024-04-20T01:45:58.617 INFO:teuthology.orchestra.run.smithi044.stdout: mgr/telemetry/nag = false 2024-04-20T01:45:58.617 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.617 INFO:teuthology.orchestra.run.smithi044.stdout:[mon] 2024-04-20T01:45:58.617 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-04-20T01:45:58.617 INFO:teuthology.orchestra.run.smithi044.stdout: debug mon = 20 2024-04-20T01:45:58.617 INFO:teuthology.orchestra.run.smithi044.stdout: debug paxos = 20 2024-04-20T01:45:58.617 INFO:teuthology.orchestra.run.smithi044.stdout: debug auth = 20 2024-04-20T01:45:58.617 INFO:teuthology.orchestra.run.smithi044.stdout: mon data avail warn = 5 2024-04-20T01:45:58.617 INFO:teuthology.orchestra.run.smithi044.stdout: mon mgr mkfs grace = 240 2024-04-20T01:45:58.617 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min pgs per osd = 4 2024-04-20T01:45:58.617 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd reporter subtree level = osd 2024-04-20T01:45:58.617 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd prime pg temp = true 2024-04-20T01:45:58.617 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min bytes per osd = 10 2024-04-20T01:45:58.617 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.618 INFO:teuthology.orchestra.run.smithi044.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T01:45:58.618 INFO:teuthology.orchestra.run.smithi044.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T01:45:58.618 INFO:teuthology.orchestra.run.smithi044.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T01:45:58.618 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.618 INFO:teuthology.orchestra.run.smithi044.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T01:45:58.618 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T01:45:58.618 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T01:45:58.618 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.618 INFO:teuthology.orchestra.run.smithi044.stdout: # 1m isn't quite enough 2024-04-20T01:45:58.618 INFO:teuthology.orchestra.run.smithi044.stdout: mon_down_mkfs_grace = 2m 2024-04-20T01:45:58.618 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.618 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_filestore_osds = false 2024-04-20T01:45:58.618 INFO:teuthology.orchestra.run.smithi044.stdout: mon scrub interval = 300 2024-04-20T01:45:58.618 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:45:58.618 INFO:teuthology.orchestra.run.smithi044.stdout:[client] 2024-04-20T01:45:58.619 INFO:teuthology.orchestra.run.smithi044.stdout: rgw cache enabled = true 2024-04-20T01:45:58.619 INFO:teuthology.orchestra.run.smithi044.stdout: rgw enable ops log = true 2024-04-20T01:45:58.619 INFO:teuthology.orchestra.run.smithi044.stdout: rgw enable usage log = true 2024-04-20T01:45:58.619 INFO:teuthology.orchestra.run.smithi044.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T01:45:58.619 INFO:teuthology.orchestra.run.smithi044.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T01:45:58.619 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.a] 2024-04-20T01:45:58.619 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.b] 2024-04-20T01:45:58.619 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.c] 2024-04-20T01:45:58.621 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-20T01:45:58.621 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-20T01:45:58.731 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-20T01:45:58.731 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:45:58.731 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-20T01:45:58.751 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:45:58.751 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-20T01:45:58.812 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi044.front.sepia.ceph.com 2024-04-20T01:45:58.813 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:45:58.813 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-20T01:45:58.813 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T01:45:58.900 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:45:58.900 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:45:58.961 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi121.front.sepia.ceph.com 2024-04-20T01:45:58.961 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T01:45:58.961 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-20T01:45:58.962 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T01:45:59.006 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T01:45:59.006 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:45:59.065 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi123.front.sepia.ceph.com 2024-04-20T01:45:59.065 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T01:45:59.065 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-20T01:45:59.066 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T01:45:59.110 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T01:45:59.110 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:45:59.169 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi165.front.sepia.ceph.com 2024-04-20T01:45:59.169 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T01:45:59.169 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-20T01:45:59.169 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T01:45:59.213 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T01:45:59.213 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:45:59.273 INFO:tasks.ceph:Setting up mon nodes... 2024-04-20T01:45:59.273 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-20T01:45:59.273 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-20T01:45:59.340 INFO:teuthology.orchestra.run.smithi044.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-20T01:45:59.343 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-20T01:45:59.423 INFO:teuthology.orchestra.run.smithi165.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-20T01:45:59.427 INFO:tasks.ceph:Setting up mds nodes... 2024-04-20T01:45:59.427 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-20T01:45:59.428 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-20T01:45:59.473 INFO:teuthology.orchestra.run.smithi165.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-20T01:45:59.487 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-20T01:45:59.487 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi044.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi121.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi165.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2024-04-20T01:45:59.487 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-20T01:45:59.521 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-04-20T01:45:59.521 INFO:tasks.ceph:role: osd.0 2024-04-20T01:45:59.521 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-20T01:45:59.522 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-20T01:45:59.603 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T01:45:59.603 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T01:45:59.603 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T01:45:59.603 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T01:45:59.603 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T01:45:59.603 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-04-20T01:45:59.603 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T01:45:59.604 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T01:45:59.604 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T01:45:59.604 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T01:46:00.032 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-04-20T01:46:00.034 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-04-20T01:46:00.035 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-20T01:46:00.224 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-20T01:46:00.267 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-20T01:46:00.343 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-04-20T01:46:00.343 INFO:tasks.ceph:role: osd.4 2024-04-20T01:46:00.343 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-20T01:46:00.343 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-20T01:46:00.425 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T01:46:00.425 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T01:46:00.425 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T01:46:00.425 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T01:46:00.425 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T01:46:00.425 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-04-20T01:46:00.425 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T01:46:00.425 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T01:46:00.425 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T01:46:00.425 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T01:46:00.846 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-04-20T01:46:00.849 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-04-20T01:46:00.849 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-04-20T01:46:00.975 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-20T01:46:01.015 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2024-04-20T01:46:01.089 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-04-20T01:46:01.090 INFO:tasks.ceph:role: osd.8 2024-04-20T01:46:01.090 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-20T01:46:01.090 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-20T01:46:01.172 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T01:46:01.173 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T01:46:01.173 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T01:46:01.173 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T01:46:01.173 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T01:46:01.173 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-04-20T01:46:01.173 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T01:46:01.173 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T01:46:01.173 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T01:46:01.173 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T01:46:01.618 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-04-20T01:46:01.620 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-04-20T01:46:01.620 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-8 2024-04-20T01:46:01.778 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-8 2024-04-20T01:46:01.820 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-12 2024-04-20T01:46:01.895 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-04-20T01:46:01.895 INFO:tasks.ceph:role: osd.12 2024-04-20T01:46:01.896 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-20T01:46:01.896 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-20T01:46:01.977 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T01:46:01.977 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T01:46:01.977 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T01:46:01.977 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T01:46:01.977 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T01:46:01.977 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-04-20T01:46:01.977 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T01:46:01.977 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T01:46:01.978 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T01:46:01.978 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T01:46:02.414 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-04-20T01:46:02.416 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-04-20T01:46:02.416 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-12 2024-04-20T01:46:02.578 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-12 2024-04-20T01:46:02.619 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:46:02.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:02.733+0000 7fc79d76b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-20T01:46:02.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:02.733+0000 7fc79d76b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-20T01:46:02.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:02.733+0000 7fc79d76b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T01:46:02.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:02.733+0000 7fc79d76b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T01:46:02.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:02.733+0000 7fc79d76b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T01:46:02.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:02.734+0000 7fc79d76b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-20T01:46:05.576 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-20T01:46:05.612 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:46:05.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:05.725+0000 7fa6f70007c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-20T01:46:05.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:05.725+0000 7fa6f70007c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-20T01:46:05.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:05.725+0000 7fa6f70007c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T01:46:05.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:05.725+0000 7fa6f70007c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T01:46:05.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:05.725+0000 7fa6f70007c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T01:46:05.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:05.726+0000 7fa6f70007c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-20T01:46:08.564 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-20T01:46:08.599 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:46:08.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:08.712+0000 7f64e7aa17c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2024-04-20T01:46:08.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:08.713+0000 7f64e7aa17c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2024-04-20T01:46:08.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:08.713+0000 7f64e7aa17c0 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-04-20T01:46:08.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:08.713+0000 7f64e7aa17c0 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-04-20T01:46:08.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:08.713+0000 7f64e7aa17c0 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-04-20T01:46:08.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:08.713+0000 7f64e7aa17c0 -1 bluestore(/var/lib/ceph/osd/ceph-8) _read_fsid unparsable uuid 2024-04-20T01:46:11.555 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2024-04-20T01:46:11.590 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 12 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:46:11.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:11.703+0000 7f913cc307c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-12/keyring: can't open /var/lib/ceph/osd/ceph-12/keyring: (2) No such file or directory 2024-04-20T01:46:11.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:11.704+0000 7f913cc307c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-12/keyring 2024-04-20T01:46:11.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:11.704+0000 7f913cc307c0 -1 bluestore(/var/lib/ceph/osd/ceph-12/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-12/block: (2) No such file or directory 2024-04-20T01:46:11.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:11.704+0000 7f913cc307c0 -1 bluestore(/var/lib/ceph/osd/ceph-12/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-12/block: (2) No such file or directory 2024-04-20T01:46:11.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:11.704+0000 7f913cc307c0 -1 bluestore(/var/lib/ceph/osd/ceph-12/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-12/block: (2) No such file or directory 2024-04-20T01:46:11.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:46:11.704+0000 7f913cc307c0 -1 bluestore(/var/lib/ceph/osd/ceph-12) _read_fsid unparsable uuid 2024-04-20T01:46:14.553 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-12 2024-04-20T01:46:14.587 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-20T01:46:14.620 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-04-20T01:46:14.620 INFO:tasks.ceph:role: osd.1 2024-04-20T01:46:14.620 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-20T01:46:14.620 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-20T01:46:14.697 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T01:46:14.697 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T01:46:14.697 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T01:46:14.697 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T01:46:14.697 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T01:46:14.698 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-04-20T01:46:14.698 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T01:46:14.698 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T01:46:14.698 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T01:46:14.698 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T01:46:15.127 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-04-20T01:46:15.129 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-04-20T01:46:15.129 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-04-20T01:46:15.298 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-20T01:46:15.332 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-20T01:46:15.401 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-04-20T01:46:15.401 INFO:tasks.ceph:role: osd.5 2024-04-20T01:46:15.401 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-20T01:46:15.401 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-20T01:46:15.476 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T01:46:15.476 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T01:46:15.476 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T01:46:15.476 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T01:46:15.476 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T01:46:15.476 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-04-20T01:46:15.476 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T01:46:15.476 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T01:46:15.477 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T01:46:15.477 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T01:46:15.899 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-04-20T01:46:15.901 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-04-20T01:46:15.901 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-20T01:46:16.012 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-20T01:46:16.044 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2024-04-20T01:46:16.111 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-04-20T01:46:16.111 INFO:tasks.ceph:role: osd.9 2024-04-20T01:46:16.111 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-20T01:46:16.111 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-20T01:46:16.185 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T01:46:16.185 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T01:46:16.185 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T01:46:16.185 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T01:46:16.185 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T01:46:16.185 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-04-20T01:46:16.185 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T01:46:16.185 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T01:46:16.185 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T01:46:16.185 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T01:46:16.632 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-04-20T01:46:16.634 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-04-20T01:46:16.634 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-9 2024-04-20T01:46:16.782 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-9 2024-04-20T01:46:16.813 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-13 2024-04-20T01:46:16.880 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-04-20T01:46:16.880 INFO:tasks.ceph:role: osd.13 2024-04-20T01:46:16.880 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-20T01:46:16.880 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-20T01:46:16.954 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T01:46:16.955 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T01:46:16.955 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T01:46:16.955 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T01:46:16.955 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T01:46:16.955 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-04-20T01:46:16.955 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T01:46:16.955 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T01:46:16.955 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T01:46:16.955 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T01:46:17.405 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-04-20T01:46:17.407 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-04-20T01:46:17.407 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-13 2024-04-20T01:46:17.592 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-13 2024-04-20T01:46:17.625 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:46:17.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:17.743+0000 7f84d65a27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-20T01:46:17.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:17.743+0000 7f84d65a27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-20T01:46:17.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:17.743+0000 7f84d65a27c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T01:46:17.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:17.743+0000 7f84d65a27c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T01:46:17.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:17.743+0000 7f84d65a27c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T01:46:17.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:17.744+0000 7f84d65a27c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-20T01:46:20.584 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-20T01:46:20.616 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:46:20.706 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:20.720+0000 7f661ac0f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-20T01:46:20.707 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:20.720+0000 7f661ac0f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-20T01:46:20.707 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:20.720+0000 7f661ac0f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T01:46:20.707 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:20.720+0000 7f661ac0f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T01:46:20.707 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:20.720+0000 7f661ac0f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T01:46:20.707 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:20.720+0000 7f661ac0f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-20T01:46:23.556 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-20T01:46:23.586 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:46:23.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:23.691+0000 7f70cc3947c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2024-04-20T01:46:23.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:23.691+0000 7f70cc3947c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2024-04-20T01:46:23.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:23.691+0000 7f70cc3947c0 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-04-20T01:46:23.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:23.691+0000 7f70cc3947c0 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-04-20T01:46:23.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:23.691+0000 7f70cc3947c0 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-04-20T01:46:23.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:23.692+0000 7f70cc3947c0 -1 bluestore(/var/lib/ceph/osd/ceph-9) _read_fsid unparsable uuid 2024-04-20T01:46:26.527 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2024-04-20T01:46:26.559 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 13 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:46:26.650 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:26.664+0000 7f02a88457c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-13/keyring: can't open /var/lib/ceph/osd/ceph-13/keyring: (2) No such file or directory 2024-04-20T01:46:26.651 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:26.664+0000 7f02a88457c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-13/keyring 2024-04-20T01:46:26.651 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:26.664+0000 7f02a88457c0 -1 bluestore(/var/lib/ceph/osd/ceph-13/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-13/block: (2) No such file or directory 2024-04-20T01:46:26.651 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:26.664+0000 7f02a88457c0 -1 bluestore(/var/lib/ceph/osd/ceph-13/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-13/block: (2) No such file or directory 2024-04-20T01:46:26.651 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:26.664+0000 7f02a88457c0 -1 bluestore(/var/lib/ceph/osd/ceph-13/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-13/block: (2) No such file or directory 2024-04-20T01:46:26.651 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T01:46:26.664+0000 7f02a88457c0 -1 bluestore(/var/lib/ceph/osd/ceph-13) _read_fsid unparsable uuid 2024-04-20T01:46:29.501 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-13 2024-04-20T01:46:29.533 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-20T01:46:29.567 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-04-20T01:46:29.568 INFO:tasks.ceph:role: osd.2 2024-04-20T01:46:29.568 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-20T01:46:29.568 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-20T01:46:29.648 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T01:46:29.648 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T01:46:29.648 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T01:46:29.648 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T01:46:29.648 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T01:46:29.649 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-04-20T01:46:29.649 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T01:46:29.649 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T01:46:29.649 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T01:46:29.649 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T01:46:30.089 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-04-20T01:46:30.092 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-04-20T01:46:30.092 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-04-20T01:46:30.265 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-20T01:46:30.303 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-20T01:46:30.374 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-04-20T01:46:30.374 INFO:tasks.ceph:role: osd.6 2024-04-20T01:46:30.374 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-20T01:46:30.374 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-20T01:46:30.451 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T01:46:30.451 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T01:46:30.451 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T01:46:30.451 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T01:46:30.451 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T01:46:30.451 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-04-20T01:46:30.451 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T01:46:30.451 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T01:46:30.451 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T01:46:30.452 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T01:46:30.894 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-04-20T01:46:30.896 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-04-20T01:46:30.896 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6 2024-04-20T01:46:31.010 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-20T01:46:31.043 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-10 2024-04-20T01:46:31.112 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-04-20T01:46:31.112 INFO:tasks.ceph:role: osd.10 2024-04-20T01:46:31.112 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-20T01:46:31.112 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-20T01:46:31.188 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T01:46:31.188 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T01:46:31.188 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T01:46:31.188 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T01:46:31.188 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T01:46:31.188 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-04-20T01:46:31.188 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T01:46:31.188 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T01:46:31.188 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T01:46:31.189 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T01:46:31.639 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-04-20T01:46:31.641 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-04-20T01:46:31.641 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-10 2024-04-20T01:46:31.800 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-10 2024-04-20T01:46:31.836 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-14 2024-04-20T01:46:31.907 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-04-20T01:46:31.907 INFO:tasks.ceph:role: osd.14 2024-04-20T01:46:31.907 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-20T01:46:31.907 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-20T01:46:31.984 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T01:46:31.984 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T01:46:31.984 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T01:46:31.984 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T01:46:31.985 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T01:46:31.985 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-04-20T01:46:31.985 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T01:46:31.985 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T01:46:31.985 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T01:46:31.985 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T01:46:32.426 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-04-20T01:46:32.428 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-04-20T01:46:32.428 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-14 2024-04-20T01:46:32.596 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-14 2024-04-20T01:46:32.636 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:46:32.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:32.757+0000 7fbb9e9467c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-20T01:46:32.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:32.757+0000 7fbb9e9467c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-20T01:46:32.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:32.757+0000 7fbb9e9467c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T01:46:32.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:32.757+0000 7fbb9e9467c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T01:46:32.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:32.757+0000 7fbb9e9467c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T01:46:32.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:32.758+0000 7fbb9e9467c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-20T01:46:35.596 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-20T01:46:35.630 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:46:35.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:35.739+0000 7f87c9dfc7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-20T01:46:35.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:35.739+0000 7f87c9dfc7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-20T01:46:35.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:35.739+0000 7f87c9dfc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T01:46:35.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:35.739+0000 7f87c9dfc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T01:46:35.726 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:35.739+0000 7f87c9dfc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T01:46:35.727 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:35.740+0000 7f87c9dfc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-20T01:46:38.581 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-20T01:46:38.614 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 10 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:46:38.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:38.726+0000 7f047eb997c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-10/keyring: can't open /var/lib/ceph/osd/ceph-10/keyring: (2) No such file or directory 2024-04-20T01:46:38.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:38.726+0000 7f047eb997c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-10/keyring 2024-04-20T01:46:38.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:38.726+0000 7f047eb997c0 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-04-20T01:46:38.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:38.726+0000 7f047eb997c0 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-04-20T01:46:38.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:38.726+0000 7f047eb997c0 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-04-20T01:46:38.714 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:38.727+0000 7f047eb997c0 -1 bluestore(/var/lib/ceph/osd/ceph-10) _read_fsid unparsable uuid 2024-04-20T01:46:41.575 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-10 2024-04-20T01:46:41.608 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 14 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:46:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:41.718+0000 7f3ad1c557c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-14/keyring: can't open /var/lib/ceph/osd/ceph-14/keyring: (2) No such file or directory 2024-04-20T01:46:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:41.718+0000 7f3ad1c557c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-14/keyring 2024-04-20T01:46:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:41.718+0000 7f3ad1c557c0 -1 bluestore(/var/lib/ceph/osd/ceph-14/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-14/block: (2) No such file or directory 2024-04-20T01:46:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:41.718+0000 7f3ad1c557c0 -1 bluestore(/var/lib/ceph/osd/ceph-14/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-14/block: (2) No such file or directory 2024-04-20T01:46:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:41.718+0000 7f3ad1c557c0 -1 bluestore(/var/lib/ceph/osd/ceph-14/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-14/block: (2) No such file or directory 2024-04-20T01:46:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T01:46:41.719+0000 7f3ad1c557c0 -1 bluestore(/var/lib/ceph/osd/ceph-14) _read_fsid unparsable uuid 2024-04-20T01:46:44.556 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-14 2024-04-20T01:46:44.589 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-20T01:46:44.624 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-04-20T01:46:44.624 INFO:tasks.ceph:role: osd.3 2024-04-20T01:46:44.624 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-20T01:46:44.624 DEBUG:teuthology.orchestra.run.smithi165:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-20T01:46:44.703 INFO:teuthology.orchestra.run.smithi165.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T01:46:44.703 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T01:46:44.703 INFO:teuthology.orchestra.run.smithi165.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T01:46:44.703 INFO:teuthology.orchestra.run.smithi165.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T01:46:44.704 INFO:teuthology.orchestra.run.smithi165.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T01:46:44.704 INFO:teuthology.orchestra.run.smithi165.stdout: = sunit=0 swidth=0 blks 2024-04-20T01:46:44.704 INFO:teuthology.orchestra.run.smithi165.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T01:46:44.704 INFO:teuthology.orchestra.run.smithi165.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T01:46:44.704 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T01:46:44.704 INFO:teuthology.orchestra.run.smithi165.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T01:46:45.134 INFO:teuthology.orchestra.run.smithi165.stdout:Discarding blocks...Done. 2024-04-20T01:46:45.138 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi165.front.sepia.ceph.com -o noatime 2024-04-20T01:46:45.138 DEBUG:teuthology.orchestra.run.smithi165:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-04-20T01:46:45.332 DEBUG:teuthology.orchestra.run.smithi165:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-20T01:46:45.369 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-20T01:46:45.439 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-04-20T01:46:45.439 INFO:tasks.ceph:role: osd.7 2024-04-20T01:46:45.439 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-20T01:46:45.439 DEBUG:teuthology.orchestra.run.smithi165:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-20T01:46:45.514 INFO:teuthology.orchestra.run.smithi165.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T01:46:45.514 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T01:46:45.514 INFO:teuthology.orchestra.run.smithi165.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T01:46:45.515 INFO:teuthology.orchestra.run.smithi165.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T01:46:45.515 INFO:teuthology.orchestra.run.smithi165.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T01:46:45.515 INFO:teuthology.orchestra.run.smithi165.stdout: = sunit=0 swidth=0 blks 2024-04-20T01:46:45.515 INFO:teuthology.orchestra.run.smithi165.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T01:46:45.515 INFO:teuthology.orchestra.run.smithi165.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T01:46:45.515 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T01:46:45.515 INFO:teuthology.orchestra.run.smithi165.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T01:46:45.934 INFO:teuthology.orchestra.run.smithi165.stdout:Discarding blocks...Done. 2024-04-20T01:46:45.936 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi165.front.sepia.ceph.com -o noatime 2024-04-20T01:46:45.936 DEBUG:teuthology.orchestra.run.smithi165:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-7 2024-04-20T01:46:46.051 DEBUG:teuthology.orchestra.run.smithi165:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-20T01:46:46.085 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/osd/ceph-11 2024-04-20T01:46:46.156 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-04-20T01:46:46.156 INFO:tasks.ceph:role: osd.11 2024-04-20T01:46:46.156 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-20T01:46:46.156 DEBUG:teuthology.orchestra.run.smithi165:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-20T01:46:46.231 INFO:teuthology.orchestra.run.smithi165.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T01:46:46.231 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T01:46:46.231 INFO:teuthology.orchestra.run.smithi165.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T01:46:46.231 INFO:teuthology.orchestra.run.smithi165.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T01:46:46.231 INFO:teuthology.orchestra.run.smithi165.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T01:46:46.231 INFO:teuthology.orchestra.run.smithi165.stdout: = sunit=0 swidth=0 blks 2024-04-20T01:46:46.232 INFO:teuthology.orchestra.run.smithi165.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T01:46:46.232 INFO:teuthology.orchestra.run.smithi165.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T01:46:46.232 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T01:46:46.232 INFO:teuthology.orchestra.run.smithi165.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T01:46:46.673 INFO:teuthology.orchestra.run.smithi165.stdout:Discarding blocks...Done. 2024-04-20T01:46:46.675 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi165.front.sepia.ceph.com -o noatime 2024-04-20T01:46:46.675 DEBUG:teuthology.orchestra.run.smithi165:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-11 2024-04-20T01:46:46.856 DEBUG:teuthology.orchestra.run.smithi165:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-11 2024-04-20T01:46:46.892 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/osd/ceph-15 2024-04-20T01:46:46.962 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-04-20T01:46:46.962 INFO:tasks.ceph:role: osd.15 2024-04-20T01:46:46.963 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-20T01:46:46.963 DEBUG:teuthology.orchestra.run.smithi165:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-20T01:46:47.039 INFO:teuthology.orchestra.run.smithi165.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T01:46:47.039 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T01:46:47.039 INFO:teuthology.orchestra.run.smithi165.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T01:46:47.039 INFO:teuthology.orchestra.run.smithi165.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-20T01:46:47.039 INFO:teuthology.orchestra.run.smithi165.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T01:46:47.039 INFO:teuthology.orchestra.run.smithi165.stdout: = sunit=0 swidth=0 blks 2024-04-20T01:46:47.040 INFO:teuthology.orchestra.run.smithi165.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T01:46:47.040 INFO:teuthology.orchestra.run.smithi165.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-20T01:46:47.040 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T01:46:47.040 INFO:teuthology.orchestra.run.smithi165.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T01:46:47.485 INFO:teuthology.orchestra.run.smithi165.stdout:Discarding blocks...Done. 2024-04-20T01:46:47.488 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi165.front.sepia.ceph.com -o noatime 2024-04-20T01:46:47.488 DEBUG:teuthology.orchestra.run.smithi165:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-15 2024-04-20T01:46:47.669 DEBUG:teuthology.orchestra.run.smithi165:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-15 2024-04-20T01:46:47.706 DEBUG:teuthology.orchestra.run.smithi165:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:46:47.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:47.812+0000 7f48bbc307c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-20T01:46:47.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:47.813+0000 7f48bbc307c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-20T01:46:47.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:47.813+0000 7f48bbc307c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T01:46:47.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:47.813+0000 7f48bbc307c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T01:46:47.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:47.813+0000 7f48bbc307c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T01:46:47.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:47.813+0000 7f48bbc307c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-20T01:46:50.647 DEBUG:teuthology.orchestra.run.smithi165:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-20T01:46:50.677 DEBUG:teuthology.orchestra.run.smithi165:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:46:50.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:50.779+0000 7f0cde70a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-20T01:46:50.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:50.779+0000 7f0cde70a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-20T01:46:50.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:50.780+0000 7f0cde70a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T01:46:50.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:50.780+0000 7f0cde70a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T01:46:50.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:50.780+0000 7f0cde70a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T01:46:50.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:50.780+0000 7f0cde70a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-20T01:46:53.623 DEBUG:teuthology.orchestra.run.smithi165:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-20T01:46:53.654 DEBUG:teuthology.orchestra.run.smithi165:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 11 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:46:53.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:53.757+0000 7f7feadc77c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-11/keyring: can't open /var/lib/ceph/osd/ceph-11/keyring: (2) No such file or directory 2024-04-20T01:46:53.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:53.757+0000 7f7feadc77c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-11/keyring 2024-04-20T01:46:53.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:53.757+0000 7f7feadc77c0 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-04-20T01:46:53.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:53.757+0000 7f7feadc77c0 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-04-20T01:46:53.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:53.757+0000 7f7feadc77c0 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-04-20T01:46:53.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:53.758+0000 7f7feadc77c0 -1 bluestore(/var/lib/ceph/osd/ceph-11) _read_fsid unparsable uuid 2024-04-20T01:46:56.592 DEBUG:teuthology.orchestra.run.smithi165:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-11 2024-04-20T01:46:56.622 DEBUG:teuthology.orchestra.run.smithi165:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 15 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:46:56.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:56.726+0000 7f42557eb7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-15/keyring: can't open /var/lib/ceph/osd/ceph-15/keyring: (2) No such file or directory 2024-04-20T01:46:56.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:56.726+0000 7f42557eb7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-15/keyring 2024-04-20T01:46:56.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:56.726+0000 7f42557eb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-15/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-15/block: (2) No such file or directory 2024-04-20T01:46:56.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:56.726+0000 7f42557eb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-15/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-15/block: (2) No such file or directory 2024-04-20T01:46:56.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:56.726+0000 7f42557eb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-15/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-15/block: (2) No such file or directory 2024-04-20T01:46:56.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-20T01:46:56.727+0000 7f42557eb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-15) _read_fsid unparsable uuid 2024-04-20T01:46:59.557 DEBUG:teuthology.orchestra.run.smithi165:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-15 2024-04-20T01:46:59.590 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-20T01:46:59.590 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:46:59.590 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-20T01:46:59.626 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:46:59.626 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-20T01:46:59.701 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:46:59.701 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-20T01:46:59.775 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:46:59.776 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2024-04-20T01:46:59.850 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:46:59.850 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-12/keyring of=/dev/stdout 2024-04-20T01:46:59.925 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T01:46:59.925 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-20T01:46:59.958 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T01:46:59.958 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-20T01:47:00.033 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T01:47:00.033 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2024-04-20T01:47:00.104 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T01:47:00.104 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-13/keyring of=/dev/stdout 2024-04-20T01:47:00.175 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T01:47:00.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-20T01:47:00.208 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T01:47:00.209 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-20T01:47:00.281 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T01:47:00.281 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-10/keyring of=/dev/stdout 2024-04-20T01:47:00.354 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T01:47:00.354 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-14/keyring of=/dev/stdout 2024-04-20T01:47:00.427 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T01:47:00.427 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-20T01:47:00.459 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T01:47:00.459 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-20T01:47:00.531 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T01:47:00.531 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-20T01:47:00.601 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T01:47:00.601 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-11/keyring of=/dev/stdout 2024-04-20T01:47:00.673 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T01:47:00.674 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-15/keyring of=/dev/stdout 2024-04-20T01:47:00.739 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T01:47:00.740 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-20T01:47:00.798 INFO:tasks.ceph:Adding keys to all mons... 2024-04-20T01:47:00.798 DEBUG:teuthology.orchestra.run.smithi044:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-20T01:47:00.801 DEBUG:teuthology.orchestra.run.smithi121:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-20T01:47:00.804 DEBUG:teuthology.orchestra.run.smithi123:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-20T01:47:00.832 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr.y] 2024-04-20T01:47:00.832 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBXHiNmKyA0FRAAzwaQFzlXh9YaN2bYw6RDhA== 2024-04-20T01:47:00.832 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.0] 2024-04-20T01:47:00.832 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBaHiNmusPDKxAA1XM3i6FjXOY+lqntJGhsNw== 2024-04-20T01:47:00.832 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.4] 2024-04-20T01:47:00.832 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBdHiNmkSFMKxAA2Wse7d4odSrFSbNddM7VIQ== 2024-04-20T01:47:00.832 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.8] 2024-04-20T01:47:00.832 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBgHiNmv0SLKhAAJycllnCbFDEwkfHNLbV5Vg== 2024-04-20T01:47:00.833 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.12] 2024-04-20T01:47:00.833 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBjHiNmELADKhAAHRe5Id8uF9GqKA8Ed1idbQ== 2024-04-20T01:47:00.833 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.1] 2024-04-20T01:47:00.833 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBpHiNmns9gLBAAXRxqrKpQ+37UKUEu/1WMkQ== 2024-04-20T01:47:00.833 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.5] 2024-04-20T01:47:00.833 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBsHiNmEMP5KhAASITSubMt623GZdk2BFum/g== 2024-04-20T01:47:00.833 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.9] 2024-04-20T01:47:00.833 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBvHiNmc2BCKRAAY19Lta+Kxx5EBQYOtkF1lA== 2024-04-20T01:47:00.833 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.13] 2024-04-20T01:47:00.833 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQByHiNmNeekJxAAx6Oxs5D1WQ+KWrbyFsJ7WA== 2024-04-20T01:47:00.833 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.2] 2024-04-20T01:47:00.833 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB4HiNmkCU1LRAA1zk9RBfcdiUYUOOfdvt2CQ== 2024-04-20T01:47:00.833 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.6] 2024-04-20T01:47:00.833 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB7HiNm4EAjLBAAo+xiZZNoaZ/jnGWxm8JOHw== 2024-04-20T01:47:00.833 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.10] 2024-04-20T01:47:00.834 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQB+HiNmRbRiKxAATFlohooG79XRgLQSf4XT7g== 2024-04-20T01:47:00.834 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.14] 2024-04-20T01:47:00.834 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCBHiNmKMjkKhAAHc+gDdXLS8RjZSZpf30GaQ== 2024-04-20T01:47:00.834 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr.x] 2024-04-20T01:47:00.834 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBXHiNmoVkiGhAAIFe4VekSzOp0T3ctiAYN1w== 2024-04-20T01:47:00.834 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.3] 2024-04-20T01:47:00.834 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCHHiNminp6MBAApOASU2SlzCNiE1AOXG5ApA== 2024-04-20T01:47:00.834 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.7] 2024-04-20T01:47:00.834 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCKHiNmvn6BLhAAl/P6RjNnEIPnvBLh7yoIcg== 2024-04-20T01:47:00.834 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.11] 2024-04-20T01:47:00.834 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCNHiNmOeAvLRAA3ORAioD62IzJQH0no+aDMQ== 2024-04-20T01:47:00.834 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.15] 2024-04-20T01:47:00.834 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCQHiNmzLlUKxAAr7F53B/3j0hzdZJozbx63A== 2024-04-20T01:47:00.834 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-04-20T01:47:00.835 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQBXHiNmlkMfHRAAGZUx8Fn6JVYRKaKxHimxdw== 2024-04-20T01:47:00.837 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.y] 2024-04-20T01:47:00.837 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBXHiNmKyA0FRAAzwaQFzlXh9YaN2bYw6RDhA== 2024-04-20T01:47:00.837 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.0] 2024-04-20T01:47:00.837 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBaHiNmusPDKxAA1XM3i6FjXOY+lqntJGhsNw== 2024-04-20T01:47:00.837 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.4] 2024-04-20T01:47:00.837 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBdHiNmkSFMKxAA2Wse7d4odSrFSbNddM7VIQ== 2024-04-20T01:47:00.837 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.8] 2024-04-20T01:47:00.837 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBgHiNmv0SLKhAAJycllnCbFDEwkfHNLbV5Vg== 2024-04-20T01:47:00.837 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.12] 2024-04-20T01:47:00.837 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBjHiNmELADKhAAHRe5Id8uF9GqKA8Ed1idbQ== 2024-04-20T01:47:00.837 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.1] 2024-04-20T01:47:00.837 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBpHiNmns9gLBAAXRxqrKpQ+37UKUEu/1WMkQ== 2024-04-20T01:47:00.837 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.5] 2024-04-20T01:47:00.837 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBsHiNmEMP5KhAASITSubMt623GZdk2BFum/g== 2024-04-20T01:47:00.838 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.9] 2024-04-20T01:47:00.838 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBvHiNmc2BCKRAAY19Lta+Kxx5EBQYOtkF1lA== 2024-04-20T01:47:00.838 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.13] 2024-04-20T01:47:00.838 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQByHiNmNeekJxAAx6Oxs5D1WQ+KWrbyFsJ7WA== 2024-04-20T01:47:00.838 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.2] 2024-04-20T01:47:00.838 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQB4HiNmkCU1LRAA1zk9RBfcdiUYUOOfdvt2CQ== 2024-04-20T01:47:00.838 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.6] 2024-04-20T01:47:00.838 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQB7HiNm4EAjLBAAo+xiZZNoaZ/jnGWxm8JOHw== 2024-04-20T01:47:00.838 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.10] 2024-04-20T01:47:00.838 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQB+HiNmRbRiKxAATFlohooG79XRgLQSf4XT7g== 2024-04-20T01:47:00.838 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.14] 2024-04-20T01:47:00.838 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCBHiNmKMjkKhAAHc+gDdXLS8RjZSZpf30GaQ== 2024-04-20T01:47:00.838 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.x] 2024-04-20T01:47:00.838 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBXHiNmoVkiGhAAIFe4VekSzOp0T3ctiAYN1w== 2024-04-20T01:47:00.839 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.3] 2024-04-20T01:47:00.839 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCHHiNminp6MBAApOASU2SlzCNiE1AOXG5ApA== 2024-04-20T01:47:00.839 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.7] 2024-04-20T01:47:00.839 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCKHiNmvn6BLhAAl/P6RjNnEIPnvBLh7yoIcg== 2024-04-20T01:47:00.839 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.11] 2024-04-20T01:47:00.839 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCNHiNmOeAvLRAA3ORAioD62IzJQH0no+aDMQ== 2024-04-20T01:47:00.839 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.15] 2024-04-20T01:47:00.839 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCQHiNmzLlUKxAAr7F53B/3j0hzdZJozbx63A== 2024-04-20T01:47:00.839 INFO:teuthology.orchestra.run.smithi123.stdout:[client.0] 2024-04-20T01:47:00.839 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBXHiNmlkMfHRAAGZUx8Fn6JVYRKaKxHimxdw== 2024-04-20T01:47:00.839 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr.y] 2024-04-20T01:47:00.839 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQBXHiNmKyA0FRAAzwaQFzlXh9YaN2bYw6RDhA== 2024-04-20T01:47:00.839 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.0] 2024-04-20T01:47:00.839 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQBaHiNmusPDKxAA1XM3i6FjXOY+lqntJGhsNw== 2024-04-20T01:47:00.840 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.4] 2024-04-20T01:47:00.840 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQBdHiNmkSFMKxAA2Wse7d4odSrFSbNddM7VIQ== 2024-04-20T01:47:00.840 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.8] 2024-04-20T01:47:00.840 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQBgHiNmv0SLKhAAJycllnCbFDEwkfHNLbV5Vg== 2024-04-20T01:47:00.840 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.12] 2024-04-20T01:47:00.840 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQBjHiNmELADKhAAHRe5Id8uF9GqKA8Ed1idbQ== 2024-04-20T01:47:00.840 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.1] 2024-04-20T01:47:00.840 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQBpHiNmns9gLBAAXRxqrKpQ+37UKUEu/1WMkQ== 2024-04-20T01:47:00.840 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.5] 2024-04-20T01:47:00.840 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQBsHiNmEMP5KhAASITSubMt623GZdk2BFum/g== 2024-04-20T01:47:00.840 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.9] 2024-04-20T01:47:00.840 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQBvHiNmc2BCKRAAY19Lta+Kxx5EBQYOtkF1lA== 2024-04-20T01:47:00.840 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.13] 2024-04-20T01:47:00.840 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQByHiNmNeekJxAAx6Oxs5D1WQ+KWrbyFsJ7WA== 2024-04-20T01:47:00.840 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.2] 2024-04-20T01:47:00.841 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQB4HiNmkCU1LRAA1zk9RBfcdiUYUOOfdvt2CQ== 2024-04-20T01:47:00.841 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.6] 2024-04-20T01:47:00.841 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQB7HiNm4EAjLBAAo+xiZZNoaZ/jnGWxm8JOHw== 2024-04-20T01:47:00.841 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.10] 2024-04-20T01:47:00.841 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQB+HiNmRbRiKxAATFlohooG79XRgLQSf4XT7g== 2024-04-20T01:47:00.841 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.14] 2024-04-20T01:47:00.841 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCBHiNmKMjkKhAAHc+gDdXLS8RjZSZpf30GaQ== 2024-04-20T01:47:00.841 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr.x] 2024-04-20T01:47:00.841 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQBXHiNmoVkiGhAAIFe4VekSzOp0T3ctiAYN1w== 2024-04-20T01:47:00.841 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.3] 2024-04-20T01:47:00.841 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCHHiNminp6MBAApOASU2SlzCNiE1AOXG5ApA== 2024-04-20T01:47:00.841 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.7] 2024-04-20T01:47:00.841 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCKHiNmvn6BLhAAl/P6RjNnEIPnvBLh7yoIcg== 2024-04-20T01:47:00.841 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.11] 2024-04-20T01:47:00.841 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCNHiNmOeAvLRAA3ORAioD62IzJQH0no+aDMQ== 2024-04-20T01:47:00.842 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.15] 2024-04-20T01:47:00.842 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQCQHiNmzLlUKxAAr7F53B/3j0hzdZJozbx63A== 2024-04-20T01:47:00.842 INFO:teuthology.orchestra.run.smithi121.stdout:[client.0] 2024-04-20T01:47:00.842 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQBXHiNmlkMfHRAAGZUx8Fn6JVYRKaKxHimxdw== 2024-04-20T01:47:00.842 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T01:47:00.879 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T01:47:00.882 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T01:47:00.943 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:00.945 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:00.947 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.006 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.008 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.011 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.067 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.069 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.071 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.127 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.129 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.132 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.187 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.189 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.191 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.247 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.249 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.251 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.306 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.309 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.311 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.367 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.370 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.372 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.426 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.429 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.431 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.486 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.488 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.490 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.547 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.549 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.552 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.608 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.610 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.612 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.668 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T01:47:01.670 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T01:47:01.672 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T01:47:01.727 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.729 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.731 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.787 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.789 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.791 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.847 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.849 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.851 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.906 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.909 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.911 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T01:47:01.966 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T01:47:01.969 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T01:47:01.971 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T01:47:02.029 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-20T01:47:02.029 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-20T01:47:02.059 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T01:47:02.168 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-20T01:47:02.199 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-20T01:47:02.226 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T01:47:02.326 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-20T01:47:02.352 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-20T01:47:02.379 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T01:47:02.477 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-20T01:47:02.503 DEBUG:teuthology.orchestra.run.smithi044:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:47:02.506 DEBUG:teuthology.orchestra.run.smithi121:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:47:02.508 DEBUG:teuthology.orchestra.run.smithi123:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-20T01:47:02.560 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-20T01:47:02.560 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-20T01:47:02.560 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-20T01:47:02.567 INFO:tasks.ceph.mon.a:Started 2024-04-20T01:47:02.567 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-20T01:47:02.567 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-20T01:47:02.570 INFO:tasks.ceph.mon.b:Started 2024-04-20T01:47:02.570 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-20T01:47:02.570 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-20T01:47:02.602 INFO:tasks.ceph.mon.c:Started 2024-04-20T01:47:02.602 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-20T01:47:02.602 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-20T01:47:02.603 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-20T01:47:02.605 INFO:tasks.ceph.mgr.y:Started 2024-04-20T01:47:02.605 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-20T01:47:02.605 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-20T01:47:02.607 INFO:tasks.ceph.mgr.x:Started 2024-04-20T01:47:02.608 INFO:tasks.ceph:Setting crush tunables to default 2024-04-20T01:47:02.608 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph osd crush tunables default 2024-04-20T01:47:03.051 INFO:teuthology.orchestra.run.smithi044.stderr:adjusted tunables profile to default 2024-04-20T01:47:03.064 INFO:tasks.ceph:check_enable_crimson: False 2024-04-20T01:47:03.064 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-20T01:47:03.064 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:47:03.064 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-20T01:47:03.094 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:47:03.094 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-20T01:47:03.166 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:47:03.166 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2024-04-20T01:47:03.235 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-20T01:47:03.236 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-12/fsid of=/dev/stdout 2024-04-20T01:47:03.306 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T01:47:03.306 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-20T01:47:03.339 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T01:47:03.339 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-20T01:47:03.372 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:03.385+0000 7f701cc56200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T01:47:03.411 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T01:47:03.412 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2024-04-20T01:47:03.451 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:03.464+0000 7f701cc56200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T01:47:03.453 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:03.467+0000 7f63afc94200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T01:47:03.481 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T01:47:03.481 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-13/fsid of=/dev/stdout 2024-04-20T01:47:03.524 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:03.537+0000 7f701cc56200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T01:47:03.544 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:03.558+0000 7f63afc94200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T01:47:03.550 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T01:47:03.550 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-20T01:47:03.579 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T01:47:03.579 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-20T01:47:03.593 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:03.607+0000 7f701cc56200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T01:47:03.628 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:03.642+0000 7f63afc94200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T01:47:03.647 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T01:47:03.647 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-10/fsid of=/dev/stdout 2024-04-20T01:47:03.708 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:03.722+0000 7f63afc94200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T01:47:03.716 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T01:47:03.716 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-14/fsid of=/dev/stdout 2024-04-20T01:47:03.766 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:03.779+0000 7f701cc56200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T01:47:03.783 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T01:47:03.783 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-20T01:47:03.810 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T01:47:03.810 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-20T01:47:03.877 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T01:47:03.877 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-11/fsid of=/dev/stdout 2024-04-20T01:47:03.905 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:03.919+0000 7f63afc94200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T01:47:03.940 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:03.954+0000 7f701cc56200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T01:47:03.943 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-20T01:47:03.943 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-15/fsid of=/dev/stdout 2024-04-20T01:47:04.012 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 3903b88f-ff2b-4252-9480-92b701b1e59b 0 2024-04-20T01:47:04.055 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:04.069+0000 7f701cc56200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T01:47:04.093 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:04.107+0000 7f63afc94200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T01:47:04.157 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:04.170+0000 7f701cc56200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T01:47:04.220 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:04.233+0000 7f63afc94200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T01:47:04.236 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:04.250+0000 7f701cc56200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T01:47:04.273 INFO:teuthology.orchestra.run.smithi165.stdout:0 2024-04-20T01:47:04.285 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new c5ff0175-b4f0-40a5-86c7-7a0a351bf070 1 2024-04-20T01:47:04.327 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:04.341+0000 7f63afc94200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T01:47:04.414 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:04.428+0000 7f63afc94200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T01:47:04.467 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:04.480+0000 7f701cc56200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T01:47:04.542 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:04.555+0000 7f701cc56200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T01:47:04.567 INFO:teuthology.orchestra.run.smithi165.stdout:1 2024-04-20T01:47:04.579 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 7af325c1-36b5-420c-a2a9-44c00a002977 2 2024-04-20T01:47:04.678 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:04.692+0000 7f63afc94200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T01:47:04.760 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:04.774+0000 7f63afc94200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T01:47:04.796 INFO:teuthology.orchestra.run.smithi165.stdout:2 2024-04-20T01:47:04.808 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 99334f43-83ea-4c3f-b91d-ae72c8361b8c 3 2024-04-20T01:47:04.981 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:04.994+0000 7f701cc56200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T01:47:05.024 INFO:teuthology.orchestra.run.smithi165.stdout:3 2024-04-20T01:47:05.037 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 8764d799-af67-4e6c-889f-9fdd91201cc2 4 2024-04-20T01:47:05.194 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:05.208+0000 7f63afc94200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T01:47:05.255 INFO:teuthology.orchestra.run.smithi165.stdout:4 2024-04-20T01:47:05.268 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 2556b2fa-4dcc-479c-a379-6f9fe96e9c77 5 2024-04-20T01:47:05.322 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:05.335+0000 7f701cc56200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T01:47:05.484 INFO:teuthology.orchestra.run.smithi165.stdout:5 2024-04-20T01:47:05.491 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:05.504+0000 7f701cc56200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T01:47:05.497 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new baeaae3a-fff4-4ccc-a08a-ff32db4531ba 6 2024-04-20T01:47:05.562 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:05.575+0000 7f63afc94200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T01:47:05.577 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:05.591+0000 7f701cc56200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T01:47:05.715 INFO:teuthology.orchestra.run.smithi165.stdout:6 2024-04-20T01:47:05.728 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 0019f005-3e62-4d9d-b334-f1d347a484ce 7 2024-04-20T01:47:05.744 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:05.758+0000 7f63afc94200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T01:47:05.834 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:05.847+0000 7f63afc94200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T01:47:05.948 INFO:teuthology.orchestra.run.smithi165.stdout:7 2024-04-20T01:47:05.961 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 6f5200a3-8980-4241-a6fc-ef604fa612b2 8 2024-04-20T01:47:06.176 INFO:teuthology.orchestra.run.smithi165.stdout:8 2024-04-20T01:47:06.189 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 29cf9aaa-3a99-4605-92f4-64f1f6a1728d 9 2024-04-20T01:47:06.332 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:06.345+0000 7f701cc56200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T01:47:06.406 INFO:teuthology.orchestra.run.smithi165.stdout:9 2024-04-20T01:47:06.418 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:06.431+0000 7f701cc56200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T01:47:06.419 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 705906c4-1514-4754-ac40-45226a837180 10 2024-04-20T01:47:06.562 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:06.575+0000 7f701cc56200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T01:47:06.634 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:06.648+0000 7f63afc94200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T01:47:06.640 INFO:teuthology.orchestra.run.smithi165.stdout:10 2024-04-20T01:47:06.654 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new b45f2645-72d1-4d72-89e9-2212f5aa074c 11 2024-04-20T01:47:06.728 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:06.742+0000 7f63afc94200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T01:47:06.879 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:06.893+0000 7f63afc94200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T01:47:06.895 INFO:teuthology.orchestra.run.smithi165.stdout:11 2024-04-20T01:47:06.908 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 3ea410b9-a368-467b-a179-2e3aba562d63 12 2024-04-20T01:47:07.146 INFO:teuthology.orchestra.run.smithi165.stdout:12 2024-04-20T01:47:07.159 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 15a5154b-e7cb-4b57-8468-dcb048d93f63 13 2024-04-20T01:47:07.173 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:07.186+0000 7f701cc56200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T01:47:07.355 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:07.368+0000 7f701cc56200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T01:47:07.376 INFO:teuthology.orchestra.run.smithi165.stdout:13 2024-04-20T01:47:07.388 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 23b53da2-f848-4e76-9632-4d9effd5b430 14 2024-04-20T01:47:07.515 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:07.528+0000 7f63afc94200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T01:47:07.530 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:07.544+0000 7f701cc56200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T01:47:07.647 INFO:teuthology.orchestra.run.smithi165.stdout:14 2024-04-20T01:47:07.662 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 546efc0f-82a3-47d2-aeec-2d849eb1faa1 15 2024-04-20T01:47:07.706 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:07.720+0000 7f63afc94200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T01:47:07.883 INFO:teuthology.orchestra.run.smithi165.stdout:15 2024-04-20T01:47:07.895 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-20T01:47:07.896 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-20T01:47:07.896 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:07.910+0000 7f63afc94200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T01:47:07.898 INFO:tasks.ceph.osd.0:Started 2024-04-20T01:47:07.898 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-20T01:47:07.898 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-20T01:47:07.900 INFO:tasks.ceph.osd.4:Started 2024-04-20T01:47:07.900 INFO:tasks.ceph.osd.8:Restarting daemon 2024-04-20T01:47:07.900 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 8 2024-04-20T01:47:07.902 INFO:tasks.ceph.osd.8:Started 2024-04-20T01:47:07.903 INFO:tasks.ceph.osd.12:Restarting daemon 2024-04-20T01:47:07.903 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 12 2024-04-20T01:47:07.905 INFO:tasks.ceph.osd.12:Started 2024-04-20T01:47:07.905 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-20T01:47:07.905 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-20T01:47:07.908 INFO:tasks.ceph.osd.1:Started 2024-04-20T01:47:07.908 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-20T01:47:07.908 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-20T01:47:07.910 INFO:tasks.ceph.osd.5:Started 2024-04-20T01:47:07.910 INFO:tasks.ceph.osd.9:Restarting daemon 2024-04-20T01:47:07.910 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 9 2024-04-20T01:47:07.912 INFO:tasks.ceph.osd.9:Started 2024-04-20T01:47:07.913 INFO:tasks.ceph.osd.13:Restarting daemon 2024-04-20T01:47:07.913 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 13 2024-04-20T01:47:07.915 INFO:tasks.ceph.osd.13:Started 2024-04-20T01:47:07.915 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-20T01:47:07.915 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-20T01:47:07.918 INFO:tasks.ceph.osd.2:Started 2024-04-20T01:47:07.918 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-20T01:47:07.918 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-20T01:47:07.920 INFO:tasks.ceph.osd.6:Started 2024-04-20T01:47:07.920 INFO:tasks.ceph.osd.10:Restarting daemon 2024-04-20T01:47:07.920 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 10 2024-04-20T01:47:07.922 INFO:tasks.ceph.osd.10:Started 2024-04-20T01:47:07.922 INFO:tasks.ceph.osd.14:Restarting daemon 2024-04-20T01:47:07.922 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 14 2024-04-20T01:47:07.925 INFO:tasks.ceph.osd.14:Started 2024-04-20T01:47:07.925 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-20T01:47:07.925 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-20T01:47:07.927 INFO:tasks.ceph.osd.3:Started 2024-04-20T01:47:07.927 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-20T01:47:07.927 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-20T01:47:07.929 INFO:tasks.ceph.osd.7:Started 2024-04-20T01:47:07.929 INFO:tasks.ceph.osd.11:Restarting daemon 2024-04-20T01:47:07.929 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 11 2024-04-20T01:47:07.931 INFO:tasks.ceph.osd.11:Started 2024-04-20T01:47:07.931 INFO:tasks.ceph.osd.15:Restarting daemon 2024-04-20T01:47:07.932 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 15 2024-04-20T01:47:07.933 INFO:tasks.ceph.osd.15:Started 2024-04-20T01:47:07.934 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T01:47:08.195 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:47:08.195 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":18,"fsid":"9c2f64c1-aa35-4337-9f5c-081b9e3a2938","created":"2024-04-20T01:47:02.934241+0000","modified":"2024-04-20T01:47:07.894719+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T01:47:07.894719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3903b88f-ff2b-4252-9480-92b701b1e59b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c5ff0175-b4f0-40a5-86c7-7a0a351bf070","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7af325c1-36b5-420c-a2a9-44c00a002977","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"99334f43-83ea-4c3f-b91d-ae72c8361b8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8764d799-af67-4e6c-889f-9fdd91201cc2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2556b2fa-4dcc-479c-a379-6f9fe96e9c77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"baeaae3a-fff4-4ccc-a08a-ff32db4531ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0019f005-3e62-4d9d-b334-f1d347a484ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"6f5200a3-8980-4241-a6fc-ef604fa612b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"29cf9aaa-3a99-4605-92f4-64f1f6a1728d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"705906c4-1514-4754-ac40-45226a837180","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"b45f2645-72d1-4d72-89e9-2212f5aa074c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"3ea410b9-a368-467b-a179-2e3aba562d63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"15a5154b-e7cb-4b57-8468-dcb048d93f63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"23b53da2-f848-4e76-9632-4d9effd5b430","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"546efc0f-82a3-47d2-aeec-2d849eb1faa1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T01:47:08.209 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-20T01:47:08.210 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-20T01:47:08.395 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:08.408+0000 7f701cc56200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T01:47:08.520 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:08.533+0000 7f98794317c0 -1 Falling back to public interface 2024-04-20T01:47:08.525 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:08.539+0000 7f58aa8cb7c0 -1 Falling back to public interface 2024-04-20T01:47:08.534 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:08.548+0000 7f45445d37c0 -1 Falling back to public interface 2024-04-20T01:47:08.538 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:08.553+0000 7f01cb8937c0 -1 Falling back to public interface 2024-04-20T01:47:08.538 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:47:08.553+0000 7ff1285067c0 -1 Falling back to public interface 2024-04-20T01:47:08.540 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:08.555+0000 7f01c2c51700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6818/0: (98) Address already in use 2024-04-20T01:47:08.541 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:47:08.555+0000 7feacb4fc7c0 -1 Falling back to public interface 2024-04-20T01:47:08.542 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:08.557+0000 7f9e061c27c0 -1 Falling back to public interface 2024-04-20T01:47:08.543 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:08.556+0000 7fa942a177c0 -1 Falling back to public interface 2024-04-20T01:47:08.548 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:47:08.563+0000 7fb5fd26b7c0 -1 Falling back to public interface 2024-04-20T01:47:08.549 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:47:08.563+0000 7ffb09b577c0 -1 Falling back to public interface 2024-04-20T01:47:08.550 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:08.563+0000 7f1b76d387c0 -1 Falling back to public interface 2024-04-20T01:47:08.550 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:08.564+0000 7fd954ae57c0 -1 Falling back to public interface 2024-04-20T01:47:08.552 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:47:08.566+0000 7f3bdbf327c0 -1 Falling back to public interface 2024-04-20T01:47:08.555 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:08.569+0000 7fb0002897c0 -1 Falling back to public interface 2024-04-20T01:47:08.560 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:08.573+0000 7f785dde97c0 -1 Falling back to public interface 2024-04-20T01:47:08.565 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:47:08.578+0000 7f0f63f667c0 -1 Falling back to public interface 2024-04-20T01:47:08.642 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:08.655+0000 7f701cc56200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T01:47:08.717 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:08.730+0000 7f701cc56200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T01:47:08.861 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:08.875+0000 7f63afc94200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T01:47:09.098 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:09.111+0000 7f701cc56200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T01:47:09.130 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:09.144+0000 7f63afc94200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T01:47:09.215 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:09.229+0000 7f63afc94200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T01:47:09.651 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-20T01:47:09.664+0000 7f63afc94200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T01:47:09.820 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T01:47:10.038 INFO:teuthology.misc.health.smithi044.stdout: 2024-04-20T01:47:10.038 INFO:teuthology.misc.health.smithi044.stdout:{"epoch":18,"fsid":"9c2f64c1-aa35-4337-9f5c-081b9e3a2938","created":"2024-04-20T01:47:02.934241+0000","modified":"2024-04-20T01:47:07.894719+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T01:47:07.894719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3903b88f-ff2b-4252-9480-92b701b1e59b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c5ff0175-b4f0-40a5-86c7-7a0a351bf070","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7af325c1-36b5-420c-a2a9-44c00a002977","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"99334f43-83ea-4c3f-b91d-ae72c8361b8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8764d799-af67-4e6c-889f-9fdd91201cc2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2556b2fa-4dcc-479c-a379-6f9fe96e9c77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"baeaae3a-fff4-4ccc-a08a-ff32db4531ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0019f005-3e62-4d9d-b334-f1d347a484ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"6f5200a3-8980-4241-a6fc-ef604fa612b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"29cf9aaa-3a99-4605-92f4-64f1f6a1728d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"705906c4-1514-4754-ac40-45226a837180","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"b45f2645-72d1-4d72-89e9-2212f5aa074c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"3ea410b9-a368-467b-a179-2e3aba562d63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"15a5154b-e7cb-4b57-8468-dcb048d93f63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"23b53da2-f848-4e76-9632-4d9effd5b430","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"546efc0f-82a3-47d2-aeec-2d849eb1faa1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T01:47:10.050 DEBUG:teuthology.misc:0 of 16 OSDs are up 2024-04-20T01:47:10.127 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:10.140+0000 7f6fd7ecc700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-20T01:47:10.127 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:47:10.140+0000 7f6fd7ecc700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-20T01:47:11.989 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:12.003+0000 7f45445d37c0 -1 osd.6 0 log_to_monitors true 2024-04-20T01:47:11.989 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:12.003+0000 7f98794317c0 -1 osd.5 0 log_to_monitors true 2024-04-20T01:47:12.006 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:12.019+0000 7fa942a177c0 -1 osd.7 0 log_to_monitors true 2024-04-20T01:47:12.024 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:12.038+0000 7f1b76d387c0 -1 osd.11 0 log_to_monitors true 2024-04-20T01:47:12.034 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:12.048+0000 7f01cb8937c0 -1 osd.2 0 log_to_monitors true 2024-04-20T01:47:12.045 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:12.059+0000 7f58aa8cb7c0 -1 osd.13 0 log_to_monitors true 2024-04-20T01:47:12.046 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:12.060+0000 7f785dde97c0 -1 osd.15 0 log_to_monitors true 2024-04-20T01:47:12.065 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:12.079+0000 7fd954ae57c0 -1 osd.1 0 log_to_monitors true 2024-04-20T01:47:12.080 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:12.095+0000 7f9e061c27c0 -1 osd.14 0 log_to_monitors true 2024-04-20T01:47:12.082 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:47:12.096+0000 7f3bdbf327c0 -1 osd.0 0 log_to_monitors true 2024-04-20T01:47:12.085 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:47:12.099+0000 7feacb4fc7c0 -1 osd.12 0 log_to_monitors true 2024-04-20T01:47:12.087 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:12.101+0000 7fb0002897c0 -1 osd.9 0 log_to_monitors true 2024-04-20T01:47:12.087 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:47:12.101+0000 7fb5fd26b7c0 -1 osd.4 0 log_to_monitors true 2024-04-20T01:47:12.087 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:47:12.101+0000 7ffb09b577c0 -1 osd.8 0 log_to_monitors true 2024-04-20T01:47:12.088 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:47:12.102+0000 7f0f63f667c0 -1 osd.3 0 log_to_monitors true 2024-04-20T01:47:12.092 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:47:12.107+0000 7ff1285067c0 -1 osd.10 0 log_to_monitors true 2024-04-20T01:47:13.967 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:13.981+0000 7f453818a700 -1 osd.6 0 waiting for initial osdmap 2024-04-20T01:47:13.972 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:13.987+0000 7f986cfe8700 -1 osd.5 0 waiting for initial osdmap 2024-04-20T01:47:13.973 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:47:13.987+0000 7feabf0b3700 -1 osd.12 0 waiting for initial osdmap 2024-04-20T01:47:13.976 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:47:13.990+0000 7fb5f0e22700 -1 osd.4 0 waiting for initial osdmap 2024-04-20T01:47:13.976 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:47:13.990+0000 7ffafd70e700 -1 osd.8 0 waiting for initial osdmap 2024-04-20T01:47:13.976 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:47:13.990+0000 7f3bcfae9700 -1 osd.0 0 waiting for initial osdmap 2024-04-20T01:47:13.977 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:47:13.990+0000 7f0f57b1d700 -1 osd.3 0 waiting for initial osdmap 2024-04-20T01:47:13.977 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:13.990+0000 7fd949e9f700 -1 osd.1 0 waiting for initial osdmap 2024-04-20T01:47:13.977 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:13.990+0000 7f01bf44a700 -1 osd.2 0 waiting for initial osdmap 2024-04-20T01:47:13.977 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:13.990+0000 7fa9365ce700 -1 osd.7 0 waiting for initial osdmap 2024-04-20T01:47:13.978 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:13.990+0000 7faff3e40700 -1 osd.9 0 waiting for initial osdmap 2024-04-20T01:47:13.978 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:13.990+0000 7f9df9d79700 -1 osd.14 0 waiting for initial osdmap 2024-04-20T01:47:13.979 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:13.993+0000 7f78519a0700 -1 osd.15 0 waiting for initial osdmap 2024-04-20T01:47:13.986 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:47:14.000+0000 7ffaf9af1700 -1 osd.8 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T01:47:13.986 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:47:14.000+0000 7feabb496700 -1 osd.12 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T01:47:13.989 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:47:14.003+0000 7fb5ed205700 -1 osd.4 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T01:47:13.989 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:14.003+0000 7f98693cb700 -1 osd.5 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T01:47:13.991 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:14.004+0000 7f01bb82d700 -1 osd.2 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T01:47:13.991 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:14.004+0000 7f9df615c700 -1 osd.14 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T01:47:13.994 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:14.007+0000 7f784dd83700 -1 osd.15 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T01:47:13.997 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:14.011+0000 7faff0223700 -1 osd.9 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T01:47:13.997 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:14.011+0000 7fd944a7f700 -1 osd.1 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T01:47:13.998 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:14.012+0000 7f453456d700 -1 osd.6 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T01:47:14.001 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:47:14.015+0000 7f0f53f00700 -1 osd.3 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T01:47:14.002 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:14.015+0000 7fa9329b1700 -1 osd.7 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T01:47:14.002 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:47:14.016+0000 7f3bcbecc700 -1 osd.0 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T01:47:14.735 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:47:14.749+0000 7ff11c0bd700 -1 osd.10 0 waiting for initial osdmap 2024-04-20T01:47:14.751 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:47:14.765+0000 7ff114498700 -1 osd.10 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T01:47:14.763 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:14.776+0000 7f1b6a8ef700 -1 osd.11 0 waiting for initial osdmap 2024-04-20T01:47:14.779 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:14.793+0000 7f1b62cca700 -1 osd.11 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T01:47:15.188 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:15.202+0000 7f589e482700 -1 osd.13 0 waiting for initial osdmap 2024-04-20T01:47:15.205 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:15.219+0000 7f589685d700 -1 osd.13 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T01:47:17.656 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T01:47:18.023 INFO:teuthology.misc.health.smithi044.stdout: 2024-04-20T01:47:18.023 INFO:teuthology.misc.health.smithi044.stdout:{"epoch":24,"fsid":"9c2f64c1-aa35-4337-9f5c-081b9e3a2938","created":"2024-04-20T01:47:02.934241+0000","modified":"2024-04-20T01:47:18.003812+0000","last_up_change":"2024-04-20T01:47:15.986754+0000","last_in_change":"2024-04-20T01:47:07.894719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T01:47:15.195181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3903b88f-ff2b-4252-9480-92b701b1e59b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":43518},{"type":"v1","addr":"172.21.15.44:6825","nonce":43518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":43518},{"type":"v1","addr":"172.21.15.44:6827","nonce":43518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":43518},{"type":"v1","addr":"172.21.15.44:6831","nonce":43518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":43518},{"type":"v1","addr":"172.21.15.44:6829","nonce":43518}]},"public_addr":"172.21.15.44:6825/43518","cluster_addr":"172.21.15.44:6827/43518","heartbeat_back_addr":"172.21.15.44:6831/43518","heartbeat_front_addr":"172.21.15.44:6829/43518","state":["exists","up"]},{"osd":1,"uuid":"c5ff0175-b4f0-40a5-86c7-7a0a351bf070","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":74361},{"type":"v1","addr":"172.21.15.121:6817","nonce":74361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":74361},{"type":"v1","addr":"172.21.15.121:6819","nonce":74361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":74361},{"type":"v1","addr":"172.21.15.121:6823","nonce":74361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":74361},{"type":"v1","addr":"172.21.15.121:6821","nonce":74361}]},"public_addr":"172.21.15.121:6817/74361","cluster_addr":"172.21.15.121:6819/74361","heartbeat_back_addr":"172.21.15.121:6823/74361","heartbeat_front_addr":"172.21.15.121:6821/74361","state":["exists","up"]},{"osd":2,"uuid":"7af325c1-36b5-420c-a2a9-44c00a002977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6809","nonce":43108},{"type":"v1","addr":"172.21.15.123:6811","nonce":43108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":43108},{"type":"v1","addr":"172.21.15.123:6816","nonce":43108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":43108},{"type":"v1","addr":"172.21.15.123:6823","nonce":43108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":43108},{"type":"v1","addr":"172.21.15.123:6821","nonce":43108}]},"public_addr":"172.21.15.123:6811/43108","cluster_addr":"172.21.15.123:6816/43108","heartbeat_back_addr":"172.21.15.123:6823/43108","heartbeat_front_addr":"172.21.15.123:6821/43108","state":["exists","up"]},{"osd":3,"uuid":"99334f43-83ea-4c3f-b91d-ae72c8361b8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":74545},{"type":"v1","addr":"172.21.15.165:6825","nonce":74545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":74545},{"type":"v1","addr":"172.21.15.165:6827","nonce":74545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":74545},{"type":"v1","addr":"172.21.15.165:6831","nonce":74545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":74545},{"type":"v1","addr":"172.21.15.165:6829","nonce":74545}]},"public_addr":"172.21.15.165:6825/74545","cluster_addr":"172.21.15.165:6827/74545","heartbeat_back_addr":"172.21.15.165:6831/74545","heartbeat_front_addr":"172.21.15.165:6829/74545","state":["exists","up"]},{"osd":4,"uuid":"8764d799-af67-4e6c-889f-9fdd91201cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":43517},{"type":"v1","addr":"172.21.15.44:6811","nonce":43517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":43517},{"type":"v1","addr":"172.21.15.44:6815","nonce":43517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":43517},{"type":"v1","addr":"172.21.15.44:6823","nonce":43517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6817","nonce":43517},{"type":"v1","addr":"172.21.15.44:6819","nonce":43517}]},"public_addr":"172.21.15.44:6811/43517","cluster_addr":"172.21.15.44:6815/43517","heartbeat_back_addr":"172.21.15.44:6823/43517","heartbeat_front_addr":"172.21.15.44:6819/43517","state":["exists","up"]},{"osd":5,"uuid":"2556b2fa-4dcc-479c-a379-6f9fe96e9c77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":74354},{"type":"v1","addr":"172.21.15.121:6801","nonce":74354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":74354},{"type":"v1","addr":"172.21.15.121:6803","nonce":74354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":74354},{"type":"v1","addr":"172.21.15.121:6807","nonce":74354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":74354},{"type":"v1","addr":"172.21.15.121:6805","nonce":74354}]},"public_addr":"172.21.15.121:6801/74354","cluster_addr":"172.21.15.121:6803/74354","heartbeat_back_addr":"172.21.15.121:6807/74354","heartbeat_front_addr":"172.21.15.121:6805/74354","state":["exists","up"]},{"osd":6,"uuid":"baeaae3a-fff4-4ccc-a08a-ff32db4531ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":43107},{"type":"v1","addr":"172.21.15.123:6801","nonce":43107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":43107},{"type":"v1","addr":"172.21.15.123:6803","nonce":43107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":43107},{"type":"v1","addr":"172.21.15.123:6807","nonce":43107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":43107},{"type":"v1","addr":"172.21.15.123:6805","nonce":43107}]},"public_addr":"172.21.15.123:6801/43107","cluster_addr":"172.21.15.123:6803/43107","heartbeat_back_addr":"172.21.15.123:6807/43107","heartbeat_front_addr":"172.21.15.123:6805/43107","state":["exists","up"]},{"osd":7,"uuid":"0019f005-3e62-4d9d-b334-f1d347a484ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":74543},{"type":"v1","addr":"172.21.15.165:6801","nonce":74543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":74543},{"type":"v1","addr":"172.21.15.165:6803","nonce":74543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":74543},{"type":"v1","addr":"172.21.15.165:6807","nonce":74543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":74543},{"type":"v1","addr":"172.21.15.165:6805","nonce":74543}]},"public_addr":"172.21.15.165:6801/74543","cluster_addr":"172.21.15.165:6803/74543","heartbeat_back_addr":"172.21.15.165:6807/74543","heartbeat_front_addr":"172.21.15.165:6805/74543","state":["exists","up"]},{"osd":8,"uuid":"6f5200a3-8980-4241-a6fc-ef604fa612b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":43515},{"type":"v1","addr":"172.21.15.44:6810","nonce":43515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":43515},{"type":"v1","addr":"172.21.15.44:6814","nonce":43515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":43515},{"type":"v1","addr":"172.21.15.44:6822","nonce":43515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":43515},{"type":"v1","addr":"172.21.15.44:6818","nonce":43515}]},"public_addr":"172.21.15.44:6810/43515","cluster_addr":"172.21.15.44:6814/43515","heartbeat_back_addr":"172.21.15.44:6822/43515","heartbeat_front_addr":"172.21.15.44:6818/43515","state":["exists","up"]},{"osd":9,"uuid":"29cf9aaa-3a99-4605-92f4-64f1f6a1728d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":74379},{"type":"v1","addr":"172.21.15.121:6825","nonce":74379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":74379},{"type":"v1","addr":"172.21.15.121:6827","nonce":74379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":74379},{"type":"v1","addr":"172.21.15.121:6831","nonce":74379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":74379},{"type":"v1","addr":"172.21.15.121:6829","nonce":74379}]},"public_addr":"172.21.15.121:6825/74379","cluster_addr":"172.21.15.121:6827/74379","heartbeat_back_addr":"172.21.15.121:6831/74379","heartbeat_front_addr":"172.21.15.121:6829/74379","state":["exists","up"]},{"osd":10,"uuid":"705906c4-1514-4754-ac40-45226a837180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":43109},{"type":"v1","addr":"172.21.15.123:6810","nonce":43109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":43109},{"type":"v1","addr":"172.21.15.123:6813","nonce":43109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":43109},{"type":"v1","addr":"172.21.15.123:6820","nonce":43109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":43109},{"type":"v1","addr":"172.21.15.123:6817","nonce":43109}]},"public_addr":"172.21.15.123:6810/43109","cluster_addr":"172.21.15.123:6813/43109","heartbeat_back_addr":"172.21.15.123:6820/43109","heartbeat_front_addr":"172.21.15.123:6817/43109","state":["exists","up"]},{"osd":11,"uuid":"b45f2645-72d1-4d72-89e9-2212f5aa074c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":74544},{"type":"v1","addr":"172.21.15.165:6809","nonce":74544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":74544},{"type":"v1","addr":"172.21.15.165:6811","nonce":74544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":74544},{"type":"v1","addr":"172.21.15.165:6815","nonce":74544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":74544},{"type":"v1","addr":"172.21.15.165:6813","nonce":74544}]},"public_addr":"172.21.15.165:6809/74544","cluster_addr":"172.21.15.165:6811/74544","heartbeat_back_addr":"172.21.15.165:6815/74544","heartbeat_front_addr":"172.21.15.165:6813/74544","state":["exists","up"]},{"osd":12,"uuid":"3ea410b9-a368-467b-a179-2e3aba562d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":43516},{"type":"v1","addr":"172.21.15.44:6801","nonce":43516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":43516},{"type":"v1","addr":"172.21.15.44:6803","nonce":43516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":43516},{"type":"v1","addr":"172.21.15.44:6807","nonce":43516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":43516},{"type":"v1","addr":"172.21.15.44:6805","nonce":43516}]},"public_addr":"172.21.15.44:6801/43516","cluster_addr":"172.21.15.44:6803/43516","heartbeat_back_addr":"172.21.15.44:6807/43516","heartbeat_front_addr":"172.21.15.44:6805/43516","state":["exists","up"]},{"osd":13,"uuid":"15a5154b-e7cb-4b57-8468-dcb048d93f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":74355},{"type":"v1","addr":"172.21.15.121:6809","nonce":74355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":74355},{"type":"v1","addr":"172.21.15.121:6811","nonce":74355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":74355},{"type":"v1","addr":"172.21.15.121:6815","nonce":74355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":74355},{"type":"v1","addr":"172.21.15.121:6813","nonce":74355}]},"public_addr":"172.21.15.121:6809/74355","cluster_addr":"172.21.15.121:6811/74355","heartbeat_back_addr":"172.21.15.121:6815/74355","heartbeat_front_addr":"172.21.15.121:6813/74355","state":["exists","up"]},{"osd":14,"uuid":"23b53da2-f848-4e76-9632-4d9effd5b430","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":43110},{"type":"v1","addr":"172.21.15.123:6825","nonce":43110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":43110},{"type":"v1","addr":"172.21.15.123:6827","nonce":43110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":43110},{"type":"v1","addr":"172.21.15.123:6831","nonce":43110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":43110},{"type":"v1","addr":"172.21.15.123:6829","nonce":43110}]},"public_addr":"172.21.15.123:6825/43110","cluster_addr":"172.21.15.123:6827/43110","heartbeat_back_addr":"172.21.15.123:6831/43110","heartbeat_front_addr":"172.21.15.123:6829/43110","state":["exists","up"]},{"osd":15,"uuid":"546efc0f-82a3-47d2-aeec-2d849eb1faa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":74562},{"type":"v1","addr":"172.21.15.165:6817","nonce":74562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":74562},{"type":"v1","addr":"172.21.15.165:6819","nonce":74562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":74562},{"type":"v1","addr":"172.21.15.165:6823","nonce":74562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":74562},{"type":"v1","addr":"172.21.15.165:6821","nonce":74562}]},"public_addr":"172.21.15.165:6817/74562","cluster_addr":"172.21.15.165:6819/74562","heartbeat_back_addr":"172.21.15.165:6823/74562","heartbeat_front_addr":"172.21.15.165:6821/74562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.124928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.093711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.008326+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.085397+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.117767+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:12.999445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.032367+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.058776+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.113013+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.110714+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.092533+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.083135+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.094847+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.059856+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.128995+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.061883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T01:47:18.038 DEBUG:teuthology.misc:16 of 16 OSDs are up 2024-04-20T01:47:18.038 INFO:tasks.ceph:Creating RBD pool 2024-04-20T01:47:18.038 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-20T01:47:19.078 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'rbd' created 2024-04-20T01:47:19.092 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-20T01:47:20.019 INFO:teuthology.orchestra.run.smithi044.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-20T01:47:20.033 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-20T01:47:20.034 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-20T01:47:20.034 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-20T01:47:20.034 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-20T01:47:20.035 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-20T01:47:20.472 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:47:20.486 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":5,"flags":0,"active_gid":4106,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":73591},{"type":"v1","addr":"172.21.15.165:6833","nonce":73591}]},"active_addr":"172.21.15.165:6833/73591","active_change":"2024-04-20T01:47:09.128573+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4105,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.165:0","nonce":1487355445}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.165:0","nonce":2652873865}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.165:0","nonce":2633397871}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.165:0","nonce":1777896646}]}]} 2024-04-20T01:47:20.489 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-20T01:47:20.490 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-20T01:47:20.490 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T01:47:20.855 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:47:20.856 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":26,"fsid":"9c2f64c1-aa35-4337-9f5c-081b9e3a2938","created":"2024-04-20T01:47:02.934241+0000","modified":"2024-04-20T01:47:20.017445+0000","last_up_change":"2024-04-20T01:47:15.986754+0000","last_in_change":"2024-04-20T01:47:07.894719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T01:47:15.195181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-20T01:47:18.448004+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3.9900000095367432,"score_stable":3.9900000095367432,"optimal_score":0.68999999761581421,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3903b88f-ff2b-4252-9480-92b701b1e59b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":43518},{"type":"v1","addr":"172.21.15.44:6825","nonce":43518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":43518},{"type":"v1","addr":"172.21.15.44:6827","nonce":43518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":43518},{"type":"v1","addr":"172.21.15.44:6831","nonce":43518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":43518},{"type":"v1","addr":"172.21.15.44:6829","nonce":43518}]},"public_addr":"172.21.15.44:6825/43518","cluster_addr":"172.21.15.44:6827/43518","heartbeat_back_addr":"172.21.15.44:6831/43518","heartbeat_front_addr":"172.21.15.44:6829/43518","state":["exists","up"]},{"osd":1,"uuid":"c5ff0175-b4f0-40a5-86c7-7a0a351bf070","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":74361},{"type":"v1","addr":"172.21.15.121:6817","nonce":74361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":74361},{"type":"v1","addr":"172.21.15.121:6819","nonce":74361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":74361},{"type":"v1","addr":"172.21.15.121:6823","nonce":74361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":74361},{"type":"v1","addr":"172.21.15.121:6821","nonce":74361}]},"public_addr":"172.21.15.121:6817/74361","cluster_addr":"172.21.15.121:6819/74361","heartbeat_back_addr":"172.21.15.121:6823/74361","heartbeat_front_addr":"172.21.15.121:6821/74361","state":["exists","up"]},{"osd":2,"uuid":"7af325c1-36b5-420c-a2a9-44c00a002977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6809","nonce":43108},{"type":"v1","addr":"172.21.15.123:6811","nonce":43108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":43108},{"type":"v1","addr":"172.21.15.123:6816","nonce":43108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":43108},{"type":"v1","addr":"172.21.15.123:6823","nonce":43108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":43108},{"type":"v1","addr":"172.21.15.123:6821","nonce":43108}]},"public_addr":"172.21.15.123:6811/43108","cluster_addr":"172.21.15.123:6816/43108","heartbeat_back_addr":"172.21.15.123:6823/43108","heartbeat_front_addr":"172.21.15.123:6821/43108","state":["exists","up"]},{"osd":3,"uuid":"99334f43-83ea-4c3f-b91d-ae72c8361b8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":74545},{"type":"v1","addr":"172.21.15.165:6825","nonce":74545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":74545},{"type":"v1","addr":"172.21.15.165:6827","nonce":74545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":74545},{"type":"v1","addr":"172.21.15.165:6831","nonce":74545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":74545},{"type":"v1","addr":"172.21.15.165:6829","nonce":74545}]},"public_addr":"172.21.15.165:6825/74545","cluster_addr":"172.21.15.165:6827/74545","heartbeat_back_addr":"172.21.15.165:6831/74545","heartbeat_front_addr":"172.21.15.165:6829/74545","state":["exists","up"]},{"osd":4,"uuid":"8764d799-af67-4e6c-889f-9fdd91201cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":43517},{"type":"v1","addr":"172.21.15.44:6811","nonce":43517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":43517},{"type":"v1","addr":"172.21.15.44:6815","nonce":43517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":43517},{"type":"v1","addr":"172.21.15.44:6823","nonce":43517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6817","nonce":43517},{"type":"v1","addr":"172.21.15.44:6819","nonce":43517}]},"public_addr":"172.21.15.44:6811/43517","cluster_addr":"172.21.15.44:6815/43517","heartbeat_back_addr":"172.21.15.44:6823/43517","heartbeat_front_addr":"172.21.15.44:6819/43517","state":["exists","up"]},{"osd":5,"uuid":"2556b2fa-4dcc-479c-a379-6f9fe96e9c77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":74354},{"type":"v1","addr":"172.21.15.121:6801","nonce":74354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":74354},{"type":"v1","addr":"172.21.15.121:6803","nonce":74354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":74354},{"type":"v1","addr":"172.21.15.121:6807","nonce":74354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":74354},{"type":"v1","addr":"172.21.15.121:6805","nonce":74354}]},"public_addr":"172.21.15.121:6801/74354","cluster_addr":"172.21.15.121:6803/74354","heartbeat_back_addr":"172.21.15.121:6807/74354","heartbeat_front_addr":"172.21.15.121:6805/74354","state":["exists","up"]},{"osd":6,"uuid":"baeaae3a-fff4-4ccc-a08a-ff32db4531ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":43107},{"type":"v1","addr":"172.21.15.123:6801","nonce":43107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":43107},{"type":"v1","addr":"172.21.15.123:6803","nonce":43107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":43107},{"type":"v1","addr":"172.21.15.123:6807","nonce":43107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":43107},{"type":"v1","addr":"172.21.15.123:6805","nonce":43107}]},"public_addr":"172.21.15.123:6801/43107","cluster_addr":"172.21.15.123:6803/43107","heartbeat_back_addr":"172.21.15.123:6807/43107","heartbeat_front_addr":"172.21.15.123:6805/43107","state":["exists","up"]},{"osd":7,"uuid":"0019f005-3e62-4d9d-b334-f1d347a484ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":74543},{"type":"v1","addr":"172.21.15.165:6801","nonce":74543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":74543},{"type":"v1","addr":"172.21.15.165:6803","nonce":74543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":74543},{"type":"v1","addr":"172.21.15.165:6807","nonce":74543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":74543},{"type":"v1","addr":"172.21.15.165:6805","nonce":74543}]},"public_addr":"172.21.15.165:6801/74543","cluster_addr":"172.21.15.165:6803/74543","heartbeat_back_addr":"172.21.15.165:6807/74543","heartbeat_front_addr":"172.21.15.165:6805/74543","state":["exists","up"]},{"osd":8,"uuid":"6f5200a3-8980-4241-a6fc-ef604fa612b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":43515},{"type":"v1","addr":"172.21.15.44:6810","nonce":43515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":43515},{"type":"v1","addr":"172.21.15.44:6814","nonce":43515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":43515},{"type":"v1","addr":"172.21.15.44:6822","nonce":43515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":43515},{"type":"v1","addr":"172.21.15.44:6818","nonce":43515}]},"public_addr":"172.21.15.44:6810/43515","cluster_addr":"172.21.15.44:6814/43515","heartbeat_back_addr":"172.21.15.44:6822/43515","heartbeat_front_addr":"172.21.15.44:6818/43515","state":["exists","up"]},{"osd":9,"uuid":"29cf9aaa-3a99-4605-92f4-64f1f6a1728d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":74379},{"type":"v1","addr":"172.21.15.121:6825","nonce":74379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":74379},{"type":"v1","addr":"172.21.15.121:6827","nonce":74379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":74379},{"type":"v1","addr":"172.21.15.121:6831","nonce":74379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":74379},{"type":"v1","addr":"172.21.15.121:6829","nonce":74379}]},"public_addr":"172.21.15.121:6825/74379","cluster_addr":"172.21.15.121:6827/74379","heartbeat_back_addr":"172.21.15.121:6831/74379","heartbeat_front_addr":"172.21.15.121:6829/74379","state":["exists","up"]},{"osd":10,"uuid":"705906c4-1514-4754-ac40-45226a837180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":43109},{"type":"v1","addr":"172.21.15.123:6810","nonce":43109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":43109},{"type":"v1","addr":"172.21.15.123:6813","nonce":43109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":43109},{"type":"v1","addr":"172.21.15.123:6820","nonce":43109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":43109},{"type":"v1","addr":"172.21.15.123:6817","nonce":43109}]},"public_addr":"172.21.15.123:6810/43109","cluster_addr":"172.21.15.123:6813/43109","heartbeat_back_addr":"172.21.15.123:6820/43109","heartbeat_front_addr":"172.21.15.123:6817/43109","state":["exists","up"]},{"osd":11,"uuid":"b45f2645-72d1-4d72-89e9-2212f5aa074c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":74544},{"type":"v1","addr":"172.21.15.165:6809","nonce":74544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":74544},{"type":"v1","addr":"172.21.15.165:6811","nonce":74544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":74544},{"type":"v1","addr":"172.21.15.165:6815","nonce":74544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":74544},{"type":"v1","addr":"172.21.15.165:6813","nonce":74544}]},"public_addr":"172.21.15.165:6809/74544","cluster_addr":"172.21.15.165:6811/74544","heartbeat_back_addr":"172.21.15.165:6815/74544","heartbeat_front_addr":"172.21.15.165:6813/74544","state":["exists","up"]},{"osd":12,"uuid":"3ea410b9-a368-467b-a179-2e3aba562d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":43516},{"type":"v1","addr":"172.21.15.44:6801","nonce":43516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":43516},{"type":"v1","addr":"172.21.15.44:6803","nonce":43516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":43516},{"type":"v1","addr":"172.21.15.44:6807","nonce":43516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":43516},{"type":"v1","addr":"172.21.15.44:6805","nonce":43516}]},"public_addr":"172.21.15.44:6801/43516","cluster_addr":"172.21.15.44:6803/43516","heartbeat_back_addr":"172.21.15.44:6807/43516","heartbeat_front_addr":"172.21.15.44:6805/43516","state":["exists","up"]},{"osd":13,"uuid":"15a5154b-e7cb-4b57-8468-dcb048d93f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":74355},{"type":"v1","addr":"172.21.15.121:6809","nonce":74355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":74355},{"type":"v1","addr":"172.21.15.121:6811","nonce":74355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":74355},{"type":"v1","addr":"172.21.15.121:6815","nonce":74355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":74355},{"type":"v1","addr":"172.21.15.121:6813","nonce":74355}]},"public_addr":"172.21.15.121:6809/74355","cluster_addr":"172.21.15.121:6811/74355","heartbeat_back_addr":"172.21.15.121:6815/74355","heartbeat_front_addr":"172.21.15.121:6813/74355","state":["exists","up"]},{"osd":14,"uuid":"23b53da2-f848-4e76-9632-4d9effd5b430","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":43110},{"type":"v1","addr":"172.21.15.123:6825","nonce":43110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":43110},{"type":"v1","addr":"172.21.15.123:6827","nonce":43110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":43110},{"type":"v1","addr":"172.21.15.123:6831","nonce":43110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":43110},{"type":"v1","addr":"172.21.15.123:6829","nonce":43110}]},"public_addr":"172.21.15.123:6825/43110","cluster_addr":"172.21.15.123:6827/43110","heartbeat_back_addr":"172.21.15.123:6831/43110","heartbeat_front_addr":"172.21.15.123:6829/43110","state":["exists","up"]},{"osd":15,"uuid":"546efc0f-82a3-47d2-aeec-2d849eb1faa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":74562},{"type":"v1","addr":"172.21.15.165:6817","nonce":74562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":74562},{"type":"v1","addr":"172.21.15.165:6819","nonce":74562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":74562},{"type":"v1","addr":"172.21.15.165:6823","nonce":74562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":74562},{"type":"v1","addr":"172.21.15.165:6821","nonce":74562}]},"public_addr":"172.21.15.165:6817/74562","cluster_addr":"172.21.15.165:6819/74562","heartbeat_back_addr":"172.21.15.165:6823/74562","heartbeat_front_addr":"172.21.15.165:6821/74562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.124928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.093711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.008326+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.085397+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.117767+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:12.999445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.032367+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.058776+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.113013+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.110714+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.092533+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.083135+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.094847+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.059856+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.128995+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.061883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T01:47:20.871 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-20T01:47:20.872 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T01:47:21.241 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:47:21.241 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"fsid":"9c2f64c1-aa35-4337-9f5c-081b9e3a2938","created":"2024-04-20T01:47:02.934241+0000","modified":"2024-04-20T01:47:21.024143+0000","last_up_change":"2024-04-20T01:47:15.986754+0000","last_in_change":"2024-04-20T01:47:07.894719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T01:47:15.195181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-20T01:47:18.448004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3.9900000095367432,"score_stable":3.9900000095367432,"optimal_score":0.68999999761581421,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3903b88f-ff2b-4252-9480-92b701b1e59b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":43518},{"type":"v1","addr":"172.21.15.44:6825","nonce":43518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":43518},{"type":"v1","addr":"172.21.15.44:6827","nonce":43518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":43518},{"type":"v1","addr":"172.21.15.44:6831","nonce":43518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":43518},{"type":"v1","addr":"172.21.15.44:6829","nonce":43518}]},"public_addr":"172.21.15.44:6825/43518","cluster_addr":"172.21.15.44:6827/43518","heartbeat_back_addr":"172.21.15.44:6831/43518","heartbeat_front_addr":"172.21.15.44:6829/43518","state":["exists","up"]},{"osd":1,"uuid":"c5ff0175-b4f0-40a5-86c7-7a0a351bf070","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":74361},{"type":"v1","addr":"172.21.15.121:6817","nonce":74361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":74361},{"type":"v1","addr":"172.21.15.121:6819","nonce":74361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":74361},{"type":"v1","addr":"172.21.15.121:6823","nonce":74361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":74361},{"type":"v1","addr":"172.21.15.121:6821","nonce":74361}]},"public_addr":"172.21.15.121:6817/74361","cluster_addr":"172.21.15.121:6819/74361","heartbeat_back_addr":"172.21.15.121:6823/74361","heartbeat_front_addr":"172.21.15.121:6821/74361","state":["exists","up"]},{"osd":2,"uuid":"7af325c1-36b5-420c-a2a9-44c00a002977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6809","nonce":43108},{"type":"v1","addr":"172.21.15.123:6811","nonce":43108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":43108},{"type":"v1","addr":"172.21.15.123:6816","nonce":43108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":43108},{"type":"v1","addr":"172.21.15.123:6823","nonce":43108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":43108},{"type":"v1","addr":"172.21.15.123:6821","nonce":43108}]},"public_addr":"172.21.15.123:6811/43108","cluster_addr":"172.21.15.123:6816/43108","heartbeat_back_addr":"172.21.15.123:6823/43108","heartbeat_front_addr":"172.21.15.123:6821/43108","state":["exists","up"]},{"osd":3,"uuid":"99334f43-83ea-4c3f-b91d-ae72c8361b8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":74545},{"type":"v1","addr":"172.21.15.165:6825","nonce":74545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":74545},{"type":"v1","addr":"172.21.15.165:6827","nonce":74545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":74545},{"type":"v1","addr":"172.21.15.165:6831","nonce":74545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":74545},{"type":"v1","addr":"172.21.15.165:6829","nonce":74545}]},"public_addr":"172.21.15.165:6825/74545","cluster_addr":"172.21.15.165:6827/74545","heartbeat_back_addr":"172.21.15.165:6831/74545","heartbeat_front_addr":"172.21.15.165:6829/74545","state":["exists","up"]},{"osd":4,"uuid":"8764d799-af67-4e6c-889f-9fdd91201cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":43517},{"type":"v1","addr":"172.21.15.44:6811","nonce":43517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":43517},{"type":"v1","addr":"172.21.15.44:6815","nonce":43517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":43517},{"type":"v1","addr":"172.21.15.44:6823","nonce":43517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6817","nonce":43517},{"type":"v1","addr":"172.21.15.44:6819","nonce":43517}]},"public_addr":"172.21.15.44:6811/43517","cluster_addr":"172.21.15.44:6815/43517","heartbeat_back_addr":"172.21.15.44:6823/43517","heartbeat_front_addr":"172.21.15.44:6819/43517","state":["exists","up"]},{"osd":5,"uuid":"2556b2fa-4dcc-479c-a379-6f9fe96e9c77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":74354},{"type":"v1","addr":"172.21.15.121:6801","nonce":74354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":74354},{"type":"v1","addr":"172.21.15.121:6803","nonce":74354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":74354},{"type":"v1","addr":"172.21.15.121:6807","nonce":74354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":74354},{"type":"v1","addr":"172.21.15.121:6805","nonce":74354}]},"public_addr":"172.21.15.121:6801/74354","cluster_addr":"172.21.15.121:6803/74354","heartbeat_back_addr":"172.21.15.121:6807/74354","heartbeat_front_addr":"172.21.15.121:6805/74354","state":["exists","up"]},{"osd":6,"uuid":"baeaae3a-fff4-4ccc-a08a-ff32db4531ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":43107},{"type":"v1","addr":"172.21.15.123:6801","nonce":43107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":43107},{"type":"v1","addr":"172.21.15.123:6803","nonce":43107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":43107},{"type":"v1","addr":"172.21.15.123:6807","nonce":43107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":43107},{"type":"v1","addr":"172.21.15.123:6805","nonce":43107}]},"public_addr":"172.21.15.123:6801/43107","cluster_addr":"172.21.15.123:6803/43107","heartbeat_back_addr":"172.21.15.123:6807/43107","heartbeat_front_addr":"172.21.15.123:6805/43107","state":["exists","up"]},{"osd":7,"uuid":"0019f005-3e62-4d9d-b334-f1d347a484ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":74543},{"type":"v1","addr":"172.21.15.165:6801","nonce":74543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":74543},{"type":"v1","addr":"172.21.15.165:6803","nonce":74543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":74543},{"type":"v1","addr":"172.21.15.165:6807","nonce":74543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":74543},{"type":"v1","addr":"172.21.15.165:6805","nonce":74543}]},"public_addr":"172.21.15.165:6801/74543","cluster_addr":"172.21.15.165:6803/74543","heartbeat_back_addr":"172.21.15.165:6807/74543","heartbeat_front_addr":"172.21.15.165:6805/74543","state":["exists","up"]},{"osd":8,"uuid":"6f5200a3-8980-4241-a6fc-ef604fa612b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":43515},{"type":"v1","addr":"172.21.15.44:6810","nonce":43515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":43515},{"type":"v1","addr":"172.21.15.44:6814","nonce":43515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":43515},{"type":"v1","addr":"172.21.15.44:6822","nonce":43515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":43515},{"type":"v1","addr":"172.21.15.44:6818","nonce":43515}]},"public_addr":"172.21.15.44:6810/43515","cluster_addr":"172.21.15.44:6814/43515","heartbeat_back_addr":"172.21.15.44:6822/43515","heartbeat_front_addr":"172.21.15.44:6818/43515","state":["exists","up"]},{"osd":9,"uuid":"29cf9aaa-3a99-4605-92f4-64f1f6a1728d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":74379},{"type":"v1","addr":"172.21.15.121:6825","nonce":74379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":74379},{"type":"v1","addr":"172.21.15.121:6827","nonce":74379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":74379},{"type":"v1","addr":"172.21.15.121:6831","nonce":74379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":74379},{"type":"v1","addr":"172.21.15.121:6829","nonce":74379}]},"public_addr":"172.21.15.121:6825/74379","cluster_addr":"172.21.15.121:6827/74379","heartbeat_back_addr":"172.21.15.121:6831/74379","heartbeat_front_addr":"172.21.15.121:6829/74379","state":["exists","up"]},{"osd":10,"uuid":"705906c4-1514-4754-ac40-45226a837180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":43109},{"type":"v1","addr":"172.21.15.123:6810","nonce":43109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":43109},{"type":"v1","addr":"172.21.15.123:6813","nonce":43109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":43109},{"type":"v1","addr":"172.21.15.123:6820","nonce":43109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":43109},{"type":"v1","addr":"172.21.15.123:6817","nonce":43109}]},"public_addr":"172.21.15.123:6810/43109","cluster_addr":"172.21.15.123:6813/43109","heartbeat_back_addr":"172.21.15.123:6820/43109","heartbeat_front_addr":"172.21.15.123:6817/43109","state":["exists","up"]},{"osd":11,"uuid":"b45f2645-72d1-4d72-89e9-2212f5aa074c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":74544},{"type":"v1","addr":"172.21.15.165:6809","nonce":74544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":74544},{"type":"v1","addr":"172.21.15.165:6811","nonce":74544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":74544},{"type":"v1","addr":"172.21.15.165:6815","nonce":74544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":74544},{"type":"v1","addr":"172.21.15.165:6813","nonce":74544}]},"public_addr":"172.21.15.165:6809/74544","cluster_addr":"172.21.15.165:6811/74544","heartbeat_back_addr":"172.21.15.165:6815/74544","heartbeat_front_addr":"172.21.15.165:6813/74544","state":["exists","up"]},{"osd":12,"uuid":"3ea410b9-a368-467b-a179-2e3aba562d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":43516},{"type":"v1","addr":"172.21.15.44:6801","nonce":43516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":43516},{"type":"v1","addr":"172.21.15.44:6803","nonce":43516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":43516},{"type":"v1","addr":"172.21.15.44:6807","nonce":43516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":43516},{"type":"v1","addr":"172.21.15.44:6805","nonce":43516}]},"public_addr":"172.21.15.44:6801/43516","cluster_addr":"172.21.15.44:6803/43516","heartbeat_back_addr":"172.21.15.44:6807/43516","heartbeat_front_addr":"172.21.15.44:6805/43516","state":["exists","up"]},{"osd":13,"uuid":"15a5154b-e7cb-4b57-8468-dcb048d93f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":74355},{"type":"v1","addr":"172.21.15.121:6809","nonce":74355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":74355},{"type":"v1","addr":"172.21.15.121:6811","nonce":74355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":74355},{"type":"v1","addr":"172.21.15.121:6815","nonce":74355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":74355},{"type":"v1","addr":"172.21.15.121:6813","nonce":74355}]},"public_addr":"172.21.15.121:6809/74355","cluster_addr":"172.21.15.121:6811/74355","heartbeat_back_addr":"172.21.15.121:6815/74355","heartbeat_front_addr":"172.21.15.121:6813/74355","state":["exists","up"]},{"osd":14,"uuid":"23b53da2-f848-4e76-9632-4d9effd5b430","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":43110},{"type":"v1","addr":"172.21.15.123:6825","nonce":43110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":43110},{"type":"v1","addr":"172.21.15.123:6827","nonce":43110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":43110},{"type":"v1","addr":"172.21.15.123:6831","nonce":43110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":43110},{"type":"v1","addr":"172.21.15.123:6829","nonce":43110}]},"public_addr":"172.21.15.123:6825/43110","cluster_addr":"172.21.15.123:6827/43110","heartbeat_back_addr":"172.21.15.123:6831/43110","heartbeat_front_addr":"172.21.15.123:6829/43110","state":["exists","up"]},{"osd":15,"uuid":"546efc0f-82a3-47d2-aeec-2d849eb1faa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":74562},{"type":"v1","addr":"172.21.15.165:6817","nonce":74562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":74562},{"type":"v1","addr":"172.21.15.165:6819","nonce":74562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":74562},{"type":"v1","addr":"172.21.15.165:6823","nonce":74562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":74562},{"type":"v1","addr":"172.21.15.165:6821","nonce":74562}]},"public_addr":"172.21.15.165:6817/74562","cluster_addr":"172.21.15.165:6819/74562","heartbeat_back_addr":"172.21.15.165:6823/74562","heartbeat_front_addr":"172.21.15.165:6821/74562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.124928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.093711+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.008326+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.085397+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.117767+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:12.999445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.032367+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.058776+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.113013+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.110714+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.092533+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.083135+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.094847+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.059856+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.128995+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.061883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T01:47:21.257 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-20T01:47:21.257 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-20T01:47:21.258 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-20T01:47:21.258 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-20T01:47:21.258 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-20T01:47:21.259 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-20T01:47:21.259 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-20T01:47:21.259 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-20T01:47:21.260 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-04-20T01:47:21.260 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-04-20T01:47:21.261 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-04-20T01:47:21.261 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-04-20T01:47:21.261 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2024-04-20T01:47:21.262 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2024-04-20T01:47:21.262 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2024-04-20T01:47:21.262 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2024-04-20T01:47:21.828 INFO:teuthology.orchestra.run.smithi044.stdout:90194313221 2024-04-20T01:47:21.828 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-04-20T01:47:21.850 INFO:teuthology.orchestra.run.smithi044.stdout:90194313221 2024-04-20T01:47:21.851 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-20T01:47:21.855 INFO:teuthology.orchestra.run.smithi044.stdout:90194313221 2024-04-20T01:47:21.856 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-20T01:47:21.865 INFO:teuthology.orchestra.run.smithi044.stdout:90194313220 2024-04-20T01:47:21.865 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-20T01:47:21.871 INFO:teuthology.orchestra.run.smithi044.stdout:90194313220 2024-04-20T01:47:21.871 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-04-20T01:47:21.883 INFO:teuthology.orchestra.run.smithi044.stdout:90194313220 2024-04-20T01:47:21.884 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-20T01:47:21.892 INFO:teuthology.orchestra.run.smithi044.stdout:90194313221 2024-04-20T01:47:21.892 INFO:teuthology.orchestra.run.smithi044.stdout:90194313219 2024-04-20T01:47:21.893 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-20T01:47:21.893 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-04-20T01:47:21.905 INFO:teuthology.orchestra.run.smithi044.stdout:90194313221 2024-04-20T01:47:21.906 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-20T01:47:21.926 INFO:teuthology.orchestra.run.smithi044.stdout:90194313219 2024-04-20T01:47:21.927 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-20T01:47:21.930 INFO:teuthology.orchestra.run.smithi044.stdout:90194313221 2024-04-20T01:47:21.930 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-04-20T01:47:21.952 INFO:teuthology.orchestra.run.smithi044.stdout:90194313219 2024-04-20T01:47:21.954 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-20T01:47:21.970 INFO:teuthology.orchestra.run.smithi044.stdout:90194313220 2024-04-20T01:47:21.970 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-04-20T01:47:21.975 INFO:teuthology.orchestra.run.smithi044.stdout:90194313220 2024-04-20T01:47:21.976 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-04-20T01:47:21.990 INFO:teuthology.orchestra.run.smithi044.stdout:94489280515 2024-04-20T01:47:21.990 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-04-20T01:47:22.063 INFO:teuthology.orchestra.run.smithi044.stdout:90194313221 2024-04-20T01:47:22.063 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-04-20T01:47:22.806 INFO:teuthology.orchestra.run.smithi044.stdout:90194313220 2024-04-20T01:47:22.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313220 for osd.0 2024-04-20T01:47:22.973 INFO:teuthology.orchestra.run.smithi044.stdout:90194313219 2024-04-20T01:47:23.027 INFO:teuthology.orchestra.run.smithi044.stdout:90194313219 2024-04-20T01:47:23.070 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.1 2024-04-20T01:47:23.093 INFO:teuthology.orchestra.run.smithi044.stdout:90194313220 2024-04-20T01:47:23.143 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.5 2024-04-20T01:47:23.196 INFO:teuthology.orchestra.run.smithi044.stdout:90194313218 2024-04-20T01:47:23.215 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313220 for osd.9 2024-04-20T01:47:23.235 INFO:teuthology.orchestra.run.smithi044.stdout:90194313220 2024-04-20T01:47:23.241 INFO:teuthology.orchestra.run.smithi044.stdout:90194313220 2024-04-20T01:47:23.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313218 for osd.6 2024-04-20T01:47:23.288 INFO:teuthology.orchestra.run.smithi044.stdout:90194313218 2024-04-20T01:47:23.324 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313220 for osd.3 2024-04-20T01:47:23.325 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313220 for osd.7 2024-04-20T01:47:23.341 INFO:teuthology.orchestra.run.smithi044.stdout:90194313218 2024-04-20T01:47:23.352 INFO:teuthology.orchestra.run.smithi044.stdout:90194313219 2024-04-20T01:47:23.360 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313218 for osd.11 2024-04-20T01:47:23.360 INFO:teuthology.orchestra.run.smithi044.stdout:90194313220 2024-04-20T01:47:23.373 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.12 2024-04-20T01:47:23.384 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313220 for osd.4 2024-04-20T01:47:23.390 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313218 for osd.2 2024-04-20T01:47:23.395 INFO:teuthology.orchestra.run.smithi044.stdout:90194313219 2024-04-20T01:47:23.412 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.8 2024-04-20T01:47:23.428 INFO:teuthology.orchestra.run.smithi044.stdout:90194313219 2024-04-20T01:47:23.451 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.15 2024-04-20T01:47:23.495 INFO:teuthology.orchestra.run.smithi044.stdout:94489280514 2024-04-20T01:47:23.505 INFO:teuthology.orchestra.run.smithi044.stdout:90194313220 2024-04-20T01:47:23.513 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280514 for osd.13 2024-04-20T01:47:23.514 INFO:teuthology.orchestra.run.smithi044.stdout:90194313220 2024-04-20T01:47:23.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313220 for osd.10 2024-04-20T01:47:23.532 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313220 for osd.14 2024-04-20T01:47:23.838 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-20T01:47:24.071 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-20T01:47:24.144 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-20T01:47:24.215 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-04-20T01:47:24.219 INFO:teuthology.orchestra.run.smithi044.stdout:90194313221 2024-04-20T01:47:24.236 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.0 2024-04-20T01:47:24.236 DEBUG:teuthology.parallel:result is None 2024-04-20T01:47:24.266 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-20T01:47:24.325 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-20T01:47:24.326 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-20T01:47:24.361 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-04-20T01:47:24.374 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-04-20T01:47:24.385 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-20T01:47:24.391 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-20T01:47:24.412 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-04-20T01:47:24.452 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-04-20T01:47:24.513 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-04-20T01:47:24.525 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-04-20T01:47:24.534 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-04-20T01:47:24.593 INFO:teuthology.orchestra.run.smithi044.stdout:90194313220 2024-04-20T01:47:24.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.1 2024-04-20T01:47:24.617 DEBUG:teuthology.parallel:result is None 2024-04-20T01:47:24.780 INFO:teuthology.orchestra.run.smithi044.stdout:90194313220 2024-04-20T01:47:24.884 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.5 2024-04-20T01:47:24.884 DEBUG:teuthology.parallel:result is None 2024-04-20T01:47:24.949 INFO:teuthology.orchestra.run.smithi044.stdout:90194313221 2024-04-20T01:47:25.019 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.9 2024-04-20T01:47:25.019 DEBUG:teuthology.parallel:result is None 2024-04-20T01:47:25.109 INFO:teuthology.orchestra.run.smithi044.stdout:90194313219 2024-04-20T01:47:25.170 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313219 for osd.6 2024-04-20T01:47:25.170 DEBUG:teuthology.parallel:result is None 2024-04-20T01:47:25.222 INFO:teuthology.orchestra.run.smithi044.stdout:90194313221 2024-04-20T01:47:25.358 INFO:teuthology.orchestra.run.smithi044.stdout:90194313219 2024-04-20T01:47:25.371 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.7 2024-04-20T01:47:25.372 DEBUG:teuthology.parallel:result is None 2024-04-20T01:47:25.409 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313219 for osd.11 2024-04-20T01:47:25.409 DEBUG:teuthology.parallel:result is None 2024-04-20T01:47:25.411 INFO:teuthology.orchestra.run.smithi044.stdout:90194313221 2024-04-20T01:47:25.453 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.3 2024-04-20T01:47:25.453 DEBUG:teuthology.parallel:result is None 2024-04-20T01:47:25.458 INFO:teuthology.orchestra.run.smithi044.stdout:90194313221 2024-04-20T01:47:25.462 INFO:teuthology.orchestra.run.smithi044.stdout:90194313220 2024-04-20T01:47:25.485 INFO:teuthology.orchestra.run.smithi044.stdout:90194313220 2024-04-20T01:47:25.489 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.15 2024-04-20T01:47:25.489 DEBUG:teuthology.parallel:result is None 2024-04-20T01:47:25.494 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.4 2024-04-20T01:47:25.495 DEBUG:teuthology.parallel:result is None 2024-04-20T01:47:25.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.12 2024-04-20T01:47:25.522 DEBUG:teuthology.parallel:result is None 2024-04-20T01:47:25.531 INFO:teuthology.orchestra.run.smithi044.stdout:90194313220 2024-04-20T01:47:25.541 INFO:teuthology.orchestra.run.smithi044.stdout:90194313219 2024-04-20T01:47:25.554 INFO:teuthology.orchestra.run.smithi044.stdout:94489280516 2024-04-20T01:47:25.557 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.8 2024-04-20T01:47:25.558 DEBUG:teuthology.parallel:result is None 2024-04-20T01:47:25.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313219 for osd.2 2024-04-20T01:47:25.562 DEBUG:teuthology.parallel:result is None 2024-04-20T01:47:25.571 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280516 for osd.13 2024-04-20T01:47:25.572 DEBUG:teuthology.parallel:result is None 2024-04-20T01:47:25.613 INFO:teuthology.orchestra.run.smithi044.stdout:90194313221 2024-04-20T01:47:25.630 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.10 2024-04-20T01:47:25.630 DEBUG:teuthology.parallel:result is None 2024-04-20T01:47:25.712 INFO:teuthology.orchestra.run.smithi044.stdout:90194313221 2024-04-20T01:47:25.729 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.14 2024-04-20T01:47:25.729 DEBUG:teuthology.parallel:result is None 2024-04-20T01:47:25.729 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-20T01:47:25.729 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T01:47:26.094 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:47:26.096 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T01:47:26.109 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-04-20T01:47:25.137152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":438184,"kb_used_data":5308,"kb_used_omap":24,"kb_used_meta":432103,"kb_avail":1509511256,"statfs":{"total":1546188226560,"available":1545739526144,"internally_reserved":0,"allocated":5435392,"data_stored":2768472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25507,"internal_metadata":442473565},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000536"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.981697+0000","last_change":"2024-04-20T01:47:20.036634+0000","last_active":"2024-04-20T01:47:20.981697+0000","last_peered":"2024-04-20T01:47:20.981697+0000","last_clean":"2024-04-20T01:47:20.981697+0000","last_became_active":"2024-04-20T01:47:20.036328+0000","last_became_peered":"2024-04-20T01:47:20.036328+0000","last_unstale":"2024-04-20T01:47:20.981697+0000","last_undegraded":"2024-04-20T01:47:20.981697+0000","last_fullsized":"2024-04-20T01:47:20.981697+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.385250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,13],"acting":[6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.985494+0000","last_change":"2024-04-20T01:47:20.110600+0000","last_active":"2024-04-20T01:47:20.985494+0000","last_peered":"2024-04-20T01:47:20.985494+0000","last_clean":"2024-04-20T01:47:20.985494+0000","last_became_active":"2024-04-20T01:47:20.110260+0000","last_became_peered":"2024-04-20T01:47:20.110260+0000","last_unstale":"2024-04-20T01:47:20.985494+0000","last_undegraded":"2024-04-20T01:47:20.985494+0000","last_fullsized":"2024-04-20T01:47:20.985494+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:47.718290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.983286+0000","last_change":"2024-04-20T01:47:20.038409+0000","last_active":"2024-04-20T01:47:20.983286+0000","last_peered":"2024-04-20T01:47:20.983286+0000","last_clean":"2024-04-20T01:47:20.983286+0000","last_became_active":"2024-04-20T01:47:20.038107+0000","last_became_peered":"2024-04-20T01:47:20.038107+0000","last_unstale":"2024-04-20T01:47:20.983286+0000","last_undegraded":"2024-04-20T01:47:20.983286+0000","last_fullsized":"2024-04-20T01:47:20.983286+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:44.125577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.983194+0000","last_change":"2024-04-20T01:47:20.069279+0000","last_active":"2024-04-20T01:47:20.983194+0000","last_peered":"2024-04-20T01:47:20.983194+0000","last_clean":"2024-04-20T01:47:20.983194+0000","last_became_active":"2024-04-20T01:47:20.069005+0000","last_became_peered":"2024-04-20T01:47:20.069005+0000","last_unstale":"2024-04-20T01:47:20.983194+0000","last_undegraded":"2024-04-20T01:47:20.983194+0000","last_fullsized":"2024-04-20T01:47:20.983194+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:22.648743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"25'157","reported_seq":207,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T01:47:19.177685+0000","last_change":"2024-04-20T01:47:17.303029+0000","last_active":"2024-04-20T01:47:19.177685+0000","last_peered":"2024-04-20T01:47:19.177685+0000","last_clean":"2024-04-20T01:47:19.177685+0000","last_became_active":"2024-04-20T01:47:17.302105+0000","last_became_peered":"2024-04-20T01:47:17.302105+0000","last_unstale":"2024-04-20T01:47:19.177685+0000","last_undegraded":"2024-04-20T01:47:19.177685+0000","last_fullsized":"2024-04-20T01:47:19.177685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_clean_scrub_stamp":"2024-04-20T01:47:15.986754+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:30.481506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,15],"acting":[13,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.043730+0000","last_change":"2024-04-20T01:47:20.039014+0000","last_active":"2024-04-20T01:47:20.043730+0000","last_peered":"2024-04-20T01:47:20.043730+0000","last_clean":"2024-04-20T01:47:20.043730+0000","last_became_active":"2024-04-20T01:47:20.038764+0000","last_became_peered":"2024-04-20T01:47:20.038764+0000","last_unstale":"2024-04-20T01:47:20.043730+0000","last_undegraded":"2024-04-20T01:47:20.043730+0000","last_fullsized":"2024-04-20T01:47:20.043730+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:26.112165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.043708+0000","last_change":"2024-04-20T01:47:20.039221+0000","last_active":"2024-04-20T01:47:20.043708+0000","last_peered":"2024-04-20T01:47:20.043708+0000","last_clean":"2024-04-20T01:47:20.043708+0000","last_became_active":"2024-04-20T01:47:20.038972+0000","last_became_peered":"2024-04-20T01:47:20.038972+0000","last_unstale":"2024-04-20T01:47:20.043708+0000","last_undegraded":"2024-04-20T01:47:20.043708+0000","last_fullsized":"2024-04-20T01:47:20.043708+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:24.167832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.983211+0000","last_change":"2024-04-20T01:47:20.075613+0000","last_active":"2024-04-20T01:47:20.983211+0000","last_peered":"2024-04-20T01:47:20.983211+0000","last_clean":"2024-04-20T01:47:20.983211+0000","last_became_active":"2024-04-20T01:47:20.075314+0000","last_became_peered":"2024-04-20T01:47:20.075314+0000","last_unstale":"2024-04-20T01:47:20.983211+0000","last_undegraded":"2024-04-20T01:47:20.983211+0000","last_fullsized":"2024-04-20T01:47:20.983211+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:35.956173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,12],"acting":[2,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.985668+0000","last_change":"2024-04-20T01:47:20.071988+0000","last_active":"2024-04-20T01:47:20.985668+0000","last_peered":"2024-04-20T01:47:20.985668+0000","last_clean":"2024-04-20T01:47:20.985668+0000","last_became_active":"2024-04-20T01:47:20.071615+0000","last_became_peered":"2024-04-20T01:47:20.071615+0000","last_unstale":"2024-04-20T01:47:20.985668+0000","last_undegraded":"2024-04-20T01:47:20.985668+0000","last_fullsized":"2024-04-20T01:47:20.985668+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:40.106777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280516,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28252,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343588,"statfs":{"total":96636764160,"available":96607834112,"internally_reserved":0,"allocated":802816,"data_stored":634104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]}]},{"osd":15,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":643566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":14,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]}]},{"osd":12,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]}]},{"osd":11,"up_from":21,"seq":90194313219,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]}]},{"osd":10,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[1,2,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]}]},{"osd":9,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,4,6,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7989999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]}]},{"osd":8,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,6,7,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9630000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.784}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9239999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3769999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]}]},{"osd":7,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]}]},{"osd":6,"up_from":21,"seq":90194313219,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[5,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]}]},{"osd":1,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":0,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":104590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]}]},{"osd":2,"up_from":21,"seq":90194313219,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,4,5,6,7,8,9,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.9020000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.226}]}]},{"osd":3,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[2,4,5,6,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]}]},{"osd":4,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,5,6,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.258}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]}]},{"osd":5,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,4,6,7,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T01:47:26.111 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T01:47:26.478 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:47:26.481 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T01:47:26.494 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-04-20T01:47:25.137152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":438184,"kb_used_data":5308,"kb_used_omap":24,"kb_used_meta":432103,"kb_avail":1509511256,"statfs":{"total":1546188226560,"available":1545739526144,"internally_reserved":0,"allocated":5435392,"data_stored":2768472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25507,"internal_metadata":442473565},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000536"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.981697+0000","last_change":"2024-04-20T01:47:20.036634+0000","last_active":"2024-04-20T01:47:20.981697+0000","last_peered":"2024-04-20T01:47:20.981697+0000","last_clean":"2024-04-20T01:47:20.981697+0000","last_became_active":"2024-04-20T01:47:20.036328+0000","last_became_peered":"2024-04-20T01:47:20.036328+0000","last_unstale":"2024-04-20T01:47:20.981697+0000","last_undegraded":"2024-04-20T01:47:20.981697+0000","last_fullsized":"2024-04-20T01:47:20.981697+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.385250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,13],"acting":[6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.985494+0000","last_change":"2024-04-20T01:47:20.110600+0000","last_active":"2024-04-20T01:47:20.985494+0000","last_peered":"2024-04-20T01:47:20.985494+0000","last_clean":"2024-04-20T01:47:20.985494+0000","last_became_active":"2024-04-20T01:47:20.110260+0000","last_became_peered":"2024-04-20T01:47:20.110260+0000","last_unstale":"2024-04-20T01:47:20.985494+0000","last_undegraded":"2024-04-20T01:47:20.985494+0000","last_fullsized":"2024-04-20T01:47:20.985494+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:47.718290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.983286+0000","last_change":"2024-04-20T01:47:20.038409+0000","last_active":"2024-04-20T01:47:20.983286+0000","last_peered":"2024-04-20T01:47:20.983286+0000","last_clean":"2024-04-20T01:47:20.983286+0000","last_became_active":"2024-04-20T01:47:20.038107+0000","last_became_peered":"2024-04-20T01:47:20.038107+0000","last_unstale":"2024-04-20T01:47:20.983286+0000","last_undegraded":"2024-04-20T01:47:20.983286+0000","last_fullsized":"2024-04-20T01:47:20.983286+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:44.125577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.983194+0000","last_change":"2024-04-20T01:47:20.069279+0000","last_active":"2024-04-20T01:47:20.983194+0000","last_peered":"2024-04-20T01:47:20.983194+0000","last_clean":"2024-04-20T01:47:20.983194+0000","last_became_active":"2024-04-20T01:47:20.069005+0000","last_became_peered":"2024-04-20T01:47:20.069005+0000","last_unstale":"2024-04-20T01:47:20.983194+0000","last_undegraded":"2024-04-20T01:47:20.983194+0000","last_fullsized":"2024-04-20T01:47:20.983194+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:22.648743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"25'157","reported_seq":207,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T01:47:19.177685+0000","last_change":"2024-04-20T01:47:17.303029+0000","last_active":"2024-04-20T01:47:19.177685+0000","last_peered":"2024-04-20T01:47:19.177685+0000","last_clean":"2024-04-20T01:47:19.177685+0000","last_became_active":"2024-04-20T01:47:17.302105+0000","last_became_peered":"2024-04-20T01:47:17.302105+0000","last_unstale":"2024-04-20T01:47:19.177685+0000","last_undegraded":"2024-04-20T01:47:19.177685+0000","last_fullsized":"2024-04-20T01:47:19.177685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_clean_scrub_stamp":"2024-04-20T01:47:15.986754+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:30.481506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,15],"acting":[13,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.043730+0000","last_change":"2024-04-20T01:47:20.039014+0000","last_active":"2024-04-20T01:47:20.043730+0000","last_peered":"2024-04-20T01:47:20.043730+0000","last_clean":"2024-04-20T01:47:20.043730+0000","last_became_active":"2024-04-20T01:47:20.038764+0000","last_became_peered":"2024-04-20T01:47:20.038764+0000","last_unstale":"2024-04-20T01:47:20.043730+0000","last_undegraded":"2024-04-20T01:47:20.043730+0000","last_fullsized":"2024-04-20T01:47:20.043730+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:26.112165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.043708+0000","last_change":"2024-04-20T01:47:20.039221+0000","last_active":"2024-04-20T01:47:20.043708+0000","last_peered":"2024-04-20T01:47:20.043708+0000","last_clean":"2024-04-20T01:47:20.043708+0000","last_became_active":"2024-04-20T01:47:20.038972+0000","last_became_peered":"2024-04-20T01:47:20.038972+0000","last_unstale":"2024-04-20T01:47:20.043708+0000","last_undegraded":"2024-04-20T01:47:20.043708+0000","last_fullsized":"2024-04-20T01:47:20.043708+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:24.167832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.983211+0000","last_change":"2024-04-20T01:47:20.075613+0000","last_active":"2024-04-20T01:47:20.983211+0000","last_peered":"2024-04-20T01:47:20.983211+0000","last_clean":"2024-04-20T01:47:20.983211+0000","last_became_active":"2024-04-20T01:47:20.075314+0000","last_became_peered":"2024-04-20T01:47:20.075314+0000","last_unstale":"2024-04-20T01:47:20.983211+0000","last_undegraded":"2024-04-20T01:47:20.983211+0000","last_fullsized":"2024-04-20T01:47:20.983211+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:35.956173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,12],"acting":[2,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.985668+0000","last_change":"2024-04-20T01:47:20.071988+0000","last_active":"2024-04-20T01:47:20.985668+0000","last_peered":"2024-04-20T01:47:20.985668+0000","last_clean":"2024-04-20T01:47:20.985668+0000","last_became_active":"2024-04-20T01:47:20.071615+0000","last_became_peered":"2024-04-20T01:47:20.071615+0000","last_unstale":"2024-04-20T01:47:20.985668+0000","last_undegraded":"2024-04-20T01:47:20.985668+0000","last_fullsized":"2024-04-20T01:47:20.985668+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:40.106777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280516,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28252,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343588,"statfs":{"total":96636764160,"available":96607834112,"internally_reserved":0,"allocated":802816,"data_stored":634104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]}]},{"osd":15,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":643566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":14,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]}]},{"osd":12,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]}]},{"osd":11,"up_from":21,"seq":90194313219,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]}]},{"osd":10,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[1,2,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]}]},{"osd":9,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,4,6,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7989999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]}]},{"osd":8,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,6,7,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9630000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.784}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9239999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3769999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]}]},{"osd":7,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]}]},{"osd":6,"up_from":21,"seq":90194313219,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[5,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]}]},{"osd":1,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":0,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":104590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.361}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]}]},{"osd":2,"up_from":21,"seq":90194313219,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,4,5,6,7,8,9,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.9020000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.226}]}]},{"osd":3,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[2,4,5,6,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]}]},{"osd":4,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,5,6,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.258}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]}]},{"osd":5,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":105176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,4,6,7,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T01:47:26.496 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-20T01:47:26.496 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-20T01:47:26.496 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-20T01:47:26.496 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T01:47:26.904 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:47:26.905 INFO:teuthology.orchestra.run.smithi044.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-20T01:47:26.920 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-20T01:47:26.920 INFO:teuthology.run_tasks:Running task thrashosds... 2024-04-20T01:47:26.932 INFO:tasks.thrashosds:config is {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-04-20T01:47:26.932 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-yuri7-testing-2024-04-18-1351-reef'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri7-testing-2024-04-18-1351-reef', 'sha1': '15eecde2bab9803889d929c7e423d07970d8cbed'}} 2024-04-20T01:47:26.932 INFO:tasks.thrashosds:config is {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-04-20T01:47:26.932 INFO:tasks.thrashosds:Beginning thrashosds... 2024-04-20T01:47:26.932 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-20T01:47:26.932 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T01:47:27.298 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:47:27.300 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T01:47:27.313 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-20T01:47:27.137693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":437872,"kb_used_data":5516,"kb_used_omap":24,"kb_used_meta":432103,"kb_avail":1509511568,"statfs":{"total":1546188226560,"available":1545739845632,"internally_reserved":0,"allocated":5648384,"data_stored":2891478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25507,"internal_metadata":442473565},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001078"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.981697+0000","last_change":"2024-04-20T01:47:20.036634+0000","last_active":"2024-04-20T01:47:20.981697+0000","last_peered":"2024-04-20T01:47:20.981697+0000","last_clean":"2024-04-20T01:47:20.981697+0000","last_became_active":"2024-04-20T01:47:20.036328+0000","last_became_peered":"2024-04-20T01:47:20.036328+0000","last_unstale":"2024-04-20T01:47:20.981697+0000","last_undegraded":"2024-04-20T01:47:20.981697+0000","last_fullsized":"2024-04-20T01:47:20.981697+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.385250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,13],"acting":[6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.985494+0000","last_change":"2024-04-20T01:47:20.110600+0000","last_active":"2024-04-20T01:47:20.985494+0000","last_peered":"2024-04-20T01:47:20.985494+0000","last_clean":"2024-04-20T01:47:20.985494+0000","last_became_active":"2024-04-20T01:47:20.110260+0000","last_became_peered":"2024-04-20T01:47:20.110260+0000","last_unstale":"2024-04-20T01:47:20.985494+0000","last_undegraded":"2024-04-20T01:47:20.985494+0000","last_fullsized":"2024-04-20T01:47:20.985494+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:47.718290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.983286+0000","last_change":"2024-04-20T01:47:20.038409+0000","last_active":"2024-04-20T01:47:20.983286+0000","last_peered":"2024-04-20T01:47:20.983286+0000","last_clean":"2024-04-20T01:47:20.983286+0000","last_became_active":"2024-04-20T01:47:20.038107+0000","last_became_peered":"2024-04-20T01:47:20.038107+0000","last_unstale":"2024-04-20T01:47:20.983286+0000","last_undegraded":"2024-04-20T01:47:20.983286+0000","last_fullsized":"2024-04-20T01:47:20.983286+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:44.125577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.983194+0000","last_change":"2024-04-20T01:47:20.069279+0000","last_active":"2024-04-20T01:47:20.983194+0000","last_peered":"2024-04-20T01:47:20.983194+0000","last_clean":"2024-04-20T01:47:20.983194+0000","last_became_active":"2024-04-20T01:47:20.069005+0000","last_became_peered":"2024-04-20T01:47:20.069005+0000","last_unstale":"2024-04-20T01:47:20.983194+0000","last_undegraded":"2024-04-20T01:47:20.983194+0000","last_fullsized":"2024-04-20T01:47:20.983194+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:22.648743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"25'157","reported_seq":208,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.033172+0000","last_change":"2024-04-20T01:47:17.303029+0000","last_active":"2024-04-20T01:47:20.033172+0000","last_peered":"2024-04-20T01:47:20.033172+0000","last_clean":"2024-04-20T01:47:20.033172+0000","last_became_active":"2024-04-20T01:47:17.302105+0000","last_became_peered":"2024-04-20T01:47:17.302105+0000","last_unstale":"2024-04-20T01:47:20.033172+0000","last_undegraded":"2024-04-20T01:47:20.033172+0000","last_fullsized":"2024-04-20T01:47:20.033172+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_clean_scrub_stamp":"2024-04-20T01:47:15.986754+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:30.481506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,15],"acting":[13,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.985219+0000","last_change":"2024-04-20T01:47:20.039014+0000","last_active":"2024-04-20T01:47:20.985219+0000","last_peered":"2024-04-20T01:47:20.985219+0000","last_clean":"2024-04-20T01:47:20.985219+0000","last_became_active":"2024-04-20T01:47:20.038764+0000","last_became_peered":"2024-04-20T01:47:20.038764+0000","last_unstale":"2024-04-20T01:47:20.985219+0000","last_undegraded":"2024-04-20T01:47:20.985219+0000","last_fullsized":"2024-04-20T01:47:20.985219+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:26.112165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:21.972435+0000","last_change":"2024-04-20T01:47:20.039221+0000","last_active":"2024-04-20T01:47:21.972435+0000","last_peered":"2024-04-20T01:47:21.972435+0000","last_clean":"2024-04-20T01:47:21.972435+0000","last_became_active":"2024-04-20T01:47:20.038972+0000","last_became_peered":"2024-04-20T01:47:20.038972+0000","last_unstale":"2024-04-20T01:47:21.972435+0000","last_undegraded":"2024-04-20T01:47:21.972435+0000","last_fullsized":"2024-04-20T01:47:21.972435+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:24.167832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.983211+0000","last_change":"2024-04-20T01:47:20.075613+0000","last_active":"2024-04-20T01:47:20.983211+0000","last_peered":"2024-04-20T01:47:20.983211+0000","last_clean":"2024-04-20T01:47:20.983211+0000","last_became_active":"2024-04-20T01:47:20.075314+0000","last_became_peered":"2024-04-20T01:47:20.075314+0000","last_unstale":"2024-04-20T01:47:20.983211+0000","last_undegraded":"2024-04-20T01:47:20.983211+0000","last_fullsized":"2024-04-20T01:47:20.983211+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:35.956173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,12],"acting":[2,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.985668+0000","last_change":"2024-04-20T01:47:20.071988+0000","last_active":"2024-04-20T01:47:20.985668+0000","last_peered":"2024-04-20T01:47:20.985668+0000","last_clean":"2024-04-20T01:47:20.985668+0000","last_became_active":"2024-04-20T01:47:20.071615+0000","last_became_peered":"2024-04-20T01:47:20.071615+0000","last_unstale":"2024-04-20T01:47:20.985668+0000","last_undegraded":"2024-04-20T01:47:20.985668+0000","last_fullsized":"2024-04-20T01:47:20.985668+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:40.106777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280517,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":643566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]}]},{"osd":15,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":643566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]},{"osd":14,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":12,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]}]},{"osd":11,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]}]},{"osd":10,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[1,2,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]}]},{"osd":9,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,4,6,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3319999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]}]},{"osd":8,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,6,7,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9630000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8009999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9830000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]}]},{"osd":7,"up_from":21,"seq":90194313222,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,4,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]}]},{"osd":6,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[5,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0529999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]}]},{"osd":1,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]}]},{"osd":0,"up_from":21,"seq":90194313222,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":114052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]},{"osd":2,"up_from":21,"seq":90194313220,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,4,5,6,7,8,9,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.9020000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.226}]}]},{"osd":3,"up_from":21,"seq":90194313222,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[2,4,5,6,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.959}]}]},{"osd":4,"up_from":21,"seq":90194313222,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,5,6,7,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.0679999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.507999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.9180000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.895}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.051}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.957000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.929}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.0229999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.987}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.720000000000001}]}]},{"osd":5,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,6,7,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.423999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T01:47:27.316 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T01:47:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:47:27.687 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T01:47:27.699 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-20T01:47:27.137693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":437872,"kb_used_data":5516,"kb_used_omap":24,"kb_used_meta":432103,"kb_avail":1509511568,"statfs":{"total":1546188226560,"available":1545739845632,"internally_reserved":0,"allocated":5648384,"data_stored":2891478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25507,"internal_metadata":442473565},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001078"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.981697+0000","last_change":"2024-04-20T01:47:20.036634+0000","last_active":"2024-04-20T01:47:20.981697+0000","last_peered":"2024-04-20T01:47:20.981697+0000","last_clean":"2024-04-20T01:47:20.981697+0000","last_became_active":"2024-04-20T01:47:20.036328+0000","last_became_peered":"2024-04-20T01:47:20.036328+0000","last_unstale":"2024-04-20T01:47:20.981697+0000","last_undegraded":"2024-04-20T01:47:20.981697+0000","last_fullsized":"2024-04-20T01:47:20.981697+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.385250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,13],"acting":[6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.985494+0000","last_change":"2024-04-20T01:47:20.110600+0000","last_active":"2024-04-20T01:47:20.985494+0000","last_peered":"2024-04-20T01:47:20.985494+0000","last_clean":"2024-04-20T01:47:20.985494+0000","last_became_active":"2024-04-20T01:47:20.110260+0000","last_became_peered":"2024-04-20T01:47:20.110260+0000","last_unstale":"2024-04-20T01:47:20.985494+0000","last_undegraded":"2024-04-20T01:47:20.985494+0000","last_fullsized":"2024-04-20T01:47:20.985494+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:47.718290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.983286+0000","last_change":"2024-04-20T01:47:20.038409+0000","last_active":"2024-04-20T01:47:20.983286+0000","last_peered":"2024-04-20T01:47:20.983286+0000","last_clean":"2024-04-20T01:47:20.983286+0000","last_became_active":"2024-04-20T01:47:20.038107+0000","last_became_peered":"2024-04-20T01:47:20.038107+0000","last_unstale":"2024-04-20T01:47:20.983286+0000","last_undegraded":"2024-04-20T01:47:20.983286+0000","last_fullsized":"2024-04-20T01:47:20.983286+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:44.125577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.983194+0000","last_change":"2024-04-20T01:47:20.069279+0000","last_active":"2024-04-20T01:47:20.983194+0000","last_peered":"2024-04-20T01:47:20.983194+0000","last_clean":"2024-04-20T01:47:20.983194+0000","last_became_active":"2024-04-20T01:47:20.069005+0000","last_became_peered":"2024-04-20T01:47:20.069005+0000","last_unstale":"2024-04-20T01:47:20.983194+0000","last_undegraded":"2024-04-20T01:47:20.983194+0000","last_fullsized":"2024-04-20T01:47:20.983194+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:22.648743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"25'157","reported_seq":208,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.033172+0000","last_change":"2024-04-20T01:47:17.303029+0000","last_active":"2024-04-20T01:47:20.033172+0000","last_peered":"2024-04-20T01:47:20.033172+0000","last_clean":"2024-04-20T01:47:20.033172+0000","last_became_active":"2024-04-20T01:47:17.302105+0000","last_became_peered":"2024-04-20T01:47:17.302105+0000","last_unstale":"2024-04-20T01:47:20.033172+0000","last_undegraded":"2024-04-20T01:47:20.033172+0000","last_fullsized":"2024-04-20T01:47:20.033172+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_clean_scrub_stamp":"2024-04-20T01:47:15.986754+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:30.481506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,15],"acting":[13,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.985219+0000","last_change":"2024-04-20T01:47:20.039014+0000","last_active":"2024-04-20T01:47:20.985219+0000","last_peered":"2024-04-20T01:47:20.985219+0000","last_clean":"2024-04-20T01:47:20.985219+0000","last_became_active":"2024-04-20T01:47:20.038764+0000","last_became_peered":"2024-04-20T01:47:20.038764+0000","last_unstale":"2024-04-20T01:47:20.985219+0000","last_undegraded":"2024-04-20T01:47:20.985219+0000","last_fullsized":"2024-04-20T01:47:20.985219+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:26.112165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:21.972435+0000","last_change":"2024-04-20T01:47:20.039221+0000","last_active":"2024-04-20T01:47:21.972435+0000","last_peered":"2024-04-20T01:47:21.972435+0000","last_clean":"2024-04-20T01:47:21.972435+0000","last_became_active":"2024-04-20T01:47:20.038972+0000","last_became_peered":"2024-04-20T01:47:20.038972+0000","last_unstale":"2024-04-20T01:47:21.972435+0000","last_undegraded":"2024-04-20T01:47:21.972435+0000","last_fullsized":"2024-04-20T01:47:21.972435+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:24.167832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.983211+0000","last_change":"2024-04-20T01:47:20.075613+0000","last_active":"2024-04-20T01:47:20.983211+0000","last_peered":"2024-04-20T01:47:20.983211+0000","last_clean":"2024-04-20T01:47:20.983211+0000","last_became_active":"2024-04-20T01:47:20.075314+0000","last_became_peered":"2024-04-20T01:47:20.075314+0000","last_unstale":"2024-04-20T01:47:20.983211+0000","last_undegraded":"2024-04-20T01:47:20.983211+0000","last_fullsized":"2024-04-20T01:47:20.983211+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:35.956173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,12],"acting":[2,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-20T01:47:20.985668+0000","last_change":"2024-04-20T01:47:20.071988+0000","last_active":"2024-04-20T01:47:20.985668+0000","last_peered":"2024-04-20T01:47:20.985668+0000","last_clean":"2024-04-20T01:47:20.985668+0000","last_became_active":"2024-04-20T01:47:20.071615+0000","last_became_peered":"2024-04-20T01:47:20.071615+0000","last_unstale":"2024-04-20T01:47:20.985668+0000","last_undegraded":"2024-04-20T01:47:20.985668+0000","last_fullsized":"2024-04-20T01:47:20.985668+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:40.106777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280517,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":643566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]}]},{"osd":15,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":643566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]},{"osd":14,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":12,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]}]},{"osd":11,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]}]},{"osd":10,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[1,2,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]}]},{"osd":9,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,4,6,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3319999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]}]},{"osd":8,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,6,7,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9630000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8009999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9830000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]}]},{"osd":7,"up_from":21,"seq":90194313222,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,4,6,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]}]},{"osd":6,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[5,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0529999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]}]},{"osd":1,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]}]},{"osd":0,"up_from":21,"seq":90194313222,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":114052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]},{"osd":2,"up_from":21,"seq":90194313220,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,4,5,6,7,8,9,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.9020000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.226}]}]},{"osd":3,"up_from":21,"seq":90194313222,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[2,4,5,6,7,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.959}]}]},{"osd":4,"up_from":21,"seq":90194313222,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,5,6,7,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.0679999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.507999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.9180000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.895}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.051}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.957000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.929}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.0229999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.987}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.720000000000001}]}]},{"osd":5,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":114638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,4,6,7,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.423999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T01:47:27.701 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-20T01:47:27.701 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-04-20T01:47:28.068 INFO:teuthology.orchestra.run.smithi044.stdout:epoch 27 2024-04-20T01:47:28.068 INFO:teuthology.orchestra.run.smithi044.stdout:fsid 9c2f64c1-aa35-4337-9f5c-081b9e3a2938 2024-04-20T01:47:28.069 INFO:teuthology.orchestra.run.smithi044.stdout:created 2024-04-20T01:47:02.934241+0000 2024-04-20T01:47:28.069 INFO:teuthology.orchestra.run.smithi044.stdout:modified 2024-04-20T01:47:21.024143+0000 2024-04-20T01:47:28.069 INFO:teuthology.orchestra.run.smithi044.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-20T01:47:28.069 INFO:teuthology.orchestra.run.smithi044.stdout:crush_version 4 2024-04-20T01:47:28.069 INFO:teuthology.orchestra.run.smithi044.stdout:full_ratio 0.95 2024-04-20T01:47:28.069 INFO:teuthology.orchestra.run.smithi044.stdout:backfillfull_ratio 0.9 2024-04-20T01:47:28.069 INFO:teuthology.orchestra.run.smithi044.stdout:nearfull_ratio 0.85 2024-04-20T01:47:28.069 INFO:teuthology.orchestra.run.smithi044.stdout:require_min_compat_client luminous 2024-04-20T01:47:28.069 INFO:teuthology.orchestra.run.smithi044.stdout:min_compat_client jewel 2024-04-20T01:47:28.069 INFO:teuthology.orchestra.run.smithi044.stdout:require_osd_release reef 2024-04-20T01:47:28.069 INFO:teuthology.orchestra.run.smithi044.stdout:stretch_mode_enabled false 2024-04-20T01:47:28.069 INFO:teuthology.orchestra.run.smithi044.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 24 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 15.38 2024-04-20T01:47:28.069 INFO:teuthology.orchestra.run.smithi044.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 27 flags hashpspool stripe_width 0 application rbd read_balance_score 3.99 2024-04-20T01:47:28.069 INFO:teuthology.orchestra.run.smithi044.stdout:max_osd 16 2024-04-20T01:47:28.070 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6824/43518,v1:172.21.15.44:6825/43518] [v2:172.21.15.44:6826/43518,v1:172.21.15.44:6827/43518] exists,up 3903b88f-ff2b-4252-9480-92b701b1e59b 2024-04-20T01:47:28.070 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6816/74361,v1:172.21.15.121:6817/74361] [v2:172.21.15.121:6818/74361,v1:172.21.15.121:6819/74361] exists,up c5ff0175-b4f0-40a5-86c7-7a0a351bf070 2024-04-20T01:47:28.070 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6809/43108,v1:172.21.15.123:6811/43108] [v2:172.21.15.123:6814/43108,v1:172.21.15.123:6816/43108] exists,up 7af325c1-36b5-420c-a2a9-44c00a002977 2024-04-20T01:47:28.070 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6824/74545,v1:172.21.15.165:6825/74545] [v2:172.21.15.165:6826/74545,v1:172.21.15.165:6827/74545] exists,up 99334f43-83ea-4c3f-b91d-ae72c8361b8c 2024-04-20T01:47:28.070 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6809/43517,v1:172.21.15.44:6811/43517] [v2:172.21.15.44:6813/43517,v1:172.21.15.44:6815/43517] exists,up 8764d799-af67-4e6c-889f-9fdd91201cc2 2024-04-20T01:47:28.070 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6800/74354,v1:172.21.15.121:6801/74354] [v2:172.21.15.121:6802/74354,v1:172.21.15.121:6803/74354] exists,up 2556b2fa-4dcc-479c-a379-6f9fe96e9c77 2024-04-20T01:47:28.070 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6800/43107,v1:172.21.15.123:6801/43107] [v2:172.21.15.123:6802/43107,v1:172.21.15.123:6803/43107] exists,up baeaae3a-fff4-4ccc-a08a-ff32db4531ba 2024-04-20T01:47:28.070 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6800/74543,v1:172.21.15.165:6801/74543] [v2:172.21.15.165:6802/74543,v1:172.21.15.165:6803/74543] exists,up 0019f005-3e62-4d9d-b334-f1d347a484ce 2024-04-20T01:47:28.070 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6808/43515,v1:172.21.15.44:6810/43515] [v2:172.21.15.44:6812/43515,v1:172.21.15.44:6814/43515] exists,up 6f5200a3-8980-4241-a6fc-ef604fa612b2 2024-04-20T01:47:28.070 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6824/74379,v1:172.21.15.121:6825/74379] [v2:172.21.15.121:6826/74379,v1:172.21.15.121:6827/74379] exists,up 29cf9aaa-3a99-4605-92f4-64f1f6a1728d 2024-04-20T01:47:28.070 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6808/43109,v1:172.21.15.123:6810/43109] [v2:172.21.15.123:6812/43109,v1:172.21.15.123:6813/43109] exists,up 705906c4-1514-4754-ac40-45226a837180 2024-04-20T01:47:28.070 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6808/74544,v1:172.21.15.165:6809/74544] [v2:172.21.15.165:6810/74544,v1:172.21.15.165:6811/74544] exists,up b45f2645-72d1-4d72-89e9-2212f5aa074c 2024-04-20T01:47:28.071 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6800/43516,v1:172.21.15.44:6801/43516] [v2:172.21.15.44:6802/43516,v1:172.21.15.44:6803/43516] exists,up 3ea410b9-a368-467b-a179-2e3aba562d63 2024-04-20T01:47:28.071 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13 up in weight 1 up_from 22 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6808/74355,v1:172.21.15.121:6809/74355] [v2:172.21.15.121:6810/74355,v1:172.21.15.121:6811/74355] exists,up 15a5154b-e7cb-4b57-8468-dcb048d93f63 2024-04-20T01:47:28.071 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6824/43110,v1:172.21.15.123:6825/43110] [v2:172.21.15.123:6826/43110,v1:172.21.15.123:6827/43110] exists,up 23b53da2-f848-4e76-9632-4d9effd5b430 2024-04-20T01:47:28.071 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] [v2:172.21.15.165:6818/74562,v1:172.21.15.165:6819/74562] exists,up 546efc0f-82a3-47d2-aeec-2d849eb1faa1 2024-04-20T01:47:28.085 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6824/43518,v1:172.21.15.44:6825/43518] [v2:172.21.15.44:6826/43518,v1:172.21.15.44:6827/43518] exists,up 3903b88f-ff2b-4252-9480-92b701b1e59b', 'osd.1 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6816/74361,v1:172.21.15.121:6817/74361] [v2:172.21.15.121:6818/74361,v1:172.21.15.121:6819/74361] exists,up c5ff0175-b4f0-40a5-86c7-7a0a351bf070', 'osd.2 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6809/43108,v1:172.21.15.123:6811/43108] [v2:172.21.15.123:6814/43108,v1:172.21.15.123:6816/43108] exists,up 7af325c1-36b5-420c-a2a9-44c00a002977', 'osd.3 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6824/74545,v1:172.21.15.165:6825/74545] [v2:172.21.15.165:6826/74545,v1:172.21.15.165:6827/74545] exists,up 99334f43-83ea-4c3f-b91d-ae72c8361b8c', 'osd.4 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6809/43517,v1:172.21.15.44:6811/43517] [v2:172.21.15.44:6813/43517,v1:172.21.15.44:6815/43517] exists,up 8764d799-af67-4e6c-889f-9fdd91201cc2', 'osd.5 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6800/74354,v1:172.21.15.121:6801/74354] [v2:172.21.15.121:6802/74354,v1:172.21.15.121:6803/74354] exists,up 2556b2fa-4dcc-479c-a379-6f9fe96e9c77', 'osd.6 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6800/43107,v1:172.21.15.123:6801/43107] [v2:172.21.15.123:6802/43107,v1:172.21.15.123:6803/43107] exists,up baeaae3a-fff4-4ccc-a08a-ff32db4531ba', 'osd.7 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6800/74543,v1:172.21.15.165:6801/74543] [v2:172.21.15.165:6802/74543,v1:172.21.15.165:6803/74543] exists,up 0019f005-3e62-4d9d-b334-f1d347a484ce', 'osd.8 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6808/43515,v1:172.21.15.44:6810/43515] [v2:172.21.15.44:6812/43515,v1:172.21.15.44:6814/43515] exists,up 6f5200a3-8980-4241-a6fc-ef604fa612b2', 'osd.9 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6824/74379,v1:172.21.15.121:6825/74379] [v2:172.21.15.121:6826/74379,v1:172.21.15.121:6827/74379] exists,up 29cf9aaa-3a99-4605-92f4-64f1f6a1728d', 'osd.10 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6808/43109,v1:172.21.15.123:6810/43109] [v2:172.21.15.123:6812/43109,v1:172.21.15.123:6813/43109] exists,up 705906c4-1514-4754-ac40-45226a837180', 'osd.11 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6808/74544,v1:172.21.15.165:6809/74544] [v2:172.21.15.165:6810/74544,v1:172.21.15.165:6811/74544] exists,up b45f2645-72d1-4d72-89e9-2212f5aa074c', 'osd.12 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6800/43516,v1:172.21.15.44:6801/43516] [v2:172.21.15.44:6802/43516,v1:172.21.15.44:6803/43516] exists,up 3ea410b9-a368-467b-a179-2e3aba562d63', 'osd.13 up in weight 1 up_from 22 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6808/74355,v1:172.21.15.121:6809/74355] [v2:172.21.15.121:6810/74355,v1:172.21.15.121:6811/74355] exists,up 15a5154b-e7cb-4b57-8468-dcb048d93f63', 'osd.14 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6824/43110,v1:172.21.15.123:6825/43110] [v2:172.21.15.123:6826/43110,v1:172.21.15.123:6827/43110] exists,up 23b53da2-f848-4e76-9632-4d9effd5b430', 'osd.15 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] [v2:172.21.15.165:6818/74562,v1:172.21.15.165:6819/74562] exists,up 546efc0f-82a3-47d2-aeec-2d849eb1faa1'] 2024-04-20T01:47:28.086 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-04-20T01:47:28.223 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:28.223 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mon.a", 2024-04-20T01:47:28.224 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster": "ceph", 2024-04-20T01:47:28.224 INFO:teuthology.orchestra.run.smithi044.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-04-20T01:47:28.224 INFO:teuthology.orchestra.run.smithi044.stdout: "admin_socket_mode": "", 2024-04-20T01:47:28.224 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-04-20T01:47:28.224 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_client_required": "cephx, none", 2024-04-20T01:47:28.224 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_cluster_required": "cephx", 2024-04-20T01:47:28.224 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_debug": "true", 2024-04-20T01:47:28.224 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-04-20T01:47:28.224 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-04-20T01:47:28.224 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_service_required": "cephx", 2024-04-20T01:47:28.224 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_service_ticket_ttl": "240.000000", 2024-04-20T01:47:28.224 INFO:teuthology.orchestra.run.smithi044.stdout: "auth_supported": "cephx", 2024-04-20T01:47:28.224 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_aio": "true", 2024-04-20T01:47:28.225 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_aio_max_queue_depth": "1024", 2024-04-20T01:47:28.225 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_aio_poll_ms": "250", 2024-04-20T01:47:28.225 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_aio_reap_max": "16", 2024-04-20T01:47:28.225 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_async_discard": "false", 2024-04-20T01:47:28.225 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_block_size": "4096", 2024-04-20T01:47:28.225 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_debug_aio": "false", 2024-04-20T01:47:28.225 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-04-20T01:47:28.225 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-04-20T01:47:28.225 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_debug_inflight_ios": "false", 2024-04-20T01:47:28.225 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_enable_discard": "false", 2024-04-20T01:47:28.225 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_flock_retry": "3", 2024-04-20T01:47:28.225 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_flock_retry_interval": "0.100000", 2024-04-20T01:47:28.225 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_inject_crash": "0", 2024-04-20T01:47:28.225 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_inject_crash_flush_delay": "2", 2024-04-20T01:47:28.225 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_ioring": "false", 2024-04-20T01:47:28.226 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_ioring_hipri": "false", 2024-04-20T01:47:28.226 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_ioring_sqthread_poll": "false", 2024-04-20T01:47:28.226 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-04-20T01:47:28.226 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_read_buffer_alignment": "4096", 2024-04-20T01:47:28.226 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-04-20T01:47:28.226 INFO:teuthology.orchestra.run.smithi044.stdout: "bdev_type": "", 2024-04-20T01:47:28.226 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_alloc_size": "1048576", 2024-04-20T01:47:28.226 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_allocator": "hybrid", 2024-04-20T01:47:28.226 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_buffered_io": "true", 2024-04-20T01:47:28.226 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_check_for_zeros": "false", 2024-04-20T01:47:28.226 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_check_volume_selector_often": "false", 2024-04-20T01:47:28.226 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-04-20T01:47:28.226 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_compact_log_sync": "false", 2024-04-20T01:47:28.227 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-04-20T01:47:28.227 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-04-20T01:47:28.227 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_log_compact_min_size": "16777216", 2024-04-20T01:47:28.227 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_log_replay_check_allocations": "true", 2024-04-20T01:47:28.227 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_max_log_runway": "4194304", 2024-04-20T01:47:28.227 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_max_prefetch": "1048576", 2024-04-20T01:47:28.227 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_min_flush_size": "524288", 2024-04-20T01:47:28.227 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_min_log_runway": "1048576", 2024-04-20T01:47:28.227 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_replay_recovery": "false", 2024-04-20T01:47:28.227 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-04-20T01:47:28.227 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_shared_alloc_size": "65536", 2024-04-20T01:47:28.228 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_sync_write": "false", 2024-04-20T01:47:28.228 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-04-20T01:47:28.228 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-04-20T01:47:28.228 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-04-20T01:47:28.228 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_allocation_from_file": "true", 2024-04-20T01:47:28.228 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_allocator": "hybrid", 2024-04-20T01:47:28.228 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-04-20T01:47:28.228 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-04-20T01:47:28.228 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-04-20T01:47:28.228 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-04-20T01:47:28.228 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-04-20T01:47:28.229 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-04-20T01:47:28.229 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blobid_prealloc": "10240", 2024-04-20T01:47:28.229 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_block_create": "true", 2024-04-20T01:47:28.229 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_block_db_create": "false", 2024-04-20T01:47:28.229 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_block_db_path": "", 2024-04-20T01:47:28.229 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_block_db_size": "0", 2024-04-20T01:47:28.229 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_block_path": "", 2024-04-20T01:47:28.229 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_block_preallocate_file": "false", 2024-04-20T01:47:28.229 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_block_size": "107374182400", 2024-04-20T01:47:28.229 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_block_wal_create": "false", 2024-04-20T01:47:28.229 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_block_wal_path": "", 2024-04-20T01:47:28.229 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_block_wal_size": "100663296", 2024-04-20T01:47:28.229 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_bluefs": "true", 2024-04-20T01:47:28.229 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-04-20T01:47:28.230 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_bluefs_env_mirror": "false", 2024-04-20T01:47:28.230 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-04-20T01:47:28.230 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-04-20T01:47:28.230 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-04-20T01:47:28.230 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-04-20T01:47:28.230 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-04-20T01:47:28.230 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-04-20T01:47:28.230 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_autotune": "true", 2024-04-20T01:47:28.230 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-04-20T01:47:28.230 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-04-20T01:47:28.230 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-04-20T01:47:28.230 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-04-20T01:47:28.230 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_size": "0", 2024-04-20T01:47:28.230 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-04-20T01:47:28.231 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-04-20T01:47:28.231 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-04-20T01:47:28.231 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-04-20T01:47:28.231 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_type": "2q", 2024-04-20T01:47:28.231 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-04-20T01:47:28.231 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_clone_cow": "true", 2024-04-20T01:47:28.231 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_compression_algorithm": "snappy", 2024-04-20T01:47:28.231 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_compression_max_blob_size": "0", 2024-04-20T01:47:28.231 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-04-20T01:47:28.231 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-04-20T01:47:28.231 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_compression_min_blob_size": "0", 2024-04-20T01:47:28.231 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-04-20T01:47:28.231 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-04-20T01:47:28.231 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_compression_mode": "none", 2024-04-20T01:47:28.232 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-04-20T01:47:28.232 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_csum_type": "crc32c", 2024-04-20T01:47:28.232 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_enforce_settings": "default", 2024-04-20T01:47:28.232 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_freelist": "false", 2024-04-20T01:47:28.232 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_fsck_abort": "false", 2024-04-20T01:47:28.232 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-04-20T01:47:28.232 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_inject_bug21040": "false", 2024-04-20T01:47:28.232 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-04-20T01:47:28.232 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_inject_read_err": "false", 2024-04-20T01:47:28.232 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_legacy_omap": "false", 2024-04-20T01:47:28.232 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-04-20T01:47:28.232 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-04-20T01:47:28.232 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-04-20T01:47:28.232 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-04-20T01:47:28.233 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_prefill": "0.000000", 2024-04-20T01:47:28.233 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-04-20T01:47:28.233 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-04-20T01:47:28.233 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-04-20T01:47:28.233 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_small_allocations": "0", 2024-04-20T01:47:28.233 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-04-20T01:47:28.233 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_default_buffered_read": "true", 2024-04-20T01:47:28.233 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_default_buffered_write": "false", 2024-04-20T01:47:28.233 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_deferred_batch_ops": "0", 2024-04-20T01:47:28.233 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-04-20T01:47:28.233 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-04-20T01:47:28.233 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-04-20T01:47:28.233 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-04-20T01:47:28.233 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-04-20T01:47:28.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-04-20T01:47:28.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-04-20T01:47:28.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fail_eio": "false", 2024-04-20T01:47:28.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-04-20T01:47:28.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-04-20T01:47:28.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-04-20T01:47:28.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-04-20T01:47:28.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_on_mkfs": "true", 2024-04-20T01:47:28.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-04-20T01:47:28.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_on_mount": "false", 2024-04-20T01:47:28.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-04-20T01:47:28.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_on_umount": "false", 2024-04-20T01:47:28.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-04-20T01:47:28.234 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-04-20T01:47:28.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-04-20T01:47:28.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-04-20T01:47:28.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-04-20T01:47:28.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-04-20T01:47:28.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-04-20T01:47:28.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-04-20T01:47:28.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_ignore_data_csum": "false", 2024-04-20T01:47:28.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-04-20T01:47:28.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_kvbackend": "rocksdb", 2024-04-20T01:47:28.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-04-20T01:47:28.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-04-20T01:47:28.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_log_op_age": "5.000000", 2024-04-20T01:47:28.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_max_alloc_size": "0", 2024-04-20T01:47:28.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_max_blob_size": "0", 2024-04-20T01:47:28.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-04-20T01:47:28.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-04-20T01:47:28.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_max_defer_interval": "3.000000", 2024-04-20T01:47:28.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_max_deferred_txc": "32", 2024-04-20T01:47:28.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_min_alloc_size": "0", 2024-04-20T01:47:28.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-04-20T01:47:28.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-04-20T01:47:28.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_nid_prealloc": "1024", 2024-04-20T01:47:28.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_prefer_deferred_size": "0", 2024-04-20T01:47:28.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-04-20T01:47:28.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-04-20T01:47:28.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_qfsck_on_mount": "true", 2024-04-20T01:47:28.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_retry_disk_reads": "3", 2024-04-20T01:47:28.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_rocksdb_cf": "true", 2024-04-20T01:47:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-04-20T01:47:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-04-20T01:47:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_rocksdb_options_annex": "", 2024-04-20T01:47:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_spdk_coremask": "0x1", 2024-04-20T01:47:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_spdk_io_sleep": "5", 2024-04-20T01:47:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_spdk_max_io_completion": "0", 2024-04-20T01:47:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_spdk_mem": "512", 2024-04-20T01:47:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_sync_submit_transaction": "false", 2024-04-20T01:47:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_throttle_bytes": "67108864", 2024-04-20T01:47:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_throttle_cost_per_io": "0", 2024-04-20T01:47:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-04-20T01:47:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-04-20T01:47:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-04-20T01:47:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-04-20T01:47:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_tracing": "false", 2024-04-20T01:47:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-04-20T01:47:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-04-20T01:47:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_volume_selection_reserved": "0", 2024-04-20T01:47:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-04-20T01:47:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-04-20T01:47:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-04-20T01:47:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-04-20T01:47:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-04-20T01:47:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-04-20T01:47:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_zero_block_detection": "false", 2024-04-20T01:47:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-04-20T01:47:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "cephfs_mirror_action_update_interval": "2", 2024-04-20T01:47:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-04-20T01:47:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-04-20T01:47:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-04-20T01:47:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-04-20T01:47:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "cephfs_mirror_mount_timeout": "10", 2024-04-20T01:47:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-04-20T01:47:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-04-20T01:47:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-04-20T01:47:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-04-20T01:47:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-04-20T01:47:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-04-20T01:47:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-04-20T01:47:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "cephx_cluster_require_signatures": "false", 2024-04-20T01:47:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "cephx_cluster_require_version": "2", 2024-04-20T01:47:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "cephx_require_signatures": "false", 2024-04-20T01:47:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "cephx_require_version": "2", 2024-04-20T01:47:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "cephx_service_require_signatures": "false", 2024-04-20T01:47:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "cephx_service_require_version": "2", 2024-04-20T01:47:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "cephx_sign_messages": "true", 2024-04-20T01:47:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "chdir": "", 2024-04-20T01:47:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "client_acl_type": "", 2024-04-20T01:47:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "client_asio_thread_count": "2", 2024-04-20T01:47:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "client_cache_mid": "0.750000", 2024-04-20T01:47:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "client_cache_size": "16384", 2024-04-20T01:47:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "client_caps_release_delay": "5", 2024-04-20T01:47:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "client_check_pool_perm": "true", 2024-04-20T01:47:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "client_collect_and_send_global_metrics": "false", 2024-04-20T01:47:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "client_debug_force_sync_read": "false", 2024-04-20T01:47:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "client_debug_getattr_caps": "false", 2024-04-20T01:47:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "client_debug_inject_tick_delay": "0", 2024-04-20T01:47:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-04-20T01:47:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client_die_on_failed_remount": "false", 2024-04-20T01:47:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client_dirsize_rbytes": "true", 2024-04-20T01:47:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client_force_lazyio": "false", 2024-04-20T01:47:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client_fs": "", 2024-04-20T01:47:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client_inject_fixed_oldest_tid": "false", 2024-04-20T01:47:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client_inject_release_failure": "false", 2024-04-20T01:47:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client_max_inline_size": "4096", 2024-04-20T01:47:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client_max_retries_on_remount_failure": "5", 2024-04-20T01:47:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client_mds_namespace": "", 2024-04-20T01:47:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client_metadata": "", 2024-04-20T01:47:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client_mount_gid": "-1", 2024-04-20T01:47:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client_mount_timeout": "300", 2024-04-20T01:47:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client_mount_uid": "-1", 2024-04-20T01:47:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: "client_mountpoint": "/", 2024-04-20T01:47:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: "client_notify_timeout": "10", 2024-04-20T01:47:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: "client_oc": "true", 2024-04-20T01:47:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: "client_oc_max_dirty": "104857600", 2024-04-20T01:47:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: "client_oc_max_dirty_age": "5.000000", 2024-04-20T01:47:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: "client_oc_max_objects": "1000", 2024-04-20T01:47:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: "client_oc_size": "209715200", 2024-04-20T01:47:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: "client_oc_target_dirty": "8388608", 2024-04-20T01:47:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: "client_permissions": "true", 2024-04-20T01:47:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: "client_quota": "true", 2024-04-20T01:47:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: "client_quota_df": "true", 2024-04-20T01:47:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: "client_readahead_max_bytes": "0", 2024-04-20T01:47:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: "client_readahead_max_periods": "4", 2024-04-20T01:47:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "client_readahead_min": "131072", 2024-04-20T01:47:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "client_reconnect_stale": "false", 2024-04-20T01:47:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "client_shutdown_timeout": "30", 2024-04-20T01:47:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "client_snapdir": ".snap", 2024-04-20T01:47:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "client_tick_interval": "1", 2024-04-20T01:47:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "client_trace": "", 2024-04-20T01:47:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "client_try_dentry_invalidate": "false", 2024-04-20T01:47:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "client_use_faked_inos": "false", 2024-04-20T01:47:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "client_use_random_mds": "false", 2024-04-20T01:47:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "clog_to_graylog": "false", 2024-04-20T01:47:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-04-20T01:47:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "clog_to_graylog_port": "12201", 2024-04-20T01:47:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "clog_to_monitors": "default=true", 2024-04-20T01:47:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "clog_to_syslog": "false", 2024-04-20T01:47:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-04-20T01:47:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "clog_to_syslog_level": "info", 2024-04-20T01:47:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "-", 2024-04-20T01:47:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_network": "", 2024-04-20T01:47:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_network_interface": "", 2024-04-20T01:47:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "compressor_zlib_isal": "false", 2024-04-20T01:47:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "compressor_zlib_level": "5", 2024-04-20T01:47:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "compressor_zlib_winsize": "-15", 2024-04-20T01:47:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "compressor_zstd_level": "1", 2024-04-20T01:47:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-04-20T01:47:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-04-20T01:47:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "crimson_alien_op_num_threads": "6", 2024-04-20T01:47:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "crimson_alien_thread_cpu_cores": "", 2024-04-20T01:47:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "crimson_osd_obc_lru_size": "10", 2024-04-20T01:47:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-04-20T01:47:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "crimson_seastar_smp": "1", 2024-04-20T01:47:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location": "", 2024-04-20T01:47:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location_hook": "", 2024-04-20T01:47:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_location_hook_timeout": "10", 2024-04-20T01:47:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "daemonize": "false", 2024-04-20T01:47:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "daos_pool": "tank", 2024-04-20T01:47:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-04-20T01:47:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-04-20T01:47:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "dbstore_db_name_prefix": "dbstore", 2024-04-20T01:47:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_alienstore": "0/5", 2024-04-20T01:47:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_allow_any_pool_priority": "false", 2024-04-20T01:47:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_asok": "1/5", 2024-04-20T01:47:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_asok_assert_abort": "false", 2024-04-20T01:47:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_asserts_on_shutdown": "true", 2024-04-20T01:47:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_auth": "20/20", 2024-04-20T01:47:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_bdev": "1/3", 2024-04-20T01:47:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_bluefs": "1/5", 2024-04-20T01:47:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_bluestore": "1/5", 2024-04-20T01:47:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_buffer": "0/1", 2024-04-20T01:47:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_ceph_exporter": "1/5", 2024-04-20T01:47:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_cephfs_mirror": "0/5", 2024-04-20T01:47:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_cephsqlite": "0/5", 2024-04-20T01:47:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_client": "0/5", 2024-04-20T01:47:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_compressor": "1/5", 2024-04-20T01:47:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_context": "0/1", 2024-04-20T01:47:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_crush": "1/1", 2024-04-20T01:47:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_crypto": "1/5", 2024-04-20T01:47:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_cyanstore": "0/5", 2024-04-20T01:47:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_deliberately_leak_memory": "false", 2024-04-20T01:47:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_disable_randomized_ping": "false", 2024-04-20T01:47:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_dpdk": "1/5", 2024-04-20T01:47:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_eventtrace": "1/5", 2024-04-20T01:47:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_filer": "0/1", 2024-04-20T01:47:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_filestore": "1/3", 2024-04-20T01:47:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_finisher": "1/1", 2024-04-20T01:47:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_fuse": "1/5", 2024-04-20T01:47:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_heartbeat_testing_span": "0", 2024-04-20T01:47:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_heartbeatmap": "1/5", 2024-04-20T01:47:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_immutable_obj_cache": "0/5", 2024-04-20T01:47:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_javaclient": "1/5", 2024-04-20T01:47:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_journal": "1/3", 2024-04-20T01:47:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_journaler": "0/5", 2024-04-20T01:47:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_kstore": "1/5", 2024-04-20T01:47:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_leveldb": "4/5", 2024-04-20T01:47:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_lockdep": "0/1", 2024-04-20T01:47:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_mclock": "1/5", 2024-04-20T01:47:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_mds": "1/5", 2024-04-20T01:47:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_mds_balancer": "1/5", 2024-04-20T01:47:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_mds_locker": "1/5", 2024-04-20T01:47:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_mds_log": "1/5", 2024-04-20T01:47:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_mds_log_expire": "1/5", 2024-04-20T01:47:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_mds_migrator": "1/5", 2024-04-20T01:47:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_memstore": "1/5", 2024-04-20T01:47:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_mgr": "2/5", 2024-04-20T01:47:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_mgrc": "1/5", 2024-04-20T01:47:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_mon": "20/20", 2024-04-20T01:47:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_monc": "0/10", 2024-04-20T01:47:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_ms": "1/1", 2024-04-20T01:47:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_none": "0/5", 2024-04-20T01:47:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_objclass": "0/5", 2024-04-20T01:47:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_objectcacher": "0/5", 2024-04-20T01:47:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_objecter": "0/1", 2024-04-20T01:47:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_optracker": "0/5", 2024-04-20T01:47:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_osd": "1/5", 2024-04-20T01:47:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_paxos": "20/20", 2024-04-20T01:47:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_perfcounter": "1/5", 2024-04-20T01:47:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_prioritycache": "1/5", 2024-04-20T01:47:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rados": "0/5", 2024-04-20T01:47:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rbd": "0/5", 2024-04-20T01:47:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rbd_mirror": "0/5", 2024-04-20T01:47:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rbd_pwl": "0/5", 2024-04-20T01:47:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rbd_replay": "0/5", 2024-04-20T01:47:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_refs": "0/0", 2024-04-20T01:47:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_reserver": "1/1", 2024-04-20T01:47:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rgw": "1/5", 2024-04-20T01:47:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rgw_access": "1/5", 2024-04-20T01:47:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rgw_datacache": "1/5", 2024-04-20T01:47:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rgw_dbstore": "1/5", 2024-04-20T01:47:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rgw_flight": "1/5", 2024-04-20T01:47:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rgw_sync": "1/5", 2024-04-20T01:47:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_rocksdb": "4/5", 2024-04-20T01:47:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore": "0/5", 2024-04-20T01:47:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_backref": "0/5", 2024-04-20T01:47:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_cache": "0/5", 2024-04-20T01:47:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_cleaner": "0/5", 2024-04-20T01:47:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_device": "0/5", 2024-04-20T01:47:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_epm": "0/5", 2024-04-20T01:47:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-04-20T01:47:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_journal": "0/5", 2024-04-20T01:47:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_lba": "0/5", 2024-04-20T01:47:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_odata": "0/5", 2024-04-20T01:47:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_omap": "0/5", 2024-04-20T01:47:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_onode": "0/5", 2024-04-20T01:47:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_t": "0/5", 2024-04-20T01:47:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_seastore_tm": "0/5", 2024-04-20T01:47:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_striper": "0/1", 2024-04-20T01:47:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_test": "0/5", 2024-04-20T01:47:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_throttle": "1/1", 2024-04-20T01:47:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_timer": "0/1", 2024-04-20T01:47:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "debug_tp": "0/5", 2024-04-20T01:47:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-04-20T01:47:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "device_failure_prediction_mode": "none", 2024-04-20T01:47:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-04-20T01:47:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-04-20T01:47:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "err_to_graylog": "false", 2024-04-20T01:47:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "err_to_journald": "false", 2024-04-20T01:47:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "err_to_stderr": "true", 2024-04-20T01:47:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "err_to_syslog": "false", 2024-04-20T01:47:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event_tracing": "false", 2024-04-20T01:47:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "exporter_addr": "0.0.0.0", 2024-04-20T01:47:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "exporter_http_port": "9926", 2024-04-20T01:47:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "exporter_prio_limit": "5", 2024-04-20T01:47:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-04-20T01:47:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "exporter_sort_metrics": "true", 2024-04-20T01:47:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "exporter_stats_period": "5", 2024-04-20T01:47:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "fake_statfs_for_testing": "0", 2024-04-20T01:47:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "fatal_signal_handlers": "true", 2024-04-20T01:47:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "filer_max_purge_ops": "10", 2024-04-20T01:47:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "filer_max_truncate_ops": "128", 2024-04-20T01:47:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_apply_finisher_threads": "1", 2024-04-20T01:47:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_blackhole": "false", 2024-04-20T01:47:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_btrfs_clone_range": "true", 2024-04-20T01:47:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_btrfs_snap": "true", 2024-04-20T01:47:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_caller_concurrency": "10", 2024-04-20T01:47:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_collect_device_partition_information": "true", 2024-04-20T01:47:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_commit_timeout": "600.000000", 2024-04-20T01:47:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_debug_inject_read_err": "false", 2024-04-20T01:47:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_debug_omap_check": "false", 2024-04-20T01:47:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_debug_random_read_err": "0.000000", 2024-04-20T01:47:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_debug_verify_split": "false", 2024-04-20T01:47:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_dump_file": "", 2024-04-20T01:47:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-04-20T01:47:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-04-20T01:47:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_fadvise": "true", 2024-04-20T01:47:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_fail_eio": "true", 2024-04-20T01:47:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_fd_cache_shards": "16", 2024-04-20T01:47:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_fd_cache_size": "128", 2024-04-20T01:47:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_fiemap": "false", 2024-04-20T01:47:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_fiemap_threshold": "4096", 2024-04-20T01:47:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-04-20T01:47:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_index_retry_probability": "0.000000", 2024-04-20T01:47:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_inject_stall": "0", 2024-04-20T01:47:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_journal_parallel": "false", 2024-04-20T01:47:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_journal_trailing": "false", 2024-04-20T01:47:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_journal_writeahead": "false", 2024-04-20T01:47:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_kill_at": "0", 2024-04-20T01:47:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-04-20T01:47:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_inline_xattr_size": "0", 2024-04-20T01:47:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-04-20T01:47:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-04-20T01:47:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-04-20T01:47:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_inline_xattrs": "0", 2024-04-20T01:47:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-04-20T01:47:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_inline_xattrs_other": "2", 2024-04-20T01:47:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-04-20T01:47:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_sync_interval": "5.000000", 2024-04-20T01:47:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_xattr_value_size": "0", 2024-04-20T01:47:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-04-20T01:47:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-04-20T01:47:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-04-20T01:47:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_merge_threshold": "-10", 2024-04-20T01:47:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_min_sync_interval": "0.010000", 2024-04-20T01:47:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_odsync_write": "false", 2024-04-20T01:47:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_omap_backend": "rocksdb", 2024-04-20T01:47:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_omap_backend_path": "", 2024-04-20T01:47:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_omap_header_cache_size": "1024", 2024-04-20T01:47:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_ondisk_finisher_threads": "1", 2024-04-20T01:47:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-04-20T01:47:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_op_thread_timeout": "60", 2024-04-20T01:47:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_op_threads": "2", 2024-04-20T01:47:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_punch_hole": "false", 2024-04-20T01:47:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-04-20T01:47:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-04-20T01:47:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-04-20T01:47:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-04-20T01:47:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-04-20T01:47:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_queue_max_bytes": "104857600", 2024-04-20T01:47:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-04-20T01:47:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-04-20T01:47:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-04-20T01:47:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_queue_max_ops": "50", 2024-04-20T01:47:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-04-20T01:47:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_seek_data_hole": "false", 2024-04-20T01:47:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_sloppy_crc": "false", 2024-04-20T01:47:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-04-20T01:47:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_splice": "false", 2024-04-20T01:47:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_split_multiple": "2", 2024-04-20T01:47:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_split_rand_factor": "20", 2024-04-20T01:47:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_update_to": "1000", 2024-04-20T01:47:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-04-20T01:47:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-04-20T01:47:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-04-20T01:47:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-04-20T01:47:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-04-20T01:47:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-04-20T01:47:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_enable": "true", 2024-04-20T01:47:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-04-20T01:47:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-04-20T01:47:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-04-20T01:47:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-04-20T01:47:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-04-20T01:47:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-04-20T01:47:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_xfs_extsize": "false", 2024-04-20T01:47:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "filestore_zfs_snap": "false", 2024-04-20T01:47:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "fio_dir": "/tmp/fio", 2024-04-20T01:47:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "9c2f64c1-aa35-4337-9f5c-081b9e3a2938", 2024-04-20T01:47:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_allow_other": "true", 2024-04-20T01:47:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_atomic_o_trunc": "true", 2024-04-20T01:47:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_big_writes": "true", 2024-04-20T01:47:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_debug": "false", 2024-04-20T01:47:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_default_permissions": "false", 2024-04-20T01:47:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_disable_pagecache": "false", 2024-04-20T01:47:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_max_write": "0", 2024-04-20T01:47:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_multithreaded": "true", 2024-04-20T01:47:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_require_active_mds": "true", 2024-04-20T01:47:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_set_user_groups": "true", 2024-04-20T01:47:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_splice_move": "true", 2024-04-20T01:47:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_splice_read": "true", 2024-04-20T01:47:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_splice_write": "true", 2024-04-20T01:47:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_syncfs_on_mksnap": "true", 2024-04-20T01:47:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "fuse_use_invalidate_cb": "true", 2024-04-20T01:47:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-04-20T01:47:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "gss_target_name": "ceph", 2024-04-20T01:47:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_file": "", 2024-04-20T01:47:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_inject_failure": "0", 2024-04-20T01:47:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_interval": "5", 2024-04-20T01:47:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "host": "smithi044", 2024-04-20T01:47:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-04-20T01:47:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-04-20T01:47:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_max_size": "1073741824", 2024-04-20T01:47:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-04-20T01:47:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-04-20T01:47:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-04-20T01:47:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-04-20T01:47:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-04-20T01:47:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-04-20T01:47:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-04-20T01:47:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-04-20T01:47:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-04-20T01:47:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "immutable_object_cache_watermark": "0.900000", 2024-04-20T01:47:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "inject_early_sigterm": "false", 2024-04-20T01:47:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "jaeger_agent_port": "6799", 2024-04-20T01:47:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "jaeger_tracing_enable": "false", 2024-04-20T01:47:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_aio": "true", 2024-04-20T01:47:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_align_min_size": "65536", 2024-04-20T01:47:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_block_align": "true", 2024-04-20T01:47:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_block_size": "4096", 2024-04-20T01:47:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_dio": "true", 2024-04-20T01:47:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_discard": "false", 2024-04-20T01:47:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_force_aio": "false", 2024-04-20T01:47:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_ignore_corruption": "false", 2024-04-20T01:47:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_max_write_bytes": "10485760", 2024-04-20T01:47:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_max_write_entries": "100", 2024-04-20T01:47:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_replay_from": "0", 2024-04-20T01:47:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_throttle_high_multiple": "0.000000", 2024-04-20T01:47:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-04-20T01:47:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-04-20T01:47:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_throttle_max_multiple": "0.000000", 2024-04-20T01:47:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_write_header_frequency": "0", 2024-04-20T01:47:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: "journal_zero_on_create": "false", 2024-04-20T01:47:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: "journaler_prefetch_periods": "10", 2024-04-20T01:47:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: "journaler_prezero_periods": "5", 2024-04-20T01:47:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: "journaler_write_head_interval": "15", 2024-04-20T01:47:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: "key": "", 2024-04-20T01:47:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "keyfile": "", 2024-04-20T01:47:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-04-20T01:47:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_backend": "rocksdb", 2024-04-20T01:47:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_default_stripe_size": "65536", 2024-04-20T01:47:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_fsck_on_mount": "false", 2024-04-20T01:47:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_fsck_on_mount_deep": "true", 2024-04-20T01:47:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_max_bytes": "67108864", 2024-04-20T01:47:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_max_ops": "512", 2024-04-20T01:47:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_nid_prealloc": "1024", 2024-04-20T01:47:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_onode_map_size": "1024", 2024-04-20T01:47:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-04-20T01:47:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_sync_submit_transaction": "false", 2024-04-20T01:47:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "kstore_sync_transaction": "false", 2024-04-20T01:47:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "leveldb_block_size": "65536", 2024-04-20T01:47:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "leveldb_bloom_size": "0", 2024-04-20T01:47:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "leveldb_cache_size": "536870912", 2024-04-20T01:47:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "leveldb_compact_on_mount": "false", 2024-04-20T01:47:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "leveldb_compression": "false", 2024-04-20T01:47:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "leveldb_log": "", 2024-04-20T01:47:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "leveldb_log_to_ceph_log": "true", 2024-04-20T01:47:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "leveldb_max_open_files": "0", 2024-04-20T01:47:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "leveldb_paranoid": "false", 2024-04-20T01:47:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "leveldb_write_buffer_size": "33554432", 2024-04-20T01:47:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "librados_thread_count": "2", 2024-04-20T01:47:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "lockdep": "false", 2024-04-20T01:47:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "lockdep_force_backtrace": "false", 2024-04-20T01:47:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "log_coarse_timestamps": "true", 2024-04-20T01:47:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-04-20T01:47:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: "log_flush_on_exit": "false", 2024-04-20T01:47:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: "log_graylog_host": "127.0.0.1", 2024-04-20T01:47:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: "log_graylog_port": "12201", 2024-04-20T01:47:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: "log_max_new": "1000", 2024-04-20T01:47:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: "log_max_recent": "10000", 2024-04-20T01:47:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: "log_stderr_prefix": "", 2024-04-20T01:47:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: "log_stop_at_utilization": "0.970000", 2024-04-20T01:47:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": "true", 2024-04-20T01:47:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_graylog": "false", 2024-04-20T01:47:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_journald": "false", 2024-04-20T01:47:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_stderr": "false", 2024-04-20T01:47:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_syslog": "false", 2024-04-20T01:47:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: "max_rotating_auth_attempts": "10", 2024-04-20T01:47:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-04-20T01:47:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_action_on_write_error": "1", 2024-04-20T01:47:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_alternate_name_max": "8192", 2024-04-20T01:47:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_asio_thread_count": "2", 2024-04-20T01:47:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_export_pin": "true", 2024-04-20T01:47:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_fragment_dirs": "true", 2024-04-20T01:47:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-04-20T01:47:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_fragment_interval": "5", 2024-04-20T01:47:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_fragment_size_max": "100000", 2024-04-20T01:47:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_idle_threshold": "0.000000", 2024-04-20T01:47:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_interval": "10", 2024-04-20T01:47:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_max": "-1", 2024-04-20T01:47:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_max_until": "-1", 2024-04-20T01:47:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_merge_size": "50", 2024-04-20T01:47:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_midchunk": "0.300000", 2024-04-20T01:47:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_min_rebalance": "0.100000", 2024-04-20T01:47:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_min_start": "0.200000", 2024-04-20T01:47:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_minchunk": "0.001000", 2024-04-20T01:47:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_mode": "0", 2024-04-20T01:47:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_need_max": "1.200000", 2024-04-20T01:47:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_need_min": "0.800000", 2024-04-20T01:47:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-04-20T01:47:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_sample_interval": "3.000000", 2024-04-20T01:47:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_split_bits": "3", 2024-04-20T01:47:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_split_rd": "25000.000000", 2024-04-20T01:47:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_split_size": "10000", 2024-04-20T01:47:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_split_wr": "10000.000000", 2024-04-20T01:47:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_target_decay": "10.000000", 2024-04-20T01:47:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-04-20T01:47:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_beacon_grace": "15.000000", 2024-04-20T01:47:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_beacon_interval": "4.000000", 2024-04-20T01:47:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_beacon_mon_down_grace": "60", 2024-04-20T01:47:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache_memory_limit": "4294967296", 2024-04-20T01:47:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache_mid": "0.700000", 2024-04-20T01:47:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache_release_free_interval": "10", 2024-04-20T01:47:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache_reservation": "0.050000", 2024-04-20T01:47:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-04-20T01:47:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache_trim_interval": "1", 2024-04-20T01:47:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache_trim_threshold": "262144", 2024-04-20T01:47:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-04-20T01:47:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-04-20T01:47:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_client_delegate_inos_pct": "50", 2024-04-20T01:47:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_client_prealloc_inos": "1000", 2024-04-20T01:47:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-04-20T01:47:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_connect_bootstrapping": "false", 2024-04-20T01:47:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_damage_table_max_entries": "10000", 2024-04-20T01:47:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-04-20T01:47:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_debug_auth_pins": "false", 2024-04-20T01:47:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_debug_frag": "false", 2024-04-20T01:47:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_debug_scatterstat": "false", 2024-04-20T01:47:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_debug_subtrees": "false", 2024-04-20T01:47:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_decay_halflife": "5.000000", 2024-04-20T01:47:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_default_dir_hash": "2", 2024-04-20T01:47:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_defer_session_stale": "true", 2024-04-20T01:47:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_deny_all_reconnect": "false", 2024-04-20T01:47:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_dir_keys_per_op": "16384", 2024-04-20T01:47:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_dir_max_commit_size": "10", 2024-04-20T01:47:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_dir_max_entries": "0", 2024-04-20T01:47:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_dir_prefetch": "true", 2024-04-20T01:47:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_dirstat_min_interval": "1.000000", 2024-04-20T01:47:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_dump_cache_after_rejoin": "false", 2024-04-20T01:47:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_dump_cache_on_map": "false", 2024-04-20T01:47:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_dump_cache_threshold_file": "0", 2024-04-20T01:47:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-04-20T01:47:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_early_reply": "true", 2024-04-20T01:47:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_enable_op_tracker": "true", 2024-04-20T01:47:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_enforce_unique_name": "true", 2024-04-20T01:47:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_export_ephemeral_distributed": "true", 2024-04-20T01:47:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-04-20T01:47:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_export_ephemeral_random": "true", 2024-04-20T01:47:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-04-20T01:47:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-04-20T01:47:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_forward_all_requests_to_auth": "false", 2024-04-20T01:47:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-04-20T01:47:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-04-20T01:47:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-04-20T01:47:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-04-20T01:47:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_health_cache_threshold": "1.500000", 2024-04-20T01:47:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_health_summarize_threshold": "10", 2024-04-20T01:47:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_heartbeat_grace": "15.000000", 2024-04-20T01:47:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_heartbeat_reset_grace": "1000", 2024-04-20T01:47:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_inject_health_dummy": "false", 2024-04-20T01:47:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-04-20T01:47:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_inject_migrator_session_race": "false", 2024-04-20T01:47:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-04-20T01:47:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-04-20T01:47:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-04-20T01:47:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_join_fs": "", 2024-04-20T01:47:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_journal_format": "1", 2024-04-20T01:47:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_create_at": "0", 2024-04-20T01:47:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_export_at": "0", 2024-04-20T01:47:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_import_at": "0", 2024-04-20T01:47:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_journal_at": "0", 2024-04-20T01:47:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_journal_expire_at": "0", 2024-04-20T01:47:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_journal_replay_at": "0", 2024-04-20T01:47:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_link_at": "0", 2024-04-20T01:47:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_mdstable_at": "0", 2024-04-20T01:47:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_openc_at": "0", 2024-04-20T01:47:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_rename_at": "0", 2024-04-20T01:47:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-04-20T01:47:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log_events_per_segment": "1024", 2024-04-20T01:47:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log_max_events": "-1", 2024-04-20T01:47:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log_max_segments": "128", 2024-04-20T01:47:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log_pause": "false", 2024-04-20T01:47:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log_segment_size": "0", 2024-04-20T01:47:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log_skip_corrupt_events": "false", 2024-04-20T01:47:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log_warn_factor": "2.000000", 2024-04-20T01:47:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_max_caps_per_client": "1048576", 2024-04-20T01:47:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_max_completed_flushes": "100000", 2024-04-20T01:47:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_max_completed_requests": "100000", 2024-04-20T01:47:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_max_export_size": "20971520", 2024-04-20T01:47:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_max_file_recover": "32", 2024-04-20T01:47:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_max_purge_files": "64", 2024-04-20T01:47:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_max_purge_ops": "8192", 2024-04-20T01:47:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-04-20T01:47:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-04-20T01:47:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_max_snaps_per_dir": "100", 2024-04-20T01:47:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_metrics_update_interval": "2", 2024-04-20T01:47:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_min_caps_per_client": "100", 2024-04-20T01:47:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_min_caps_working_set": "10000", 2024-04-20T01:47:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-04-20T01:47:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_numa_node": "-1", 2024-04-20T01:47:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-04-20T01:47:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_op_complaint_time": "30.000000", 2024-04-20T01:47:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_op_history_duration": "600", 2024-04-20T01:47:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_op_history_size": "20", 2024-04-20T01:47:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_op_history_slow_op_size": "20", 2024-04-20T01:47:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_op_history_slow_op_threshold": "10", 2024-04-20T01:47:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_op_log_threshold": "5", 2024-04-20T01:47:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_ping_grace": "15", 2024-04-20T01:47:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_ping_interval": "5", 2024-04-20T01:47:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-04-20T01:47:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-04-20T01:47:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_recall_max_caps": "30000", 2024-04-20T01:47:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-04-20T01:47:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_recall_max_decay_threshold": "131072", 2024-04-20T01:47:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-04-20T01:47:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_recall_warning_threshold": "262144", 2024-04-20T01:47:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_reconnect_timeout": "45.000000", 2024-04-20T01:47:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_replay_interval": "1.000000", 2024-04-20T01:47:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-04-20T01:47:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-04-20T01:47:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_root_ino_gid": "0", 2024-04-20T01:47:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_root_ino_uid": "0", 2024-04-20T01:47:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-04-20T01:47:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_session_blocklist_on_evict": "true", 2024-04-20T01:47:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_session_blocklist_on_timeout": "true", 2024-04-20T01:47:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-04-20T01:47:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-04-20T01:47:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-04-20T01:47:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-04-20T01:47:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-04-20T01:47:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_session_metadata_threshold": "16777216", 2024-04-20T01:47:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-04-20T01:47:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_shutdown_check": "0", 2024-04-20T01:47:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_skip_ino": "0", 2024-04-20T01:47:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sleep_rank_change": "0.000000", 2024-04-20T01:47:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_snap_max_uid": "4294967294", 2024-04-20T01:47:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_snap_min_uid": "0", 2024-04-20T01:47:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_snap_rstat": "false", 2024-04-20T01:47:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_standby_replay_damaged": "false", 2024-04-20T01:47:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_symlink_recovery": "true", 2024-04-20T01:47:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_task_status_update_interval": "2.000000", 2024-04-20T01:47:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_thrash_exports": "0", 2024-04-20T01:47:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_thrash_fragments": "0", 2024-04-20T01:47:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_tick_interval": "5.000000", 2024-04-20T01:47:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_valgrind_exit": "false", 2024-04-20T01:47:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_verify_backtrace": "1", 2024-04-20T01:47:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_verify_scatter": "false", 2024-04-20T01:47:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_wipe_ino_prealloc": "false", 2024-04-20T01:47:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_wipe_sessions": "false", 2024-04-20T01:47:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool_debug": "false", 2024-04-20T01:47:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "memstore_debug_omit_block_device_write": "false", 2024-04-20T01:47:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "memstore_device_bytes": "1073741824", 2024-04-20T01:47:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "memstore_page_set": "false", 2024-04-20T01:47:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "memstore_page_size": "65536", 2024-04-20T01:47:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_client_bytes": "134217728", 2024-04-20T01:47:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_client_messages": "512", 2024-04-20T01:47:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-04-20T01:47:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_connect_retry_interval": "1.000000", 2024-04-20T01:47:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-04-20T01:47:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-04-20T01:47:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_disabled_modules": "", 2024-04-20T01:47:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-04-20T01:47:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_max_pg_num_change": "128", 2024-04-20T01:47:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_mds_bytes": "134217728", 2024-04-20T01:47:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_mds_messages": "128", 2024-04-20T01:47:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-04-20T01:47:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_mon_bytes": "134217728", 2024-04-20T01:47:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_mon_messages": "128", 2024-04-20T01:47:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_osd_bytes": "536870912", 2024-04-20T01:47:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_osd_messages": "8192", 2024-04-20T01:47:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_pool": "true", 2024-04-20T01:47:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_service_beacon_grace": "60.000000", 2024-04-20T01:47:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_standby_modules": "true", 2024-04-20T01:47:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_stats_period": "5", 2024-04-20T01:47:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_stats_threshold": "5", 2024-04-20T01:47:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_tick_period": "2", 2024-04-20T01:47:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-04-20T01:47:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_accept_timeout_factor": "2.000000", 2024-04-20T01:47:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_allow_pool_delete": "true", 2024-04-20T01:47:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_allow_pool_size_one": "true", 2024-04-20T01:47:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_auth_validate_all_caps": "true", 2024-04-20T01:47:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-04-20T01:47:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-04-20T01:47:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_bytes": "104857600", 2024-04-20T01:47:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_directed_command_retry": "5", 2024-04-20T01:47:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_hunt_interval": "3.000000", 2024-04-20T01:47:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-04-20T01:47:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-04-20T01:47:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-04-20T01:47:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_hunt_parallel": "3", 2024-04-20T01:47:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_log_interval": "1.000000", 2024-04-20T01:47:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-04-20T01:47:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_ping_interval": "10.000000", 2024-04-20T01:47:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_client_ping_timeout": "30.000000", 2024-04-20T01:47:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_clock_drift_allowed": "1.000000", 2024-04-20T01:47:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-04-20T01:47:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-04-20T01:47:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_file_level": "debug", 2024-04-20T01:47:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_to_file": "true", 2024-04-20T01:47:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_to_graylog": "false", 2024-04-20T01:47:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-04-20T01:47:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-04-20T01:47:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_to_journald": "false", 2024-04-20T01:47:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_to_stderr": "false", 2024-04-20T01:47:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-04-20T01:47:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-04-20T01:47:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-04-20T01:47:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_compact_on_bootstrap": "false", 2024-04-20T01:47:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_compact_on_start": "false", 2024-04-20T01:47:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_compact_on_trim": "true", 2024-04-20T01:47:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_con_tracker_persist_interval": "10", 2024-04-20T01:47:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_con_tracker_score_halflife": "43200", 2024-04-20T01:47:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_config_key_max_entry_size": "65536", 2024-04-20T01:47:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_cpu_threads": "4", 2024-04-20T01:47:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_crush_min_required_version": "hammer", 2024-04-20T01:47:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_daemon_bytes": "419430400", 2024-04-20T01:47:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-04-20T01:47:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_data_avail_crit": "5", 2024-04-20T01:47:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_data_avail_warn": "5", 2024-04-20T01:47:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_data_size_warn": "16106127360", 2024-04-20T01:47:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_block_osdmap_trim": "false", 2024-04-20T01:47:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-04-20T01:47:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_dump_json": "false", 2024-04-20T01:47:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-04-20T01:47:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_dump_transactions": "false", 2024-04-20T01:47:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_extra_checks": "false", 2024-04-20T01:47:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-04-20T01:47:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-04-20T01:47:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_no_require_quincy": "false", 2024-04-20T01:47:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_no_require_reef": "false", 2024-04-20T01:47:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-04-20T01:47:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_delta_reset_interval": "10.000000", 2024-04-20T01:47:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_dns_srv_name": "ceph-mon", 2024-04-20T01:47:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_down_mkfs_grace": "120", 2024-04-20T01:47:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_election_default_strategy": "1", 2024-04-20T01:47:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_election_timeout": "5.000000", 2024-04-20T01:47:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-04-20T01:47:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_elector_ping_divisor": "2", 2024-04-20T01:47:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_elector_ping_timeout": "2.000000", 2024-04-20T01:47:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_enable_op_tracker": "true", 2024-04-20T01:47:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_fake_pool_delete": "false", 2024-04-20T01:47:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_force_quorum_join": "false", 2024-04-20T01:47:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_globalid_prealloc": "10000", 2024-04-20T01:47:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_health_detail_to_clog": "false", 2024-04-20T01:47:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_health_log_update_period": "5", 2024-04-20T01:47:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_health_max_detail": "50", 2024-04-20T01:47:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_health_to_clog": "true", 2024-04-20T01:47:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_health_to_clog_interval": "600", 2024-04-20T01:47:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-04-20T01:47:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_host": "172.21.15.44,172.21.15.121,172.21.15.123", 2024-04-20T01:47:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_host_override": "", 2024-04-20T01:47:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_initial_members": "", 2024-04-20T01:47:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-04-20T01:47:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-04-20T01:47:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-04-20T01:47:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-04-20T01:47:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_keyvaluedb": "rocksdb", 2024-04-20T01:47:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_lease": "5.000000", 2024-04-20T01:47:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-04-20T01:47:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-04-20T01:47:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_log_full_interval": "50", 2024-04-20T01:47:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_log_max": "10000", 2024-04-20T01:47:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_log_max_summary": "50", 2024-04-20T01:47:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_max_log_entries_per_event": "4096", 2024-04-20T01:47:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_max_log_epochs": "500", 2024-04-20T01:47:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_max_mdsmap_epochs": "500", 2024-04-20T01:47:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_max_mgrmap_epochs": "500", 2024-04-20T01:47:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_max_osd": "10000", 2024-04-20T01:47:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_max_pg_per_osd": "10000", 2024-04-20T01:47:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_max_pool_pg_num": "65536", 2024-04-20T01:47:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-04-20T01:47:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-04-20T01:47:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_mds_force_trim_to": "0", 2024-04-20T01:47:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_mds_skip_sanity": "false", 2024-04-20T01:47:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_memory_autotune": "true", 2024-04-20T01:47:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_memory_target": "2147483648", 2024-04-20T01:47:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_mgr_beacon_grace": "30", 2024-04-20T01:47:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-04-20T01:47:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_mgr_digest_period": "5", 2024-04-20T01:47:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_mgr_inactive_grace": "60", 2024-04-20T01:47:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_mgr_mkfs_grace": "240", 2024-04-20T01:47:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-04-20T01:47:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_min_osdmap_epochs": "500", 2024-04-20T01:47:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_op_complaint_time": "30", 2024-04-20T01:47:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_op_history_duration": "600", 2024-04-20T01:47:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_op_history_size": "20", 2024-04-20T01:47:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_op_history_slow_op_size": "20", 2024-04-20T01:47:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_op_history_slow_op_threshold": "10", 2024-04-20T01:47:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_op_log_threshold": "5", 2024-04-20T01:47:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-04-20T01:47:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-04-20T01:47:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-04-20T01:47:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_auto_mark_in": "false", 2024-04-20T01:47:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_auto_mark_new_in": "true", 2024-04-20T01:47:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-04-20T01:47:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-04-20T01:47:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_cache_size": "500", 2024-04-20T01:47:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_cache_size_min": "134217728", 2024-04-20T01:47:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_crush_smoke_test": "true", 2024-04-20T01:47:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_destroyed_out_interval": "600", 2024-04-20T01:47:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_down_out_interval": "600", 2024-04-20T01:47:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-04-20T01:47:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-04-20T01:47:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_force_trim_to": "0", 2024-04-20T01:47:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_full_ratio": "0.950000", 2024-04-20T01:47:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-04-20T01:47:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_laggy_halflife": "3600", 2024-04-20T01:47:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_laggy_max_interval": "300", 2024-04-20T01:47:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_laggy_weight": "0.300000", 2024-04-20T01:47:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-04-20T01:47:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_max_creating_pgs": "1024", 2024-04-20T01:47:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_max_initial_pgs": "1024", 2024-04-20T01:47:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_min_down_reporters": "2", 2024-04-20T01:47:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-04-20T01:47:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-04-20T01:47:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-04-20T01:47:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_prime_pg_temp": "true", 2024-04-20T01:47:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-04-20T01:47:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-04-20T01:47:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_report_timeout": "900", 2024-04-20T01:47:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-04-20T01:47:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-04-20T01:47:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_warn_num_repaired": "10", 2024-04-20T01:47:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osd_warn_op_age": "32.000000", 2024-04-20T01:47:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-04-20T01:47:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osdmap_full_prune_interval": "10", 2024-04-20T01:47:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osdmap_full_prune_min": "10000", 2024-04-20T01:47:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-04-20T01:47:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-04-20T01:47:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_pg_stuck_threshold": "60", 2024-04-20T01:47:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-04-20T01:47:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_pg_warn_min_objects": "10000", 2024-04-20T01:47:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_pg_warn_min_per_osd": "0", 2024-04-20T01:47:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-04-20T01:47:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_pool_quota_crit_threshold": "0", 2024-04-20T01:47:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_pool_quota_warn_threshold": "0", 2024-04-20T01:47:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_probe_timeout": "2.000000", 2024-04-20T01:47:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_reweight_max_change": "0.050000", 2024-04-20T01:47:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_reweight_max_osds": "4", 2024-04-20T01:47:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-04-20T01:47:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-04-20T01:47:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-04-20T01:47:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-04-20T01:47:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-04-20T01:47:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_scrub_interval": "300", 2024-04-20T01:47:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_scrub_max_keys": "100", 2024-04-20T01:47:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_scrub_timeout": "300", 2024-04-20T01:47:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_session_timeout": "300", 2024-04-20T01:47:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_smart_report_timeout": "5", 2024-04-20T01:47:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_stat_smooth_intervals": "6", 2024-04-20T01:47:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-04-20T01:47:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_stretch_pool_min_size": "2", 2024-04-20T01:47:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_stretch_pool_size": "4", 2024-04-20T01:47:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-04-20T01:47:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_subscribe_interval": "86400.000000", 2024-04-20T01:47:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_sync_debug": "false", 2024-04-20T01:47:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_sync_max_payload_keys": "2000", 2024-04-20T01:47:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_sync_max_payload_size": "1048576", 2024-04-20T01:47:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_sync_provider_kill_at": "0", 2024-04-20T01:47:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_sync_requester_kill_at": "0", 2024-04-20T01:47:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_sync_timeout": "60.000000", 2024-04-20T01:47:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_target_pg_per_osd": "100", 2024-04-20T01:47:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_tick_interval": "5", 2024-04-20T01:47:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_timecheck_interval": "300.000000", 2024-04-20T01:47:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-04-20T01:47:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_use_min_delay_socket": "false", 2024-04-20T01:47:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_older_version_delay": "604800", 2024-04-20T01:47:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-04-20T01:47:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-04-20T01:47:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-04-20T01:47:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_filestore_osds": "false", 2024-04-20T01:47:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-04-20T01:47:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-04-20T01:47:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-04-20T01:47:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_misplaced": "false", 2024-04-20T01:47:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-04-20T01:47:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_older_version": "true", 2024-04-20T01:47:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-04-20T01:47:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_pool_no_app": "true", 2024-04-20T01:47:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-04-20T01:47:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-04-20T01:47:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-04-20T01:47:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-04-20T01:47:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_on_too_few_osds": "false", 2024-04-20T01:47:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-04-20T01:47:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-04-20T01:47:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: "monmap": "", 2024-04-20T01:47:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-04-20T01:47:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-04-20T01:47:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-04-20T01:47:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-04-20T01:47:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-04-20T01:47:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-04-20T01:47:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: "motr_tracing_enabled": "false", 2024-04-20T01:47:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_op_threads": "3", 2024-04-20T01:47:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_buffer_size": "131072", 2024-04-20T01:47:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_cm": "false", 2024-04-20T01:47:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_device_name": "", 2024-04-20T01:47:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_dscp": "96", 2024-04-20T01:47:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-04-20T01:47:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_gid_idx": "0", 2024-04-20T01:47:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_local_gid": "", 2024-04-20T01:47:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_polling_us": "1000", 2024-04-20T01:47:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_port_num": "1", 2024-04-20T01:47:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-04-20T01:47:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-04-20T01:47:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_roce_ver": "1", 2024-04-20T01:47:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_send_buffers": "1024", 2024-04-20T01:47:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_sl": "3", 2024-04-20T01:47:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_support_srq": "true", 2024-04-20T01:47:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_rdma_type": "ib", 2024-04-20T01:47:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_async_reap_threshold": "5", 2024-04-20T01:47:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_bind_before_connect": "false", 2024-04-20T01:47:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_bind_ipv4": "true", 2024-04-20T01:47:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_bind_ipv6": "false", 2024-04-20T01:47:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_bind_msgr1": "true", 2024-04-20T01:47:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_bind_msgr2": "true", 2024-04-20T01:47:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_bind_port_max": "7568", 2024-04-20T01:47:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_bind_port_min": "6800", 2024-04-20T01:47:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_bind_prefer_ipv4": "false", 2024-04-20T01:47:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_bind_retry_count": "3", 2024-04-20T01:47:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_bind_retry_delay": "5", 2024-04-20T01:47:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_blackhole_client": "false", 2024-04-20T01:47:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_blackhole_mds": "false", 2024-04-20T01:47:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_blackhole_mgr": "false", 2024-04-20T01:47:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_blackhole_mon": "false", 2024-04-20T01:47:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_blackhole_osd": "false", 2024-04-20T01:47:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_client_mode": "crc secure", 2024-04-20T01:47:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_cluster_mode": "crc secure", 2024-04-20T01:47:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_cluster_type": "", 2024-04-20T01:47:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_compress_secure": "false", 2024-04-20T01:47:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_connection_idle_timeout": "900", 2024-04-20T01:47:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_connection_ready_timeout": "10", 2024-04-20T01:47:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_crc_data": "true", 2024-04-20T01:47:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_crc_header": "true", 2024-04-20T01:47:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_die_on_bad_msg": "false", 2024-04-20T01:47:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_die_on_bug": "true", 2024-04-20T01:47:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_die_on_old_message": "true", 2024-04-20T01:47:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_die_on_skipped_message": "false", 2024-04-20T01:47:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_die_on_unhandled_msg": "false", 2024-04-20T01:47:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-04-20T01:47:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_coremask": "0xF", 2024-04-20T01:47:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-04-20T01:47:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_devs_allowlist": "", 2024-04-20T01:47:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_enable_tso": "true", 2024-04-20T01:47:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-04-20T01:47:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-04-20T01:47:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_hugepages": "", 2024-04-20T01:47:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_hw_flow_control": "true", 2024-04-20T01:47:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-04-20T01:47:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_lro": "true", 2024-04-20T01:47:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_memory_channel": "4", 2024-04-20T01:47:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-04-20T01:47:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_pmd": "", 2024-04-20T01:47:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_port_id": "0", 2024-04-20T01:47:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-04-20T01:47:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dump_corrupt_message_level": "1", 2024-04-20T01:47:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_dump_on_send": "false", 2024-04-20T01:47:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_initial_backoff": "0.200000", 2024-04-20T01:47:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_inject_delay_max": "1.000000", 2024-04-20T01:47:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_inject_delay_probability": "0.000000", 2024-04-20T01:47:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_inject_delay_type": "", 2024-04-20T01:47:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_inject_internal_delays": "0.000000", 2024-04-20T01:47:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_inject_network_congestion": "0", 2024-04-20T01:47:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_inject_socket_failures": "5000", 2024-04-20T01:47:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_learn_addr_from_peer": "true", 2024-04-20T01:47:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_max_accept_failures": "4", 2024-04-20T01:47:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_max_backoff": "15.000000", 2024-04-20T01:47:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_mon_client_mode": "secure crc", 2024-04-20T01:47:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_mon_cluster_mode": "secure crc", 2024-04-20T01:47:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_mon_service_mode": "secure crc", 2024-04-20T01:47:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_osd_compress_min_size": "1024", 2024-04-20T01:47:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_osd_compress_mode": "none", 2024-04-20T01:47:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_osd_compression_algorithm": "snappy", 2024-04-20T01:47:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-04-20T01:47:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_pq_min_cost": "65536", 2024-04-20T01:47:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_public_type": "", 2024-04-20T01:47:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_service_mode": "crc secure", 2024-04-20T01:47:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_tcp_listen_backlog": "512", 2024-04-20T01:47:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_tcp_nodelay": "true", 2024-04-20T01:47:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-04-20T01:47:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_tcp_rcvbuf": "0", 2024-04-20T01:47:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "ms_type": "async+posix", 2024-04-20T01:47:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "no_config_file": "false", 2024-04-20T01:47:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter_completion_locks_per_session": "32", 2024-04-20T01:47:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter_debug_inject_relock_delay": "false", 2024-04-20T01:47:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter_inflight_op_bytes": "104857600", 2024-04-20T01:47:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter_inflight_ops": "1024", 2024-04-20T01:47:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter_inject_no_watch_ping": "false", 2024-04-20T01:47:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-04-20T01:47:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter_tick_interval": "5.000000", 2024-04-20T01:47:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter_timeout": "10.000000", 2024-04-20T01:47:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: "objectstore_blackhole": "false", 2024-04-20T01:47:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: "openssl_engine_opts": "", 2024-04-20T01:47:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_agent_delay_time": "5.000000", 2024-04-20T01:47:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_agent_hist_halflife": "1000", 2024-04-20T01:47:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_agent_max_low_ops": "2", 2024-04-20T01:47:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_agent_max_ops": "4", 2024-04-20T01:47:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-04-20T01:47:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_agent_quantize_effort": "0.100000", 2024-04-20T01:47:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_agent_slop": "0.020000", 2024-04-20T01:47:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-04-20T01:47:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_allow_recovery_below_min_size": "true", 2024-04-20T01:47:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_asio_thread_count": "2", 2024-04-20T01:47:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_async_recovery_min_cost": "100", 2024-04-20T01:47:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_auto_mark_unfound_lost": "false", 2024-04-20T01:47:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_backfill_retry_interval": "30.000000", 2024-04-20T01:47:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_backfill_scan_max": "512", 2024-04-20T01:47:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_backfill_scan_min": "64", 2024-04-20T01:47:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_backoff_on_degraded": "false", 2024-04-20T01:47:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_backoff_on_peering": "false", 2024-04-20T01:47:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_backoff_on_unfound": "true", 2024-04-20T01:47:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_beacon_report_interval": "300", 2024-04-20T01:47:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bench_duration": "30", 2024-04-20T01:47:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-04-20T01:47:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bench_max_block_size": "67108864", 2024-04-20T01:47:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bench_small_size_max_iops": "100", 2024-04-20T01:47:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_blkin_trace_all": "false", 2024-04-20T01:47:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_blocked_scrub_grace_period": "120", 2024-04-20T01:47:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-04-20T01:47:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-04-20T01:47:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-04-20T01:47:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_check_for_log_corruption": "false", 2024-04-20T01:47:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-04-20T01:47:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-20T01:47:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-04-20T01:47:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-20T01:47:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_class_update_on_start": "true", 2024-04-20T01:47:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_client_message_cap": "256", 2024-04-20T01:47:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_client_message_size_cap": "524288000", 2024-04-20T01:47:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_client_op_priority": "63", 2024-04-20T01:47:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_client_watch_timeout": "30", 2024-04-20T01:47:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_command_max_records": "256", 2024-04-20T01:47:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_compact_on_start": "false", 2024-04-20T01:47:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-04-20T01:47:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_crush_chooseleaf_type": "0", 2024-04-20T01:47:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_crush_initial_weight": "-1.000000", 2024-04-20T01:47:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_crush_update_on_start": "true", 2024-04-20T01:47:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_crush_update_weight_set": "true", 2024-04-20T01:47:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-04-20T01:47:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-04-20T01:47:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-04-20T01:47:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_drop_ping_duration": "0", 2024-04-20T01:47:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-04-20T01:47:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_feed_pullee": "-1", 2024-04-20T01:47:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-04-20T01:47:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-04-20T01:47:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-04-20T01:47:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_misdirected_ops": "false", 2024-04-20T01:47:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_no_acting_change": "false", 2024-04-20T01:47:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_no_purge_strays": "false", 2024-04-20T01:47:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_op_order": "false", 2024-04-20T01:47:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_pg_log_writeout": "false", 2024-04-20T01:47:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_pretend_recovery_active": "false", 2024-04-20T01:47:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-04-20T01:47:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-04-20T01:47:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_shutdown": "false", 2024-04-20T01:47:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-04-20T01:47:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-04-20T01:47:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_verify_cached_snaps": "false", 2024-04-20T01:47:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_verify_missing_on_start": "false", 2024-04-20T01:47:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_verify_snaps": "false", 2024-04-20T01:47:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-04-20T01:47:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-04-20T01:47:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_deep_scrub_keys": "1024", 2024-04-20T01:47:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-04-20T01:47:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-04-20T01:47:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-04-20T01:47:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_deep_scrub_stride": "524288", 2024-04-20T01:47:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-04-20T01:47:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_default_data_pool_replay_window": "5", 2024-04-20T01:47:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_default_notify_timeout": "30", 2024-04-20T01:47:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_delete_sleep": "0.000000", 2024-04-20T01:47:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-04-20T01:47:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-04-20T01:47:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-04-20T01:47:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_discard_disconnected_ops": "true", 2024-04-20T01:47:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_enable_op_tracker": "true", 2024-04-20T01:47:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-04-20T01:47:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_extblkdev_plugins": "vdo", 2024-04-20T01:47:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-04-20T01:47:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-04-20T01:47:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_fast_info": "true", 2024-04-20T01:47:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_fast_shutdown": "true", 2024-04-20T01:47:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-04-20T01:47:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_fast_shutdown_timeout": "15", 2024-04-20T01:47:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-04-20T01:47:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-04-20T01:47:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-04-20T01:47:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_function_tracing": "false", 2024-04-20T01:47:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_heartbeat_grace": "20", 2024-04-20T01:47:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_heartbeat_interval": "6", 2024-04-20T01:47:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-04-20T01:47:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_heartbeat_min_peers": "10", 2024-04-20T01:47:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_heartbeat_min_size": "2000", 2024-04-20T01:47:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_heartbeat_stale": "600", 2024-04-20T01:47:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-04-20T01:47:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_hit_set_max_size": "100000", 2024-04-20T01:47:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_hit_set_min_size": "1000", 2024-04-20T01:47:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-04-20T01:47:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-04-20T01:47:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-04-20T01:47:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-04-20T01:47:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-04-20T01:47:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_journal_flush_on_shutdown": "true", 2024-04-20T01:47:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_journal_size": "5120", 2024-04-20T01:47:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_kill_backfill_at": "0", 2024-04-20T01:47:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_loop_before_reset_tphandle": "64", 2024-04-20T01:47:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_map_cache_size": "50", 2024-04-20T01:47:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_map_dedup": "true", 2024-04-20T01:47:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_map_message_max": "40", 2024-04-20T01:47:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_map_message_max_bytes": "10485760", 2024-04-20T01:47:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_map_share_max_epochs": "40", 2024-04-20T01:47:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_attr_name_len": "100", 2024-04-20T01:47:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_attr_size": "0", 2024-04-20T01:47:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_backfills": "1", 2024-04-20T01:47:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_markdown_count": "5", 2024-04-20T01:47:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_markdown_period": "600", 2024-04-20T01:47:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_object_name_len": "2048", 2024-04-20T01:47:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_object_namespace_len": "256", 2024-04-20T01:47:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_object_size": "134217728", 2024-04-20T01:47:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-04-20T01:47:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_omap_entries_per_request": "1024", 2024-04-20T01:47:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_pg_blocked_by": "16", 2024-04-20T01:47:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_pg_log_entries": "10000", 2024-04-20T01:47:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-04-20T01:47:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_pgls": "1024", 2024-04-20T01:47:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_push_cost": "8388608", 2024-04-20T01:47:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_push_objects": "10", 2024-04-20T01:47:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_scrubs": "3", 2024-04-20T01:47:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-04-20T01:47:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_trimming_pgs": "2", 2024-04-20T01:47:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_write_op_reply_len": "64", 2024-04-20T01:47:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_max_write_size": "90", 2024-04-20T01:47:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-04-20T01:47:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-04-20T01:47:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-04-20T01:47:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-04-20T01:47:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-04-20T01:47:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-04-20T01:47:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-04-20T01:47:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_override_recovery_settings": "false", 2024-04-20T01:47:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_profile": "balanced", 2024-04-20T01:47:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-04-20T01:47:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-04-20T01:47:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-04-20T01:47:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-04-20T01:47:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-04-20T01:47:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-04-20T01:47:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-04-20T01:47:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-04-20T01:47:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-04-20T01:47:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-04-20T01:47:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mclock_skip_benchmark": "false", 2024-04-20T01:47:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_memory_base": "805306368", 2024-04-20T01:47:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_memory_cache_min": "134217728", 2024-04-20T01:47:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-04-20T01:47:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-04-20T01:47:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_memory_target": "4294967296", 2024-04-20T01:47:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_memory_target_autotune": "false", 2024-04-20T01:47:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-04-20T01:47:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_min_pg_log_entries": "250", 2024-04-20T01:47:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_min_recovery_priority": "0", 2024-04-20T01:47:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mon_heartbeat_interval": "30", 2024-04-20T01:47:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-04-20T01:47:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mon_report_interval": "5", 2024-04-20T01:47:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mon_report_max_in_flight": "2", 2024-04-20T01:47:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-04-20T01:47:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_num_cache_shards": "32", 2024-04-20T01:47:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_num_op_tracker_shard": "32", 2024-04-20T01:47:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_numa_auto_affinity": "true", 2024-04-20T01:47:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_numa_node": "-1", 2024-04-20T01:47:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_numa_prefer_iface": "true", 2024-04-20T01:47:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_object_clean_region_max_num_intervals": "1000", 2024-04-20T01:47:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_objecter_finishers": "1", 2024-04-20T01:47:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_objectstore": "bluestore", 2024-04-20T01:47:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_objectstore_fuse": "false", 2024-04-20T01:47:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_objectstore_tracing": "false", 2024-04-20T01:47:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_complaint_time": "30.000000", 2024-04-20T01:47:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_history_duration": "600", 2024-04-20T01:47:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_history_size": "20", 2024-04-20T01:47:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_history_slow_op_size": "20", 2024-04-20T01:47:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-04-20T01:47:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_log_threshold": "5", 2024-04-20T01:47:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_num_shards": "0", 2024-04-20T01:47:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_num_shards_hdd": "5", 2024-04-20T01:47:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_num_shards_ssd": "8", 2024-04-20T01:47:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_num_threads_per_shard": "0", 2024-04-20T01:47:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-04-20T01:47:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-04-20T01:47:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-04-20T01:47:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_pq_min_cost": "65536", 2024-04-20T01:47:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_queue": "mclock_scheduler", 2024-04-20T01:47:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_queue_cut_off": "high", 2024-04-20T01:47:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_thread_suicide_timeout": "150", 2024-04-20T01:47:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_op_thread_timeout": "15", 2024-04-20T01:47:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_open_classes_on_start": "true", 2024-04-20T01:47:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_os_flags": "0", 2024-04-20T01:47:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_peering_op_priority": "255", 2024-04-20T01:47:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pg_delete_cost": "1048576", 2024-04-20T01:47:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pg_delete_priority": "5", 2024-04-20T01:47:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-04-20T01:47:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-04-20T01:47:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pg_log_dups_tracked": "3000", 2024-04-20T01:47:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pg_log_trim_max": "10000", 2024-04-20T01:47:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pg_log_trim_min": "100", 2024-04-20T01:47:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-04-20T01:47:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pg_object_context_cache_count": "64", 2024-04-20T01:47:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pg_stat_report_interval_max": "500", 2024-04-20T01:47:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-04-20T01:47:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-04-20T01:47:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-04-20T01:47:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-04-20T01:47:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-04-20T01:47:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-04-20T01:47:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_crimson": "false", 2024-04-20T01:47:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_crush_rule": "-1", 2024-04-20T01:47:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_ec_fast_read": "false", 2024-04-20T01:47:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-04-20T01:47:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_flag_bulk": "false", 2024-04-20T01:47:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-04-20T01:47:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_flag_nodelete": "false", 2024-04-20T01:47:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-04-20T01:47:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-04-20T01:47:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_flags": "0", 2024-04-20T01:47:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-04-20T01:47:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_min_size": "0", 2024-04-20T01:47:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-04-20T01:47:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_pg_num": "32", 2024-04-20T01:47:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_pgp_num": "0", 2024-04-20T01:47:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-04-20T01:47:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_size": "2", 2024-04-20T01:47:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_default_type": "replicated", 2024-04-20T01:47:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-04-20T01:47:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pool_use_gmt_hitset": "true", 2024-04-20T01:47:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_push_per_object_cost": "1000", 2024-04-20T01:47:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_read_ec_check_for_errors": "false", 2024-04-20T01:47:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recover_clone_overlap": "true", 2024-04-20T01:47:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recover_clone_overlap_limit": "10", 2024-04-20T01:47:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_cost": "20971520", 2024-04-20T01:47:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_delay_start": "0.000000", 2024-04-20T01:47:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_max_active": "0", 2024-04-20T01:47:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_max_active_hdd": "3", 2024-04-20T01:47:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_max_active_ssd": "10", 2024-04-20T01:47:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_max_chunk": "8388608", 2024-04-20T01:47:28.329 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-04-20T01:47:28.329 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_max_single_start": "1", 2024-04-20T01:47:28.329 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_op_priority": "3", 2024-04-20T01:47:28.329 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_op_warn_multiple": "16", 2024-04-20T01:47:28.329 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_priority": "5", 2024-04-20T01:47:28.329 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_retry_interval": "30.000000", 2024-04-20T01:47:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_sleep": "0.000000", 2024-04-20T01:47:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-04-20T01:47:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-04-20T01:47:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-04-20T01:47:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_repair_during_recovery": "false", 2024-04-20T01:47:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_requested_scrub_priority": "120", 2024-04-20T01:47:28.331 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-04-20T01:47:28.331 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_rollback_to_cluster_snap": "", 2024-04-20T01:47:28.331 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_auto_repair": "false", 2024-04-20T01:47:28.331 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-04-20T01:47:28.331 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-04-20T01:47:28.331 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_begin_hour": "0", 2024-04-20T01:47:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_begin_week_day": "0", 2024-04-20T01:47:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_chunk_max": "25", 2024-04-20T01:47:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_chunk_min": "5", 2024-04-20T01:47:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_cost": "52428800", 2024-04-20T01:47:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_during_recovery": "false", 2024-04-20T01:47:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_end_hour": "0", 2024-04-20T01:47:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_end_week_day": "0", 2024-04-20T01:47:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_event_cost": "4096", 2024-04-20T01:47:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-04-20T01:47:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-04-20T01:47:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_invalid_stats": "true", 2024-04-20T01:47:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_load_threshold": "0.500000", 2024-04-20T01:47:28.334 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_max_interval": "604800.000000", 2024-04-20T01:47:28.334 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_max_preemptions": "5", 2024-04-20T01:47:28.334 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_min_interval": "86400.000000", 2024-04-20T01:47:28.334 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_priority": "5", 2024-04-20T01:47:28.334 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_reservation_timeout": "5000", 2024-04-20T01:47:28.334 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_sleep": "0.000000", 2024-04-20T01:47:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_scrub_slow_reservation_response": "2200", 2024-04-20T01:47:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-04-20T01:47:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-04-20T01:47:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_shutdown_pgref_assert": "false", 2024-04-20T01:47:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_skip_data_digest": "false", 2024-04-20T01:47:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_smart_report_timeout": "5", 2024-04-20T01:47:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_snap_trim_cost": "1048576", 2024-04-20T01:47:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_snap_trim_priority": "5", 2024-04-20T01:47:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_snap_trim_sleep": "0.000000", 2024-04-20T01:47:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-04-20T01:47:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-04-20T01:47:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-04-20T01:47:28.337 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-04-20T01:47:28.337 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_stats_update_period_scrubbing": "15", 2024-04-20T01:47:28.337 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-04-20T01:47:28.337 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_target_transaction_size": "30", 2024-04-20T01:47:28.337 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-04-20T01:47:28.337 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-04-20T01:47:28.338 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-04-20T01:47:28.338 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-04-20T01:47:28.338 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-04-20T01:47:28.338 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-04-20T01:47:28.338 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-04-20T01:47:28.339 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tier_default_cache_mode": "writeback", 2024-04-20T01:47:28.339 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-04-20T01:47:28.339 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-04-20T01:47:28.339 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_tracing": "false", 2024-04-20T01:47:28.339 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_use_stale_snap": "false", 2024-04-20T01:47:28.339 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-04-20T01:47:28.340 INFO:teuthology.orchestra.run.smithi044.stdout: "osdc_blkin_trace_all": "false", 2024-04-20T01:47:28.340 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_kill_at": "0", 2024-04-20T01:47:28.340 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_max_join_drift": "10", 2024-04-20T01:47:28.340 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_min": "500", 2024-04-20T01:47:28.340 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_min_wait": "0.050000", 2024-04-20T01:47:28.340 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_propose_interval": "1.000000", 2024-04-20T01:47:28.341 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_service_trim_max": "500", 2024-04-20T01:47:28.341 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_service_trim_max_multiplier": "20", 2024-04-20T01:47:28.341 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_service_trim_min": "250", 2024-04-20T01:47:28.341 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_stash_full_interval": "25", 2024-04-20T01:47:28.341 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_trim_max": "500", 2024-04-20T01:47:28.341 INFO:teuthology.orchestra.run.smithi044.stdout: "paxos_trim_min": "250", 2024-04-20T01:47:28.342 INFO:teuthology.orchestra.run.smithi044.stdout: "perf": "true", 2024-04-20T01:47:28.342 INFO:teuthology.orchestra.run.smithi044.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-04-20T01:47:28.342 INFO:teuthology.orchestra.run.smithi044.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-04-20T01:47:28.342 INFO:teuthology.orchestra.run.smithi044.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-04-20T01:47:28.342 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "-", 2024-04-20T01:47:28.342 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrv": "", 2024-04-20T01:47:28.343 INFO:teuthology.orchestra.run.smithi044.stdout: "public_bind_addr": "-", 2024-04-20T01:47:28.343 INFO:teuthology.orchestra.run.smithi044.stdout: "public_network": "", 2024-04-20T01:47:28.343 INFO:teuthology.orchestra.run.smithi044.stdout: "public_network_interface": "", 2024-04-20T01:47:28.343 INFO:teuthology.orchestra.run.smithi044.stdout: "qat_compressor_enabled": "false", 2024-04-20T01:47:28.343 INFO:teuthology.orchestra.run.smithi044.stdout: "qat_compressor_session_max_number": "256", 2024-04-20T01:47:28.343 INFO:teuthology.orchestra.run.smithi044.stdout: "rados_mon_op_timeout": "0", 2024-04-20T01:47:28.344 INFO:teuthology.orchestra.run.smithi044.stdout: "rados_osd_op_timeout": "0", 2024-04-20T01:47:28.344 INFO:teuthology.orchestra.run.smithi044.stdout: "rados_tracing": "false", 2024-04-20T01:47:28.344 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_atime_update_interval": "60", 2024-04-20T01:47:28.344 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-04-20T01:47:28.344 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_balance_parent_reads": "false", 2024-04-20T01:47:28.344 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_balance_snap_reads": "false", 2024-04-20T01:47:28.345 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_blkin_trace_all": "false", 2024-04-20T01:47:28.345 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_blocklist_expire_seconds": "0", 2024-04-20T01:47:28.345 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_blocklist_on_break_lock": "true", 2024-04-20T01:47:28.345 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_cache": "true", 2024-04-20T01:47:28.345 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_cache_block_writes_upfront": "false", 2024-04-20T01:47:28.345 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_cache_max_dirty": "25165824", 2024-04-20T01:47:28.346 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-04-20T01:47:28.346 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_cache_max_dirty_object": "0", 2024-04-20T01:47:28.346 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_cache_policy": "writearound", 2024-04-20T01:47:28.346 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_cache_size": "33554432", 2024-04-20T01:47:28.346 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_cache_target_dirty": "16777216", 2024-04-20T01:47:28.346 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-04-20T01:47:28.347 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_clone_copy_on_read": "false", 2024-04-20T01:47:28.347 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_compression_hint": "none", 2024-04-20T01:47:28.347 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_concurrent_management_ops": "10", 2024-04-20T01:47:28.347 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-04-20T01:47:28.347 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_default_clone_format": "auto", 2024-04-20T01:47:28.347 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_default_data_pool": "", 2024-04-20T01:47:28.348 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_default_features": "61", 2024-04-20T01:47:28.348 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_default_format": "2", 2024-04-20T01:47:28.348 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_default_map_options": "", 2024-04-20T01:47:28.348 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_default_order": "22", 2024-04-20T01:47:28.348 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_default_pool": "rbd", 2024-04-20T01:47:28.348 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-04-20T01:47:28.349 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_default_stripe_count": "0", 2024-04-20T01:47:28.349 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_default_stripe_unit": "0", 2024-04-20T01:47:28.349 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_disable_zero_copy_writes": "true", 2024-04-20T01:47:28.349 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_discard_granularity_bytes": "65536", 2024-04-20T01:47:28.349 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-04-20T01:47:28.349 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_enable_alloc_hint": "true", 2024-04-20T01:47:28.350 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-04-20T01:47:28.350 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_io_scheduler": "simple", 2024-04-20T01:47:28.350 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-04-20T01:47:28.350 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_commit_age": "5.000000", 2024-04-20T01:47:28.350 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-04-20T01:47:28.350 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-04-20T01:47:28.351 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-04-20T01:47:28.351 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-04-20T01:47:28.351 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_object_flush_interval": "0", 2024-04-20T01:47:28.351 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-04-20T01:47:28.351 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-04-20T01:47:28.351 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_order": "24", 2024-04-20T01:47:28.352 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_pool": "", 2024-04-20T01:47:28.352 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_journal_splay_width": "4", 2024-04-20T01:47:28.352 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_localize_parent_reads": "false", 2024-04-20T01:47:28.352 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_localize_snap_reads": "false", 2024-04-20T01:47:28.352 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-04-20T01:47:28.352 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-04-20T01:47:28.353 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-04-20T01:47:28.353 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-04-20T01:47:28.353 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-04-20T01:47:28.353 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-04-20T01:47:28.353 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_image_policy_type": "simple", 2024-04-20T01:47:28.353 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-04-20T01:47:28.354 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-04-20T01:47:28.354 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-04-20T01:47:28.354 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-04-20T01:47:28.354 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-04-20T01:47:28.354 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-04-20T01:47:28.354 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-04-20T01:47:28.355 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_memory_autotune": "true", 2024-04-20T01:47:28.355 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_memory_base": "805306368", 2024-04-20T01:47:28.355 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-04-20T01:47:28.355 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-04-20T01:47:28.355 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-04-20T01:47:28.355 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-04-20T01:47:28.356 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_memory_target": "4294967296", 2024-04-20T01:47:28.356 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-04-20T01:47:28.356 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-04-20T01:47:28.356 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-04-20T01:47:28.356 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirroring_delete_delay": "0", 2024-04-20T01:47:28.356 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-04-20T01:47:28.357 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirroring_replay_delay": "0", 2024-04-20T01:47:28.357 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-04-20T01:47:28.357 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-04-20T01:47:28.357 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_move_to_trash_on_remove": "false", 2024-04-20T01:47:28.357 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-04-20T01:47:28.358 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_mtime_update_interval": "60", 2024-04-20T01:47:28.358 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_non_blocking_aio": "true", 2024-04-20T01:47:28.358 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_op_thread_timeout": "60", 2024-04-20T01:47:28.358 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_op_threads": "1", 2024-04-20T01:47:28.358 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_parent_cache_enabled": "false", 2024-04-20T01:47:28.358 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_persistent_cache_mode": "disabled", 2024-04-20T01:47:28.359 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_persistent_cache_path": "/tmp", 2024-04-20T01:47:28.359 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_persistent_cache_size": "1073741824", 2024-04-20T01:47:28.359 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_plugins": "", 2024-04-20T01:47:28.359 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_bps_burst": "0", 2024-04-20T01:47:28.359 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-04-20T01:47:28.359 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_bps_limit": "0", 2024-04-20T01:47:28.360 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_exclude_ops": "0", 2024-04-20T01:47:28.360 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_iops_burst": "0", 2024-04-20T01:47:28.360 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-04-20T01:47:28.360 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_iops_limit": "0", 2024-04-20T01:47:28.360 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_read_bps_burst": "0", 2024-04-20T01:47:28.360 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-04-20T01:47:28.361 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_read_bps_limit": "0", 2024-04-20T01:47:28.361 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_read_iops_burst": "0", 2024-04-20T01:47:28.361 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-04-20T01:47:28.361 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_read_iops_limit": "0", 2024-04-20T01:47:28.361 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_schedule_tick_min": "50", 2024-04-20T01:47:28.361 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_write_bps_burst": "0", 2024-04-20T01:47:28.362 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-04-20T01:47:28.362 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_write_bps_limit": "0", 2024-04-20T01:47:28.362 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_write_iops_burst": "0", 2024-04-20T01:47:28.362 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-04-20T01:47:28.362 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_qos_write_iops_limit": "0", 2024-04-20T01:47:28.363 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_quiesce_notification_attempts": "10", 2024-04-20T01:47:28.363 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_read_from_replica_policy": "default", 2024-04-20T01:47:28.363 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-04-20T01:47:28.363 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_readahead_max_bytes": "524288", 2024-04-20T01:47:28.363 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_readahead_trigger_requests": "10", 2024-04-20T01:47:28.363 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_request_timed_out_seconds": "30", 2024-04-20T01:47:28.364 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_skip_partial_discard": "true", 2024-04-20T01:47:28.364 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-04-20T01:47:28.364 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_tracing": "false", 2024-04-20T01:47:28.364 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_validate_names": "true", 2024-04-20T01:47:28.364 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_validate_pool": "true", 2024-04-20T01:47:28.364 INFO:teuthology.orchestra.run.smithi044.stdout: "restapi_base_url": "", 2024-04-20T01:47:28.365 INFO:teuthology.orchestra.run.smithi044.stdout: "restapi_log_level": "", 2024-04-20T01:47:28.365 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_acl_grants_max_num": "100", 2024-04-20T01:47:28.365 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_admin_entry": "admin", 2024-04-20T01:47:28.365 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-04-20T01:47:28.365 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_backend_store": "rados", 2024-04-20T01:47:28.365 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_barbican_url": "", 2024-04-20T01:47:28.366 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_beast_enable_async": "true", 2024-04-20T01:47:28.366 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-04-20T01:47:28.366 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-04-20T01:47:28.366 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_bucket_index_max_aio": "128", 2024-04-20T01:47:28.366 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-04-20T01:47:28.366 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-04-20T01:47:28.367 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_bucket_quota_ttl": "600", 2024-04-20T01:47:28.367 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-04-20T01:47:28.367 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_cache_enabled": "true", 2024-04-20T01:47:28.367 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_cache_expiry_interval": "900", 2024-04-20T01:47:28.367 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_cache_lru_size": "10000", 2024-04-20T01:47:28.368 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_config_store": "rados", 2024-04-20T01:47:28.368 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_content_length_compat": "false", 2024-04-20T01:47:28.368 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_copy_obj_progress": "true", 2024-04-20T01:47:28.368 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-04-20T01:47:28.368 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_cors_rules_max_num": "100", 2024-04-20T01:47:28.368 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_cross_domain_policy": "", 2024-04-20T01:47:28.369 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_default_encryption_key": "", 2024-04-20T01:47:28.369 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_kmip_addr": "", 2024-04-20T01:47:28.369 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_kmip_ca_path": "", 2024-04-20T01:47:28.369 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_kmip_client_cert": "", 2024-04-20T01:47:28.369 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_kmip_client_key": "", 2024-04-20T01:47:28.369 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-04-20T01:47:28.370 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_kmip_password": "", 2024-04-20T01:47:28.370 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-04-20T01:47:28.370 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_kmip_username": "", 2024-04-20T01:47:28.370 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_require_ssl": "true", 2024-04-20T01:47:28.370 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-04-20T01:47:28.370 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-04-20T01:47:28.371 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-04-20T01:47:28.371 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-04-20T01:47:28.371 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-04-20T01:47:28.371 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-04-20T01:47:28.372 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-04-20T01:47:28.372 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-04-20T01:47:28.372 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-04-20T01:47:28.372 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-04-20T01:47:28.372 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-04-20T01:47:28.372 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-04-20T01:47:28.372 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-04-20T01:47:28.372 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-04-20T01:47:28.372 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_suppress_logs": "true", 2024-04-20T01:47:28.372 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_vault_addr": "", 2024-04-20T01:47:28.373 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_vault_auth": "token", 2024-04-20T01:47:28.373 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_vault_namespace": "", 2024-04-20T01:47:28.373 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_vault_prefix": "", 2024-04-20T01:47:28.373 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-04-20T01:47:28.373 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-04-20T01:47:28.373 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-04-20T01:47:28.373 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-04-20T01:47:28.373 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_vault_token_file": "", 2024-04-20T01:47:28.373 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-04-20T01:47:28.373 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_curl_buffersize": "524288", 2024-04-20T01:47:28.373 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_curl_low_speed_limit": "1024", 2024-04-20T01:47:28.373 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_curl_low_speed_time": "300", 2024-04-20T01:47:28.373 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_curl_tcp_keepalive": "0", 2024-04-20T01:47:28.373 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-04-20T01:47:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-04-20T01:47:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-04-20T01:47:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-04-20T01:47:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-04-20T01:47:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_d3n_l1_fadvise": "4", 2024-04-20T01:47:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-04-20T01:47:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-04-20T01:47:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-04-20T01:47:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-04-20T01:47:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_data_log_changes_size": "1000", 2024-04-20T01:47:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_data_log_num_shards": "128", 2024-04-20T01:47:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-04-20T01:47:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_data_log_window": "30", 2024-04-20T01:47:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_data_notify_interval_msec": "0", 2024-04-20T01:47:28.374 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_data_sync_poll_interval": "20", 2024-04-20T01:47:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_data_sync_spawn_window": "20", 2024-04-20T01:47:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-04-20T01:47:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-04-20T01:47:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_default_data_log_backing": "fifo", 2024-04-20T01:47:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-04-20T01:47:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_default_region_info_oid": "default.region", 2024-04-20T01:47:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-04-20T01:47:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-04-20T01:47:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_defer_to_bucket_acls": "", 2024-04-20T01:47:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-04-20T01:47:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-04-20T01:47:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-04-20T01:47:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-04-20T01:47:28.375 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-04-20T01:47:28.376 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-04-20T01:47:28.376 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-04-20T01:47:28.376 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-04-20T01:47:28.376 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_data_res": "500.000000", 2024-04-20T01:47:28.376 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-04-20T01:47:28.376 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-04-20T01:47:28.376 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-04-20T01:47:28.376 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-04-20T01:47:28.376 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dns_name": "", 2024-04-20T01:47:28.376 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dns_s3website_name": "", 2024-04-20T01:47:28.376 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_dynamic_resharding": "true", 2024-04-20T01:47:28.376 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-04-20T01:47:28.376 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_enable_gc_threads": "true", 2024-04-20T01:47:28.376 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_enable_lc_threads": "true", 2024-04-20T01:47:28.376 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_enable_ops_log": "false", 2024-04-20T01:47:28.377 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_enable_quota_threads": "true", 2024-04-20T01:47:28.377 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_enable_static_website": "false", 2024-04-20T01:47:28.377 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_enable_usage_log": "false", 2024-04-20T01:47:28.377 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_enforce_swift_acls": "true", 2024-04-20T01:47:28.377 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_exit_timeout_secs": "120", 2024-04-20T01:47:28.377 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_expose_bucket": "false", 2024-04-20T01:47:28.377 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_extended_http_attrs": "", 2024-04-20T01:47:28.377 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_filter": "none", 2024-04-20T01:47:28.377 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-04-20T01:47:28.377 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_frontends": "beast port=7480", 2024-04-20T01:47:28.377 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_gc_max_concurrent_io": "10", 2024-04-20T01:47:28.377 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_gc_max_deferred": "50", 2024-04-20T01:47:28.377 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-04-20T01:47:28.377 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_gc_max_objs": "32", 2024-04-20T01:47:28.377 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_gc_max_queue_size": "134213632", 2024-04-20T01:47:28.378 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_gc_max_trim_chunk": "16", 2024-04-20T01:47:28.378 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_gc_obj_min_wait": "7200", 2024-04-20T01:47:28.378 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_gc_processor_max_time": "3600", 2024-04-20T01:47:28.378 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_gc_processor_period": "3600", 2024-04-20T01:47:28.378 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-04-20T01:47:28.378 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_get_obj_window_size": "16777216", 2024-04-20T01:47:28.378 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_healthcheck_disabling_path": "", 2024-04-20T01:47:28.378 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ignore_get_invalid_range": "false", 2024-04-20T01:47:28.378 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_init_timeout": "300", 2024-04-20T01:47:28.378 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-04-20T01:47:28.378 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-04-20T01:47:28.378 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-04-20T01:47:28.378 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-04-20T01:47:28.378 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_admin_domain": "", 2024-04-20T01:47:28.379 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_admin_password": "", 2024-04-20T01:47:28.379 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_admin_password_path": "", 2024-04-20T01:47:28.379 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_admin_project": "", 2024-04-20T01:47:28.379 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_admin_tenant": "", 2024-04-20T01:47:28.379 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_admin_token": "", 2024-04-20T01:47:28.379 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_admin_token_path": "", 2024-04-20T01:47:28.379 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_admin_user": "", 2024-04-20T01:47:28.379 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_api_version": "2", 2024-04-20T01:47:28.379 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_barbican_domain": "", 2024-04-20T01:47:28.379 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_barbican_password": "", 2024-04-20T01:47:28.379 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_barbican_project": "", 2024-04-20T01:47:28.379 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_barbican_tenant": "", 2024-04-20T01:47:28.379 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_barbican_user": "", 2024-04-20T01:47:28.379 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-04-20T01:47:28.380 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_implicit_tenants": "false", 2024-04-20T01:47:28.380 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-04-20T01:47:28.380 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_service_token_enabled": "false", 2024-04-20T01:47:28.380 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_token_cache_size": "10000", 2024-04-20T01:47:28.380 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_url": "", 2024-04-20T01:47:28.380 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_keystone_verify_ssl": "true", 2024-04-20T01:47:28.380 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_lc_debug_interval": "-1", 2024-04-20T01:47:28.380 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_lc_lock_max_time": "90", 2024-04-20T01:47:28.380 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_lc_max_objs": "32", 2024-04-20T01:47:28.380 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_lc_max_rules": "1000", 2024-04-20T01:47:28.380 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_lc_max_worker": "3", 2024-04-20T01:47:28.380 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_lc_max_wp_worker": "3", 2024-04-20T01:47:28.380 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_lc_thread_delay": "0", 2024-04-20T01:47:28.380 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-04-20T01:47:28.381 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ldap_dnattr": "uid", 2024-04-20T01:47:28.381 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-04-20T01:47:28.381 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ldap_searchfilter": "", 2024-04-20T01:47:28.381 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-04-20T01:47:28.381 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ldap_uri": "ldaps://", 2024-04-20T01:47:28.381 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-04-20T01:47:28.381 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-04-20T01:47:28.381 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-04-20T01:47:28.381 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_log_http_headers": "", 2024-04-20T01:47:28.381 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_log_nonexistent_bucket": "false", 2024-04-20T01:47:28.381 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-04-20T01:47:28.381 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_log_object_name_utc": "false", 2024-04-20T01:47:28.381 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-04-20T01:47:28.382 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_attr_name_len": "0", 2024-04-20T01:47:28.382 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_attr_size": "0", 2024-04-20T01:47:28.382 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_attrs_num_in_req": "0", 2024-04-20T01:47:28.382 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_chunk_size": "4194304", 2024-04-20T01:47:28.382 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_concurrent_requests": "1024", 2024-04-20T01:47:28.382 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-04-20T01:47:28.382 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_dynamic_shards": "1999", 2024-04-20T01:47:28.382 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_listing_results": "1000", 2024-04-20T01:47:28.382 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_notify_retries": "10", 2024-04-20T01:47:28.382 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_objs_per_shard": "100000", 2024-04-20T01:47:28.382 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_put_param_size": "1048576", 2024-04-20T01:47:28.382 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_put_size": "5368709120", 2024-04-20T01:47:28.382 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_max_slo_entries": "1000", 2024-04-20T01:47:28.382 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_md_log_max_shards": "64", 2024-04-20T01:47:28.382 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_md_notify_interval_msec": "200", 2024-04-20T01:47:28.383 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_meta_sync_poll_interval": "20", 2024-04-20T01:47:28.383 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_meta_sync_spawn_window": "20", 2024-04-20T01:47:28.383 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-04-20T01:47:28.383 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_mp_lock_max_time": "600", 2024-04-20T01:47:28.383 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-04-20T01:47:28.383 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_multipart_min_part_size": "5242880", 2024-04-20T01:47:28.383 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-04-20T01:47:28.383 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-04-20T01:47:28.383 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_fhcache_size": "2017", 2024-04-20T01:47:28.383 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-04-20T01:47:28.383 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-04-20T01:47:28.383 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_lru_lanes": "5", 2024-04-20T01:47:28.383 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_max_gc": "300", 2024-04-20T01:47:28.383 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-04-20T01:47:28.384 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_run_gc_threads": "false", 2024-04-20T01:47:28.384 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_run_lc_threads": "false", 2024-04-20T01:47:28.384 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_run_quota_threads": "false", 2024-04-20T01:47:28.384 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_run_sync_thread": "false", 2024-04-20T01:47:28.384 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-04-20T01:47:28.384 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-04-20T01:47:28.384 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_num_async_rados_threads": "32", 2024-04-20T01:47:28.384 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_num_control_oids": "8", 2024-04-20T01:47:28.384 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_numa_node": "-1", 2024-04-20T01:47:28.384 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_obj_stripe_size": "4194304", 2024-04-20T01:47:28.384 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-04-20T01:47:28.384 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_objexp_chunk_size": "100", 2024-04-20T01:47:28.384 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_objexp_gc_interval": "600", 2024-04-20T01:47:28.384 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_objexp_hints_num_shards": "127", 2024-04-20T01:47:28.384 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-04-20T01:47:28.385 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-04-20T01:47:28.385 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_op_thread_timeout": "600", 2024-04-20T01:47:28.385 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_op_tracing": "false", 2024-04-20T01:47:28.385 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_opa_token": "", 2024-04-20T01:47:28.385 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_opa_url": "", 2024-04-20T01:47:28.385 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_opa_verify_ssl": "true", 2024-04-20T01:47:28.385 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-04-20T01:47:28.385 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-04-20T01:47:28.385 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ops_log_rados": "false", 2024-04-20T01:47:28.385 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_ops_log_socket_path": "", 2024-04-20T01:47:28.385 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-04-20T01:47:28.385 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-04-20T01:47:28.385 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-04-20T01:47:28.385 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_period_push_interval": "2.000000", 2024-04-20T01:47:28.386 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_period_push_interval_max": "30.000000", 2024-04-20T01:47:28.386 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_period_root_pool": ".rgw.root", 2024-04-20T01:47:28.386 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-04-20T01:47:28.386 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_print_continue": "true", 2024-04-20T01:47:28.386 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_print_prohibited_content_length": "false", 2024-04-20T01:47:28.386 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-04-20T01:47:28.386 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-04-20T01:47:28.386 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-04-20T01:47:28.386 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-04-20T01:47:28.386 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_rados_tracing": "false", 2024-04-20T01:47:28.386 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_realm": "", 2024-04-20T01:47:28.386 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_realm_id": "", 2024-04-20T01:47:28.386 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-04-20T01:47:28.386 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_region": "", 2024-04-20T01:47:28.387 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_region_root_pool": ".rgw.root", 2024-04-20T01:47:28.387 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_relaxed_region_enforcement": "false", 2024-04-20T01:47:28.387 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-04-20T01:47:28.387 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-04-20T01:47:28.387 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_request_uri": "", 2024-04-20T01:47:28.387 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_reshard_batch_size": "64", 2024-04-20T01:47:28.387 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-04-20T01:47:28.387 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_reshard_max_aio": "128", 2024-04-20T01:47:28.387 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_reshard_num_logs": "16", 2024-04-20T01:47:28.387 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_reshard_thread_interval": "600", 2024-04-20T01:47:28.387 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_resolve_cname": "false", 2024-04-20T01:47:28.387 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_rest_getusage_op_compat": "false", 2024-04-20T01:47:28.387 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_run_sync_thread": "true", 2024-04-20T01:47:28.387 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-04-20T01:47:28.387 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_s3_auth_use_keystone": "false", 2024-04-20T01:47:28.388 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_s3_auth_use_ldap": "false", 2024-04-20T01:47:28.388 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_s3_auth_use_rados": "true", 2024-04-20T01:47:28.389 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_s3_auth_use_sts": "false", 2024-04-20T01:47:28.389 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-04-20T01:47:28.389 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_s3_success_create_obj_status": "0", 2024-04-20T01:47:28.389 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-04-20T01:47:28.389 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_scheduler_type": "throttler", 2024-04-20T01:47:28.389 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_script_uri": "", 2024-04-20T01:47:28.389 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_service_provider_name": "", 2024-04-20T01:47:28.389 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-04-20T01:47:28.390 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sts_client_id": "", 2024-04-20T01:47:28.390 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sts_client_secret": "", 2024-04-20T01:47:28.390 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sts_entry": "sts", 2024-04-20T01:47:28.390 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sts_key": "sts", 2024-04-20T01:47:28.390 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sts_max_session_duration": "43200", 2024-04-20T01:47:28.390 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sts_min_session_duration": "900", 2024-04-20T01:47:28.390 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sts_token_introspection_url": "", 2024-04-20T01:47:28.390 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_account_in_url": "false", 2024-04-20T01:47:28.390 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_auth_entry": "auth", 2024-04-20T01:47:28.390 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_auth_url": "", 2024-04-20T01:47:28.390 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_custom_header": "", 2024-04-20T01:47:28.390 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_enforce_content_length": "false", 2024-04-20T01:47:28.390 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_need_stats": "true", 2024-04-20T01:47:28.390 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_tenant_name": "", 2024-04-20T01:47:28.391 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_token_expiration": "86400", 2024-04-20T01:47:28.391 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_url": "", 2024-04-20T01:47:28.391 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_url_prefix": "swift", 2024-04-20T01:47:28.391 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_swift_versioning_enabled": "false", 2024-04-20T01:47:28.391 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-04-20T01:47:28.391 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-04-20T01:47:28.391 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_lease_period": "120", 2024-04-20T01:47:28.391 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-04-20T01:47:28.391 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_log_trim_interval": "1200", 2024-04-20T01:47:28.391 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-04-20T01:47:28.391 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-04-20T01:47:28.391 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-04-20T01:47:28.391 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_obj_etag_verify": "false", 2024-04-20T01:47:28.391 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_trace_history_size": "4096", 2024-04-20T01:47:28.391 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-04-20T01:47:28.392 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-04-20T01:47:28.392 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_thread_pool_size": "512", 2024-04-20T01:47:28.392 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_torrent_comment": "", 2024-04-20T01:47:28.392 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_torrent_createby": "", 2024-04-20T01:47:28.392 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_torrent_encoding": "", 2024-04-20T01:47:28.392 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_torrent_flag": "false", 2024-04-20T01:47:28.392 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_torrent_origin": "", 2024-04-20T01:47:28.392 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_torrent_sha_unit": "524288", 2024-04-20T01:47:28.392 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_torrent_tracker": "", 2024-04-20T01:47:28.392 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_trust_forwarded_https": "false", 2024-04-20T01:47:28.392 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-04-20T01:47:28.392 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_usage_log_tick_interval": "30", 2024-04-20T01:47:28.392 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_usage_max_shards": "32", 2024-04-20T01:47:28.392 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_usage_max_user_shards": "1", 2024-04-20T01:47:28.393 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_use_opa_authz": "false", 2024-04-20T01:47:28.393 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-04-20T01:47:28.393 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_user_default_quota_max_size": "-1", 2024-04-20T01:47:28.393 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_user_max_buckets": "1000", 2024-04-20T01:47:28.393 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_user_policies_max_num": "100", 2024-04-20T01:47:28.393 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-04-20T01:47:28.393 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-04-20T01:47:28.393 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_user_quota_sync_interval": "86400", 2024-04-20T01:47:28.393 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-04-20T01:47:28.393 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_user_unique_email": "true", 2024-04-20T01:47:28.393 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_verify_ssl": "true", 2024-04-20T01:47:28.393 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_website_routing_rules_max_num": "50", 2024-04-20T01:47:28.393 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_zone": "", 2024-04-20T01:47:28.393 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_zone_id": "", 2024-04-20T01:47:28.393 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-04-20T01:47:28.394 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_zonegroup": "", 2024-04-20T01:47:28.394 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_zonegroup_id": "", 2024-04-20T01:47:28.394 INFO:teuthology.orchestra.run.smithi044.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-04-20T01:47:28.394 INFO:teuthology.orchestra.run.smithi044.stdout: "rgwlc_auto_session_clear": "true", 2024-04-20T01:47:28.394 INFO:teuthology.orchestra.run.smithi044.stdout: "rgwlc_skip_bucket_step": "false", 2024-04-20T01:47:28.394 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_block_size": "4096", 2024-04-20T01:47:28.394 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-04-20T01:47:28.394 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-04-20T01:47:28.394 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-04-20T01:47:28.394 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-04-20T01:47:28.394 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_cache_shard_bits": "4", 2024-04-20T01:47:28.394 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_cache_size": "536870912", 2024-04-20T01:47:28.394 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_cache_type": "binned_lru", 2024-04-20T01:47:28.394 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-04-20T01:47:28.395 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-04-20T01:47:28.395 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-04-20T01:47:28.395 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_collect_compaction_stats": "false", 2024-04-20T01:47:28.395 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_collect_extended_stats": "false", 2024-04-20T01:47:28.395 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_collect_memory_stats": "false", 2024-04-20T01:47:28.395 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-04-20T01:47:28.395 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_index_type": "binary_search", 2024-04-20T01:47:28.395 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_log_to_ceph_log": "true", 2024-04-20T01:47:28.395 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_metadata_block_size": "4096", 2024-04-20T01:47:28.395 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_partition_filters": "false", 2024-04-20T01:47:28.395 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_perf": "false", 2024-04-20T01:47:28.395 INFO:teuthology.orchestra.run.smithi044.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-04-20T01:47:28.395 INFO:teuthology.orchestra.run.smithi044.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-04-20T01:47:28.395 INFO:teuthology.orchestra.run.smithi044.stdout: "rotating_keys_renewal_timeout": "10", 2024-04-20T01:47:28.396 INFO:teuthology.orchestra.run.smithi044.stdout: "run_dir": "/var/run/ceph", 2024-04-20T01:47:28.396 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_block_create": "true", 2024-04-20T01:47:28.396 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_cache_lru_size": "67108864", 2024-04-20T01:47:28.396 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_cbjournal_size": "5368709120", 2024-04-20T01:47:28.396 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_default_max_object_size": "16777216", 2024-04-20T01:47:28.396 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-04-20T01:47:28.396 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_device_size": "53687091200", 2024-04-20T01:47:28.396 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_journal_batch_capacity": "16", 2024-04-20T01:47:28.396 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-04-20T01:47:28.396 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-04-20T01:47:28.396 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_journal_iodepth_limit": "5", 2024-04-20T01:47:28.396 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_main_device_type": "SSD", 2024-04-20T01:47:28.396 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_max_concurrent_transactions": "8", 2024-04-20T01:47:28.396 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-04-20T01:47:28.396 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-04-20T01:47:28.396 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-04-20T01:47:28.397 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-04-20T01:47:28.397 INFO:teuthology.orchestra.run.smithi044.stdout: "seastore_segment_size": "67108864", 2024-04-20T01:47:28.397 INFO:teuthology.orchestra.run.smithi044.stdout: "set_keepcaps": "false", 2024-04-20T01:47:28.397 INFO:teuthology.orchestra.run.smithi044.stdout: "setgroup": "", 2024-04-20T01:47:28.397 INFO:teuthology.orchestra.run.smithi044.stdout: "setuser": "", 2024-04-20T01:47:28.397 INFO:teuthology.orchestra.run.smithi044.stdout: "setuser_match_path": "", 2024-04-20T01:47:28.397 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_misplaced_ratio": "0.050000", 2024-04-20T01:47:28.397 INFO:teuthology.orchestra.run.smithi044.stdout: "thp": "false", 2024-04-20T01:47:28.397 INFO:teuthology.orchestra.run.smithi044.stdout: "threadpool_default_timeout": "60", 2024-04-20T01:47:28.397 INFO:teuthology.orchestra.run.smithi044.stdout: "threadpool_empty_queue_max_wait": "2", 2024-04-20T01:47:28.397 INFO:teuthology.orchestra.run.smithi044.stdout: "throttler_perf_counter": "true" 2024-04-20T01:47:28.397 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:28.400 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-04-20T01:47:28.559 INFO:teuthology.orchestra.run.smithi044.stdout:mon.a: {} 2024-04-20T01:47:28.571 INFO:teuthology.orchestra.run.smithi044.stdout:mon.b: {} 2024-04-20T01:47:28.583 INFO:teuthology.orchestra.run.smithi044.stdout:mon.c: {} 2024-04-20T01:47:28.584 INFO:teuthology.orchestra.run.smithi044.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-20T01:47:28.584 INFO:teuthology.orchestra.run.smithi044.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-20T01:47:28.584 INFO:teuthology.orchestra.run.smithi044.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-20T01:47:28.596 DEBUG:teuthology.orchestra.run.smithi165:> type ceph-objectstore-tool 2024-04-20T01:47:28.617 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-20T01:47:28.617 DEBUG:teuthology.orchestra.run.smithi121:> type ceph-objectstore-tool 2024-04-20T01:47:28.637 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-20T01:47:28.637 DEBUG:teuthology.orchestra.run.smithi123:> type ceph-objectstore-tool 2024-04-20T01:47:28.658 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-20T01:47:28.659 DEBUG:teuthology.orchestra.run.smithi044:> type ceph-objectstore-tool 2024-04-20T01:47:28.677 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-20T01:47:28.678 INFO:teuthology.run_tasks:Running task rados... 2024-04-20T01:47:28.689 INFO:tasks.rados:Beginning rados... 2024-04-20T01:47:28.690 DEBUG:teuthology.run_tasks:Unwinding manager rados 2024-04-20T01:47:28.707 INFO:tasks.rados:joining rados 2024-04-20T01:47:28.707 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-04-20T01:47:28.707 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-20T01:47:28.707 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-20T01:47:28.707 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-04-20T01:47:28.708 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-04-20T01:47:28.708 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-20T01:47:28.708 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-04-20T01:47:28.709 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-20T01:47:28.709 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-04-20T01:47:28.710 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-20T01:47:28.710 INFO:tasks.rados:clients are ['client.0'] 2024-04-20T01:47:28.710 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set shecprofile c=2 crush-failure-domain=osd k=4 m=3 name=shecprofile plugin=shec 2024-04-20T01:47:28.719 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:47:28.733+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:47:28.810 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:28.824+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:47:28.884 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:28.885 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:47:28.885 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-04-20T01:47:28.885 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:28.900 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-20T01:47:28.909 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:47:28.910 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:47:28.924+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:47:28.934 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:47:28.969 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:47:28.991 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:47:29.011 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:47:29.025+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:47:29.012 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:47:29.035 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:47:29.055 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:47:29.059 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:29.060 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:29.072 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-20T01:47:29.078 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:47:29.100 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:47:29.112 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:47:29.125+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:47:29.123 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:47:29.144 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:47:29.166 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:47:29.187 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:47:29.214 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:29.226+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:29.223 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:29.223 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:29.235 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-20T01:47:29.313 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:47:29.327+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:47:29.411 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:47:29.414 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:47:29.428+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:47:29.422 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:29.422 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:29.432 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:47:29.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:47:29.446+0000 7fc83b7fe700 0 -- 172.21.15.44:0/1009480835 >> [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] conn(0x7fc83c21f680 msgr2=0x7fc83c221a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-20T01:47:29.434 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-20T01:47:29.515 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:29.529+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:47:29.616 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:29.630+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:47:29.621 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:29.622 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:29.632 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-20T01:47:29.653 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:47:29.655 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:47:29.655 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:47:29.656 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:47:29.656 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:47:29.656 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:47:29.656 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:47:29.656 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:47:29.656 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:47:29.656 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:47:29.656 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:47:29.656 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:47:29.656 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:47:29.656 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:47:29.656 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:47:29.656 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:47:29.656 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:47:29.691 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is set 2024-04-20T01:47:29.692 INFO:teuthology.orchestra.run.smithi044.stderr:load: shec 2024-04-20T01:47:29.707 INFO:tasks.rados:starting run 0 out of 1 2024-04-20T01:47:29.707 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-04-20T01:47:29.707 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 16 erasure shecprofile 2024-04-20T01:47:29.717 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:47:29.731+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:47:29.783 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:29.783 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:29.796 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-20T01:47:29.818 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:29.832+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:47:29.920 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:29.934+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:29.983 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:29.983 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:29.995 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-20T01:47:30.020 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:30.034+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:30.121 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:30.135+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:30.139 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:30.140 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:30.151 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-20T01:47:30.223 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:30.237+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:30.295 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:30.295 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:30.307 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-20T01:47:30.323 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:30.337+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:30.425 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:30.438+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:47:30.450 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:30.451 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:30.462 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-20T01:47:30.526 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:47:30.540+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:47:30.605 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:30.605 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:30.616 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-20T01:47:30.627 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:30.641+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:47:30.728 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:30.742+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:47:30.807 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:30.807 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:30.819 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-20T01:47:30.828 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:30.843+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:30.929 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:47:30.943+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:47:30.960 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:30.960 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:30.971 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-20T01:47:31.030 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:47:31.045+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:47:31.098 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:31.098 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:31.108 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-20T01:47:31.131 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:47:31.145+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:47:31.247 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:31.247 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:31.247 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:31.246+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:47:31.248 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-20T01:47:31.333 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:31.347+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:31.377 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:31.377 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:31.387 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-20T01:47:31.471 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:31.485+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:47:31.535 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:31.549+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:47:31.554 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:31.554 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:31.564 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-20T01:47:31.636 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:47:31.650+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:47:31.670 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-20T01:47:31.696 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'unique_pool_0' created 2024-04-20T01:47:31.708 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-20T01:47:31.716 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-04-20T01:47:31.737 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:31.737 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:31.738 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:31.751+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:31.748 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-20T01:47:31.838 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:31.852+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:31.857 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:47:31.879 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:47:31.901 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:47:31.919 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:31.919 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:31.923 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:47:31.930 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-20T01:47:31.944 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:47:31.964 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:47:31.973 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:31.987+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:47:31.985 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:47:32.006 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:47:32.029 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:47:32.041 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:32.055+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:47:32.049 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:47:32.058 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:32.059 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:47:32.059 INFO:teuthology.orchestra.run.smithi121.stdout: "complaint_time": 30, 2024-04-20T01:47:32.059 INFO:teuthology.orchestra.run.smithi121.stdout: "num_blocked_ops": 0 2024-04-20T01:47:32.059 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:32.069 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-20T01:47:32.070 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:47:32.091 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:47:32.112 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:47:32.132 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:47:32.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:47:32.147+0000 7f879527f700 0 -- 172.21.15.44:0/3027425709 >> [v2:172.21.15.123:6824/43110,v1:172.21.15.123:6825/43110] conn(0x7f879021c7e0 msgr2=0x7f879021ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-20T01:47:32.153 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:32.167+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:47:32.237 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:32.237 INFO:teuthology.orchestra.run.smithi121.stdout: "size": 20, 2024-04-20T01:47:32.237 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 600, 2024-04-20T01:47:32.237 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [] 2024-04-20T01:47:32.237 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:32.242 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:47:32.257+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:47:32.247 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-20T01:47:32.343 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:32.357+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:47:32.356 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:47:32.378 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:47:32.380 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:47:32.380 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:47:32.380 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:47:32.380 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:47:32.380 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:47:32.380 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:47:32.380 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:47:32.380 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:47:32.380 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:47:32.381 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:47:32.381 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:47:32.381 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:47:32.381 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:47:32.381 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:47:32.381 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:47:32.381 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:47:32.419 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:32.420 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:47:32.420 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 0 2024-04-20T01:47:32.420 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:32.430 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-20T01:47:32.443 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:47:32.458+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:47:32.545 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:47:32.559+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:47:32.597 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:32.597 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:47:32.597 INFO:teuthology.orchestra.run.smithi121.stdout: "complaint_time": 30, 2024-04-20T01:47:32.597 INFO:teuthology.orchestra.run.smithi121.stdout: "num_blocked_ops": 0 2024-04-20T01:47:32.597 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:32.607 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-20T01:47:32.645 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:32.659+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-04-20T01:47:32.705 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is set 2024-04-20T01:47:32.744 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:32.744 INFO:teuthology.orchestra.run.smithi121.stdout: "size": 20, 2024-04-20T01:47:32.744 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 600, 2024-04-20T01:47:32.744 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-04-20T01:47:32.744 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.744 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-20T01:47:32.744 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:17.004323+0000", 2024-04-20T01:47:32.744 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 15.75227194, 2024-04-20T01:47:32.744 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.29931750600000001, 2024-04-20T01:47:32.745 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:32.745 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:32.745 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:32.745 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:32.745 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:32.745 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 1 2024-04-20T01:47:32.745 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.745 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:32.745 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.745 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:32.745 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004323+0000", 2024-04-20T01:47:32.745 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.745 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.745 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.746 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:32.746 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004323+0000", 2024-04-20T01:47:32.746 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.746 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.746 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.746 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:32.746 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004321+0000", 2024-04-20T01:47:32.746 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:32.746 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.746 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.746 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:32.746 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004337+0000", 2024-04-20T01:47:32.746 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5866999999999999e-05 2024-04-20T01:47:32.746 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.746 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.747 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:32.747 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004349+0000", 2024-04-20T01:47:32.747 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1802e-05 2024-04-20T01:47:32.747 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.747 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.747 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:32.747 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004360+0000", 2024-04-20T01:47:32.747 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1348999999999999e-05 2024-04-20T01:47:32.747 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.747 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.747 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.747 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004411+0000", 2024-04-20T01:47:32.747 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.0385e-05 2024-04-20T01:47:32.747 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.748 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.748 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for peered", 2024-04-20T01:47:32.748 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004419+0000", 2024-04-20T01:47:32.748 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4430000000000003e-06 2024-04-20T01:47:32.748 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.748 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.748 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.748 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.302330+0000", 2024-04-20T01:47:32.748 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.29791090199999998 2024-04-20T01:47:32.748 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.748 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.748 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:32.748 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.302488+0000", 2024-04-20T01:47:32.748 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00015830799999999999 2024-04-20T01:47:32.749 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.749 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.749 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:32.749 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.302650+0000", 2024-04-20T01:47:32.749 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00016196900000000001 2024-04-20T01:47:32.749 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.749 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.749 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:32.749 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.303204+0000", 2024-04-20T01:47:32.749 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00055385000000000005 2024-04-20T01:47:32.749 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.749 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.749 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:32.749 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.303574+0000", 2024-04-20T01:47:32.750 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00037030800000000002 2024-04-20T01:47:32.750 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.750 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.750 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:32.750 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.303614+0000", 2024-04-20T01:47:32.750 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.9161e-05 2024-04-20T01:47:32.750 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.750 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.750 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:32.750 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.303640+0000", 2024-04-20T01:47:32.750 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6752999999999999e-05 2024-04-20T01:47:32.750 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.750 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.750 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.750 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.751 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.751 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-20T01:47:32.751 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:17.550637+0000", 2024-04-20T01:47:32.751 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 15.205957937000001, 2024-04-20T01:47:32.751 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0073606729999999999, 2024-04-20T01:47:32.751 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:32.751 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:32.751 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:32.751 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:32.751 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:32.751 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 110 2024-04-20T01:47:32.751 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.751 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:32.751 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.752 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:32.752 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550637+0000", 2024-04-20T01:47:32.752 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.752 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.752 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.752 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:32.752 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550637+0000", 2024-04-20T01:47:32.752 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.752 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.752 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.752 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:32.752 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550636+0000", 2024-04-20T01:47:32.752 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:32.752 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.752 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.753 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:32.753 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550640+0000", 2024-04-20T01:47:32.753 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2939999999999999e-06 2024-04-20T01:47:32.753 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.753 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.753 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:32.753 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550647+0000", 2024-04-20T01:47:32.753 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.1149999999999999e-06 2024-04-20T01:47:32.753 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.753 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.753 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:32.753 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550652+0000", 2024-04-20T01:47:32.753 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.2839999999999997e-06 2024-04-20T01:47:32.753 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.753 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.754 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.754 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.552918+0000", 2024-04-20T01:47:32.754 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0022659870000000001 2024-04-20T01:47:32.754 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.754 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.754 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:32.754 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.552968+0000", 2024-04-20T01:47:32.754 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.9963000000000003e-05 2024-04-20T01:47:32.754 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.754 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.754 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:32.754 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.553074+0000", 2024-04-20T01:47:32.754 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000105754 2024-04-20T01:47:32.754 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.755 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.755 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:32.755 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.555605+0000", 2024-04-20T01:47:32.755 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0025312300000000002 2024-04-20T01:47:32.755 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.755 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.755 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:32.755 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.557928+0000", 2024-04-20T01:47:32.755 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0023232959999999999 2024-04-20T01:47:32.755 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.755 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.755 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:32.755 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.557960+0000", 2024-04-20T01:47:32.755 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.1795000000000003e-05 2024-04-20T01:47:32.755 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.756 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.756 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:32.756 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.557997+0000", 2024-04-20T01:47:32.756 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7604000000000001e-05 2024-04-20T01:47:32.756 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.756 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.756 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.756 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.756 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.756 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-20T01:47:32.756 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:17.550658+0000", 2024-04-20T01:47:32.756 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 15.205937183, 2024-04-20T01:47:32.756 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.007403799, 2024-04-20T01:47:32.756 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:32.756 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:32.757 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:32.757 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:32.757 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:32.757 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 111 2024-04-20T01:47:32.757 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.757 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:32.757 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.757 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:32.757 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550658+0000", 2024-04-20T01:47:32.757 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.757 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.757 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.757 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:32.757 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550658+0000", 2024-04-20T01:47:32.758 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.758 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.758 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.758 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:32.758 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550657+0000", 2024-04-20T01:47:32.758 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:32.758 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.758 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.758 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:32.758 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550661+0000", 2024-04-20T01:47:32.758 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.6370000000000001e-06 2024-04-20T01:47:32.758 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.758 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.758 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:32.758 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550666+0000", 2024-04-20T01:47:32.759 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.5489999999999997e-06 2024-04-20T01:47:32.759 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.759 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.759 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:32.759 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550673+0000", 2024-04-20T01:47:32.759 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.8669999999999996e-06 2024-04-20T01:47:32.759 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.759 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.759 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.759 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.553288+0000", 2024-04-20T01:47:32.759 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0026148489999999998 2024-04-20T01:47:32.759 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.759 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.759 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:32.759 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.553350+0000", 2024-04-20T01:47:32.760 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.211e-05 2024-04-20T01:47:32.760 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.760 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.760 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:32.760 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.553428+0000", 2024-04-20T01:47:32.760 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.8077999999999993e-05 2024-04-20T01:47:32.760 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.760 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.760 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:32.760 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.555629+0000", 2024-04-20T01:47:32.760 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0022009030000000001 2024-04-20T01:47:32.760 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.760 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.760 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:32.761 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.558004+0000", 2024-04-20T01:47:32.761 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.002374807 2024-04-20T01:47:32.761 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.761 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.761 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:32.761 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.558029+0000", 2024-04-20T01:47:32.761 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4964e-05 2024-04-20T01:47:32.761 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.761 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.761 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:32.761 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.558061+0000", 2024-04-20T01:47:32.761 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.2435999999999999e-05 2024-04-20T01:47:32.761 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.761 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.762 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.762 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.762 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.762 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-20T01:47:32.762 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:17.550679+0000", 2024-04-20T01:47:32.762 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 15.205915753999999, 2024-04-20T01:47:32.762 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0074439629999999996, 2024-04-20T01:47:32.762 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:32.762 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:32.762 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:32.762 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:32.762 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:32.762 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 112 2024-04-20T01:47:32.762 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.763 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:32.763 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.763 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:32.763 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550679+0000", 2024-04-20T01:47:32.763 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.763 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.763 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.763 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:32.763 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550679+0000", 2024-04-20T01:47:32.763 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.763 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.763 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.763 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:32.763 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550679+0000", 2024-04-20T01:47:32.763 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:32.764 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.764 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.764 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:32.764 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550764+0000", 2024-04-20T01:47:32.764 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4913999999999994e-05 2024-04-20T01:47:32.764 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.764 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.764 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:32.764 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550772+0000", 2024-04-20T01:47:32.764 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.1880000000000002e-06 2024-04-20T01:47:32.764 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.764 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.764 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:32.764 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550780+0000", 2024-04-20T01:47:32.764 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.5390000000000007e-06 2024-04-20T01:47:32.765 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.765 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.765 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.765 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.553758+0000", 2024-04-20T01:47:32.765 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0029777750000000002 2024-04-20T01:47:32.765 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.765 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.765 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:32.765 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.553814+0000", 2024-04-20T01:47:32.765 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.5698000000000001e-05 2024-04-20T01:47:32.765 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.765 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.765 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:32.765 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.553902+0000", 2024-04-20T01:47:32.765 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8743000000000001e-05 2024-04-20T01:47:32.766 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.766 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.766 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:32.766 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.555721+0000", 2024-04-20T01:47:32.766 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0018180799999999999 2024-04-20T01:47:32.766 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.766 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.766 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:32.766 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.558066+0000", 2024-04-20T01:47:32.766 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0023455250000000002 2024-04-20T01:47:32.766 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.766 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.766 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:32.766 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.558089+0000", 2024-04-20T01:47:32.767 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3183000000000001e-05 2024-04-20T01:47:32.767 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.767 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.767 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:32.767 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.558123+0000", 2024-04-20T01:47:32.767 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.3699999999999999e-05 2024-04-20T01:47:32.767 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.767 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.767 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.767 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.767 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.767 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-20T01:47:32.767 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:17.550788+0000", 2024-04-20T01:47:32.767 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 15.205806537000001, 2024-04-20T01:47:32.767 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0075609989999999997, 2024-04-20T01:47:32.768 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:32.768 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:32.768 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:32.768 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:32.768 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:32.768 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 113 2024-04-20T01:47:32.768 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.768 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:32.768 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.768 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:32.768 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550788+0000", 2024-04-20T01:47:32.768 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.768 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.768 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.768 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:32.768 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550788+0000", 2024-04-20T01:47:32.769 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.769 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.769 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.769 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:32.769 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550788+0000", 2024-04-20T01:47:32.769 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:32.769 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.769 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.769 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:32.769 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550792+0000", 2024-04-20T01:47:32.769 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.3939999999999998e-06 2024-04-20T01:47:32.769 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.769 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.769 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:32.769 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550799+0000", 2024-04-20T01:47:32.770 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.3339999999999996e-06 2024-04-20T01:47:32.770 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.770 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.770 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:32.770 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550805+0000", 2024-04-20T01:47:32.770 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3240000000000002e-06 2024-04-20T01:47:32.770 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.770 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.770 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.770 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.554150+0000", 2024-04-20T01:47:32.770 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0033454539999999999 2024-04-20T01:47:32.770 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.770 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.771 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:32.771 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.554201+0000", 2024-04-20T01:47:32.771 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.0504999999999998e-05 2024-04-20T01:47:32.771 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.771 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.771 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:32.771 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.554278+0000", 2024-04-20T01:47:32.771 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.7269999999999997e-05 2024-04-20T01:47:32.771 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.771 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.771 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:32.771 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.558128+0000", 2024-04-20T01:47:32.771 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0038501709999999999 2024-04-20T01:47:32.771 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.771 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.772 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:32.772 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.558293+0000", 2024-04-20T01:47:32.772 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00016487199999999999 2024-04-20T01:47:32.772 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.772 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.772 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:32.772 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.558326+0000", 2024-04-20T01:47:32.772 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.2762999999999998e-05 2024-04-20T01:47:32.772 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.772 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.772 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:32.772 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.558349+0000", 2024-04-20T01:47:32.772 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3510999999999999e-05 2024-04-20T01:47:32.772 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.772 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.773 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.773 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.773 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.773 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-20T01:47:32.773 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:17.550813+0000", 2024-04-20T01:47:32.773 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 15.205782197, 2024-04-20T01:47:32.773 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0076324469999999997, 2024-04-20T01:47:32.773 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:32.773 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:32.773 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:32.773 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:32.773 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:32.773 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 114 2024-04-20T01:47:32.773 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.774 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:32.774 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.774 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:32.774 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550813+0000", 2024-04-20T01:47:32.774 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.774 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.774 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.774 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:32.774 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550813+0000", 2024-04-20T01:47:32.774 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.774 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.774 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.774 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:32.774 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550812+0000", 2024-04-20T01:47:32.775 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:32.775 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.775 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.775 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:32.775 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550816+0000", 2024-04-20T01:47:32.775 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.8879999999999999e-06 2024-04-20T01:47:32.775 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.775 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.775 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:32.775 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550824+0000", 2024-04-20T01:47:32.775 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.8669999999999994e-06 2024-04-20T01:47:32.775 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.775 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.775 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:32.776 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550828+0000", 2024-04-20T01:47:32.776 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.7530000000000001e-06 2024-04-20T01:47:32.776 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.776 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.776 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.776 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.554753+0000", 2024-04-20T01:47:32.776 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.003924716 2024-04-20T01:47:32.776 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.776 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.776 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:32.776 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.554805+0000", 2024-04-20T01:47:32.776 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.1842000000000003e-05 2024-04-20T01:47:32.776 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.776 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.776 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:32.777 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.554880+0000", 2024-04-20T01:47:32.777 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.5368000000000001e-05 2024-04-20T01:47:32.777 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.777 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.777 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:32.777 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.558131+0000", 2024-04-20T01:47:32.777 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00325034 2024-04-20T01:47:32.777 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.777 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.777 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:32.777 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.558379+0000", 2024-04-20T01:47:32.777 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00024860799999999999 2024-04-20T01:47:32.777 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.777 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.777 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:32.778 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.558405+0000", 2024-04-20T01:47:32.778 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5958000000000001e-05 2024-04-20T01:47:32.778 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.778 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.778 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:32.778 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.558445+0000", 2024-04-20T01:47:32.778 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.9693000000000001e-05 2024-04-20T01:47:32.778 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.778 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.778 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.778 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.778 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.778 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-20T01:47:32.778 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:17.550842+0000", 2024-04-20T01:47:32.779 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 15.205752449, 2024-04-20T01:47:32.779 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.01010514, 2024-04-20T01:47:32.779 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:32.779 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:32.779 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:32.779 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:32.779 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:32.779 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 115 2024-04-20T01:47:32.779 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.779 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:32.779 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.779 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:32.779 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550842+0000", 2024-04-20T01:47:32.779 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.779 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.780 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.780 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:32.780 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550842+0000", 2024-04-20T01:47:32.780 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.780 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.780 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.780 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:32.780 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550842+0000", 2024-04-20T01:47:32.780 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:32.780 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.780 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.780 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:32.780 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550928+0000", 2024-04-20T01:47:32.780 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.5882000000000002e-05 2024-04-20T01:47:32.780 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.781 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.781 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:32.781 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550936+0000", 2024-04-20T01:47:32.781 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8529999999999998e-06 2024-04-20T01:47:32.781 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.781 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.781 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:32.781 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550943+0000", 2024-04-20T01:47:32.781 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.3899999999999998e-06 2024-04-20T01:47:32.781 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.781 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.781 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.781 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.555089+0000", 2024-04-20T01:47:32.781 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0041463919999999996 2024-04-20T01:47:32.782 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.782 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.782 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:32.782 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.555138+0000", 2024-04-20T01:47:32.782 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.8835000000000003e-05 2024-04-20T01:47:32.782 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.782 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.782 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:32.782 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.555226+0000", 2024-04-20T01:47:32.782 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.7855000000000005e-05 2024-04-20T01:47:32.782 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.782 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.782 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:32.782 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.558493+0000", 2024-04-20T01:47:32.783 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0032671290000000001 2024-04-20T01:47:32.783 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.783 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.783 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:32.783 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.560871+0000", 2024-04-20T01:47:32.783 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0023779029999999998 2024-04-20T01:47:32.783 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.783 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.783 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:32.783 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.560908+0000", 2024-04-20T01:47:32.783 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.731e-05 2024-04-20T01:47:32.783 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.783 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.783 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:32.784 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.560947+0000", 2024-04-20T01:47:32.784 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.9152000000000001e-05 2024-04-20T01:47:32.784 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.784 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.784 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.784 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.784 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.784 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-20T01:47:32.784 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:17.550951+0000", 2024-04-20T01:47:32.784 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 15.205644217, 2024-04-20T01:47:32.784 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010063809999999999, 2024-04-20T01:47:32.784 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:32.784 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:32.784 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:32.784 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:32.785 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:32.785 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 116 2024-04-20T01:47:32.785 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.785 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:32.785 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.785 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:32.785 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550951+0000", 2024-04-20T01:47:32.785 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.785 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.785 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.785 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:32.785 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550951+0000", 2024-04-20T01:47:32.785 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.785 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.786 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.786 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:32.786 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550950+0000", 2024-04-20T01:47:32.786 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:32.786 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.786 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.786 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:32.786 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550954+0000", 2024-04-20T01:47:32.786 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.104e-06 2024-04-20T01:47:32.786 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.786 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.786 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:32.786 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550971+0000", 2024-04-20T01:47:32.786 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7000999999999999e-05 2024-04-20T01:47:32.786 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.787 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.787 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:32.787 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550978+0000", 2024-04-20T01:47:32.787 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.8510000000000001e-06 2024-04-20T01:47:32.787 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.787 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.787 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.787 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.555745+0000", 2024-04-20T01:47:32.787 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.004767477 2024-04-20T01:47:32.787 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.787 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.787 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:32.787 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.555795+0000", 2024-04-20T01:47:32.787 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.0031999999999998e-05 2024-04-20T01:47:32.788 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.788 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.788 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:32.788 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.555874+0000", 2024-04-20T01:47:32.788 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.8752999999999996e-05 2024-04-20T01:47:32.788 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.788 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.788 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:32.788 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.558609+0000", 2024-04-20T01:47:32.788 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0027350859999999999 2024-04-20T01:47:32.788 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.788 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.788 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:32.788 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.560955+0000", 2024-04-20T01:47:32.788 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0023458009999999998 2024-04-20T01:47:32.789 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.789 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.789 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:32.789 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.560980+0000", 2024-04-20T01:47:32.789 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5386000000000001e-05 2024-04-20T01:47:32.789 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.789 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.789 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:32.789 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.561014+0000", 2024-04-20T01:47:32.789 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.3915999999999999e-05 2024-04-20T01:47:32.789 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.789 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.789 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.789 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.789 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.790 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-20T01:47:32.790 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:17.550985+0000", 2024-04-20T01:47:32.790 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 15.205610006000001, 2024-04-20T01:47:32.790 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010091007000000001, 2024-04-20T01:47:32.790 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:32.790 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:32.790 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:32.790 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:32.790 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:32.790 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 117 2024-04-20T01:47:32.790 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.790 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:32.790 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.790 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:32.791 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550985+0000", 2024-04-20T01:47:32.791 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.791 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.791 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.791 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:32.791 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550985+0000", 2024-04-20T01:47:32.791 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.791 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.791 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.791 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:32.791 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550984+0000", 2024-04-20T01:47:32.791 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:32.791 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.791 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.791 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:32.792 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550989+0000", 2024-04-20T01:47:32.792 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.391e-06 2024-04-20T01:47:32.792 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.792 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.792 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:32.792 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.550997+0000", 2024-04-20T01:47:32.792 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.816e-06 2024-04-20T01:47:32.792 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.792 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.792 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:32.792 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.551003+0000", 2024-04-20T01:47:32.792 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3000000000000001e-06 2024-04-20T01:47:32.792 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.792 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.792 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.793 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.556082+0000", 2024-04-20T01:47:32.793 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0050794389999999998 2024-04-20T01:47:32.793 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.793 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.793 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:32.793 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.556134+0000", 2024-04-20T01:47:32.793 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.1992e-05 2024-04-20T01:47:32.793 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.793 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.793 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:32.793 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.556215+0000", 2024-04-20T01:47:32.793 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.0475000000000005e-05 2024-04-20T01:47:32.793 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.793 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.793 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:32.794 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.558815+0000", 2024-04-20T01:47:32.794 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0026005199999999998 2024-04-20T01:47:32.794 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.794 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.794 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:32.794 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.561020+0000", 2024-04-20T01:47:32.794 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0022044360000000002 2024-04-20T01:47:32.794 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.794 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.794 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:32.794 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.561044+0000", 2024-04-20T01:47:32.794 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4275000000000001e-05 2024-04-20T01:47:32.794 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.794 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.795 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:32.795 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.561076+0000", 2024-04-20T01:47:32.795 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.1847999999999997e-05 2024-04-20T01:47:32.795 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.795 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.795 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.795 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.795 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.795 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-20T01:47:32.795 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:17.669983+0000", 2024-04-20T01:47:32.795 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 15.086611721000001, 2024-04-20T01:47:32.795 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00660569, 2024-04-20T01:47:32.795 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:32.795 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:32.795 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:32.796 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:32.796 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:32.796 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 139 2024-04-20T01:47:32.796 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.796 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:32.796 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.796 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:32.796 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.669983+0000", 2024-04-20T01:47:32.796 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.796 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.796 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.796 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:32.796 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.669983+0000", 2024-04-20T01:47:32.797 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.797 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.797 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.797 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:32.797 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.669983+0000", 2024-04-20T01:47:32.797 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:47:32.797 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.797 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.797 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:32.797 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.669985+0000", 2024-04-20T01:47:32.797 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0049999999999999e-06 2024-04-20T01:47:32.797 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.797 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.797 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:32.797 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.669988+0000", 2024-04-20T01:47:32.798 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.3019999999999999e-06 2024-04-20T01:47:32.798 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.798 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.798 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:32.798 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.669991+0000", 2024-04-20T01:47:32.798 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5220000000000002e-06 2024-04-20T01:47:32.798 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.798 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.798 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.798 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.674908+0000", 2024-04-20T01:47:32.798 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0049171739999999999 2024-04-20T01:47:32.798 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.798 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.798 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:32.799 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.674987+0000", 2024-04-20T01:47:32.799 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.8726999999999993e-05 2024-04-20T01:47:32.799 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.799 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.799 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:32.799 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.675065+0000", 2024-04-20T01:47:32.799 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.8417000000000005e-05 2024-04-20T01:47:32.799 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.799 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.799 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:32.799 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.676466+0000", 2024-04-20T01:47:32.799 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0014015169999999999 2024-04-20T01:47:32.799 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.799 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.800 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:32.800 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.676507+0000", 2024-04-20T01:47:32.800 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.0522999999999998e-05 2024-04-20T01:47:32.800 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.800 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.800 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:32.800 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.676555+0000", 2024-04-20T01:47:32.800 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.7960999999999997e-05 2024-04-20T01:47:32.800 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.800 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.800 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:32.800 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.676589+0000", 2024-04-20T01:47:32.800 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.3748000000000003e-05 2024-04-20T01:47:32.800 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.800 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.801 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.801 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.801 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.801 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-20T01:47:32.801 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:17.669993+0000", 2024-04-20T01:47:32.801 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 15.086601676000001, 2024-04-20T01:47:32.801 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0068691519999999999, 2024-04-20T01:47:32.801 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:32.801 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:32.801 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:32.801 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:32.801 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:32.801 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 140 2024-04-20T01:47:32.801 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.801 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:32.802 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.802 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:32.802 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.669993+0000", 2024-04-20T01:47:32.802 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.802 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.802 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.802 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:32.802 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.669993+0000", 2024-04-20T01:47:32.802 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.802 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.802 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.802 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:32.802 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.669993+0000", 2024-04-20T01:47:32.802 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:47:32.803 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.803 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.803 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:32.803 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.670042+0000", 2024-04-20T01:47:32.803 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.9444999999999998e-05 2024-04-20T01:47:32.803 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.803 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.803 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:32.803 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.670046+0000", 2024-04-20T01:47:32.803 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.1339999999999997e-06 2024-04-20T01:47:32.803 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.803 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.803 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:32.803 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.670050+0000", 2024-04-20T01:47:32.803 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.1669999999999999e-06 2024-04-20T01:47:32.804 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.804 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.804 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.804 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.675535+0000", 2024-04-20T01:47:32.804 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0054852490000000002 2024-04-20T01:47:32.804 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.804 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.804 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:32.804 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.675617+0000", 2024-04-20T01:47:32.804 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.2583000000000005e-05 2024-04-20T01:47:32.804 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.804 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.804 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:32.804 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.675690+0000", 2024-04-20T01:47:32.804 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.2562000000000001e-05 2024-04-20T01:47:32.805 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.805 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.805 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:32.805 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.676470+0000", 2024-04-20T01:47:32.805 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00078030799999999996 2024-04-20T01:47:32.805 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.805 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.805 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:32.805 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.676822+0000", 2024-04-20T01:47:32.805 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00035168200000000003 2024-04-20T01:47:32.805 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.805 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.805 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:32.805 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.676845+0000", 2024-04-20T01:47:32.806 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2782e-05 2024-04-20T01:47:32.806 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.806 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.806 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:32.806 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.676862+0000", 2024-04-20T01:47:32.806 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7441000000000001e-05 2024-04-20T01:47:32.806 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.806 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.806 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.806 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.806 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.806 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-20T01:47:32.806 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:17.670053+0000", 2024-04-20T01:47:32.806 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 15.086541990000001, 2024-04-20T01:47:32.806 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0073552490000000003, 2024-04-20T01:47:32.807 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:32.807 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:32.807 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:32.807 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:32.807 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:32.807 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 141 2024-04-20T01:47:32.807 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.807 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:32.807 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.807 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:32.807 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.670053+0000", 2024-04-20T01:47:32.807 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.807 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.807 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.808 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:32.808 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.670053+0000", 2024-04-20T01:47:32.808 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.808 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.808 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.808 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:32.808 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.670052+0000", 2024-04-20T01:47:32.808 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:32.808 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.808 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.808 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:32.808 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.670055+0000", 2024-04-20T01:47:32.808 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2060000000000001e-06 2024-04-20T01:47:32.808 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.808 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.809 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:32.809 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.670058+0000", 2024-04-20T01:47:32.809 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.864e-06 2024-04-20T01:47:32.809 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.809 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.809 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:32.809 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.670060+0000", 2024-04-20T01:47:32.809 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.48e-06 2024-04-20T01:47:32.809 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.809 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.809 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.809 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.676160+0000", 2024-04-20T01:47:32.809 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0060996009999999996 2024-04-20T01:47:32.809 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.810 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.810 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:32.810 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.676205+0000", 2024-04-20T01:47:32.810 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5210000000000003e-05 2024-04-20T01:47:32.810 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.810 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.810 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:32.810 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.676277+0000", 2024-04-20T01:47:32.810 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.2337999999999998e-05 2024-04-20T01:47:32.810 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.810 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.810 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:32.810 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.677073+0000", 2024-04-20T01:47:32.810 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000795549 2024-04-20T01:47:32.810 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.811 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.811 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:32.811 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.677363+0000", 2024-04-20T01:47:32.811 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00029028700000000002 2024-04-20T01:47:32.811 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.811 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.811 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:32.811 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.677389+0000", 2024-04-20T01:47:32.811 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5687999999999999e-05 2024-04-20T01:47:32.811 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.811 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.811 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:32.811 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.677408+0000", 2024-04-20T01:47:32.811 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9293e-05 2024-04-20T01:47:32.812 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.812 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.812 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.812 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.812 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.812 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-20T01:47:32.812 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:17.792472+0000", 2024-04-20T01:47:32.812 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.964123126, 2024-04-20T01:47:32.812 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0075204210000000002, 2024-04-20T01:47:32.812 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:32.812 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:32.812 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:32.812 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:32.812 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:32.813 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 165 2024-04-20T01:47:32.813 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.813 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:32.813 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.813 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:32.813 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.792472+0000", 2024-04-20T01:47:32.813 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.813 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.813 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.813 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:32.813 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.792472+0000", 2024-04-20T01:47:32.813 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.813 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.813 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.814 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:32.814 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.792471+0000", 2024-04-20T01:47:32.814 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:32.814 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.814 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.814 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:32.814 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.792475+0000", 2024-04-20T01:47:32.814 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.5939999999999998e-06 2024-04-20T01:47:32.814 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.814 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.814 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:32.814 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.792480+0000", 2024-04-20T01:47:32.814 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.5990000000000001e-06 2024-04-20T01:47:32.814 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.814 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.815 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:32.815 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.792486+0000", 2024-04-20T01:47:32.815 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.1309999999999997e-06 2024-04-20T01:47:32.815 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.815 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.815 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.815 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.797562+0000", 2024-04-20T01:47:32.815 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0050766580000000004 2024-04-20T01:47:32.815 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.815 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.815 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:32.815 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.797610+0000", 2024-04-20T01:47:32.815 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.8198000000000002e-05 2024-04-20T01:47:32.815 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.816 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.816 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:32.816 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.797699+0000", 2024-04-20T01:47:32.816 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8951999999999998e-05 2024-04-20T01:47:32.816 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.816 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.816 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:32.816 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.798836+0000", 2024-04-20T01:47:32.816 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0011367899999999999 2024-04-20T01:47:32.816 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.816 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.816 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:32.816 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.799954+0000", 2024-04-20T01:47:32.816 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0011176890000000001 2024-04-20T01:47:32.817 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.817 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.817 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:32.817 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.799976+0000", 2024-04-20T01:47:32.817 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1875999999999999e-05 2024-04-20T01:47:32.817 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.817 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.817 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:32.817 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.799992+0000", 2024-04-20T01:47:32.817 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6328999999999999e-05 2024-04-20T01:47:32.817 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.817 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.817 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.817 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.818 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.818 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:187 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-20T01:47:32.818 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:17.900472+0000", 2024-04-20T01:47:32.818 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.856123009000001, 2024-04-20T01:47:32.818 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0065913259999999998, 2024-04-20T01:47:32.818 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:32.818 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:32.818 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:32.818 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:32.818 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:32.818 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 187 2024-04-20T01:47:32.818 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.818 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:32.818 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.818 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:32.819 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.900472+0000", 2024-04-20T01:47:32.819 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.819 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.819 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.819 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:32.819 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.900472+0000", 2024-04-20T01:47:32.819 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.819 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.819 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.819 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:32.819 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.900471+0000", 2024-04-20T01:47:32.819 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:32.819 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.819 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.820 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:32.820 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.900475+0000", 2024-04-20T01:47:32.820 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.569e-06 2024-04-20T01:47:32.820 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.820 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.820 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:32.820 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.900481+0000", 2024-04-20T01:47:32.820 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.665e-06 2024-04-20T01:47:32.820 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.820 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.820 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:32.820 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.900487+0000", 2024-04-20T01:47:32.820 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.2979999999999997e-06 2024-04-20T01:47:32.820 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.820 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.821 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.821 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.904645+0000", 2024-04-20T01:47:32.821 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0041580020000000001 2024-04-20T01:47:32.821 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.821 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.821 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:32.821 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.904689+0000", 2024-04-20T01:47:32.821 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.3650999999999999e-05 2024-04-20T01:47:32.821 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.821 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.821 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:32.821 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.904769+0000", 2024-04-20T01:47:32.821 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.0140000000000002e-05 2024-04-20T01:47:32.821 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.821 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.822 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:32.822 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.906239+0000", 2024-04-20T01:47:32.822 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001470044 2024-04-20T01:47:32.822 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.822 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.822 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:32.822 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.907018+0000", 2024-04-20T01:47:32.822 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00077906700000000004 2024-04-20T01:47:32.822 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.822 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.822 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:32.822 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.907042+0000", 2024-04-20T01:47:32.822 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.442e-05 2024-04-20T01:47:32.823 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.823 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.823 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:32.823 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.907063+0000", 2024-04-20T01:47:32.823 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.084e-05 2024-04-20T01:47:32.823 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.823 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.823 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.823 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.823 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.823 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:188 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-20T01:47:32.823 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:17.900492+0000", 2024-04-20T01:47:32.823 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.856102391, 2024-04-20T01:47:32.823 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0093460699999999997, 2024-04-20T01:47:32.824 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:32.824 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:32.824 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:32.824 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:32.824 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:32.824 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 188 2024-04-20T01:47:32.824 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.824 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:32.824 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.824 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:32.824 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.900492+0000", 2024-04-20T01:47:32.824 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.824 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.824 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.825 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:32.825 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.900492+0000", 2024-04-20T01:47:32.825 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.825 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.825 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.825 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:32.825 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.900492+0000", 2024-04-20T01:47:32.825 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:32.825 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.825 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.825 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:32.825 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.900544+0000", 2024-04-20T01:47:32.825 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.1610000000000002e-05 2024-04-20T01:47:32.825 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.825 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.826 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:32.826 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.900551+0000", 2024-04-20T01:47:32.826 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.9229999999999996e-06 2024-04-20T01:47:32.826 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.826 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.826 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:32.826 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.900556+0000", 2024-04-20T01:47:32.826 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.4489999999999998e-06 2024-04-20T01:47:32.826 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.826 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.826 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.826 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.904989+0000", 2024-04-20T01:47:32.826 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0044334520000000001 2024-04-20T01:47:32.826 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.827 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.827 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:32.827 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.905035+0000", 2024-04-20T01:47:32.827 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5318999999999998e-05 2024-04-20T01:47:32.827 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.827 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.827 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:32.827 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.905107+0000", 2024-04-20T01:47:32.827 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.1808000000000001e-05 2024-04-20T01:47:32.827 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.827 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.827 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:32.827 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.906980+0000", 2024-04-20T01:47:32.827 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001872987 2024-04-20T01:47:32.828 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.828 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.828 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:32.828 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.909785+0000", 2024-04-20T01:47:32.828 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.002805779 2024-04-20T01:47:32.828 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.828 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.828 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:32.828 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.909816+0000", 2024-04-20T01:47:32.828 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.1089e-05 2024-04-20T01:47:32.828 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.828 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.828 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:32.828 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.909838+0000", 2024-04-20T01:47:32.828 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2022e-05 2024-04-20T01:47:32.829 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.829 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.829 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.829 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.829 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.829 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:189 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-20T01:47:32.829 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:17.900562+0000", 2024-04-20T01:47:32.829 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.856032867, 2024-04-20T01:47:32.829 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0093152240000000004, 2024-04-20T01:47:32.829 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:32.829 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:32.829 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:32.829 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:32.829 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:32.830 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 189 2024-04-20T01:47:32.830 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.830 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:32.830 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.830 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:32.830 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.900562+0000", 2024-04-20T01:47:32.830 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.830 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.830 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.830 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:32.830 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.900562+0000", 2024-04-20T01:47:32.830 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.830 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.830 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.830 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:32.831 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.900561+0000", 2024-04-20T01:47:32.831 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:32.831 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.831 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.831 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:32.831 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.900567+0000", 2024-04-20T01:47:32.831 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3890000000000004e-06 2024-04-20T01:47:32.831 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.831 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.831 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:32.831 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.900573+0000", 2024-04-20T01:47:32.831 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.789e-06 2024-04-20T01:47:32.831 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.831 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.832 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:32.832 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.900577+0000", 2024-04-20T01:47:32.832 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.7480000000000001e-06 2024-04-20T01:47:32.832 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.832 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.832 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.832 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.905746+0000", 2024-04-20T01:47:32.832 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0051686659999999997 2024-04-20T01:47:32.832 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.832 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.832 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:32.832 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.905792+0000", 2024-04-20T01:47:32.832 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5586e-05 2024-04-20T01:47:32.832 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.832 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.833 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:32.833 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.905863+0000", 2024-04-20T01:47:32.833 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.1656000000000006e-05 2024-04-20T01:47:32.833 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.833 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.833 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:32.833 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.907793+0000", 2024-04-20T01:47:32.833 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0019299230000000001 2024-04-20T01:47:32.833 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.833 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.833 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:32.833 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.909842+0000", 2024-04-20T01:47:32.833 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.002049188 2024-04-20T01:47:32.833 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.833 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.834 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:32.834 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.909862+0000", 2024-04-20T01:47:32.834 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9724e-05 2024-04-20T01:47:32.834 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.834 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.834 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:32.834 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.909877+0000", 2024-04-20T01:47:32.834 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4983999999999999e-05 2024-04-20T01:47:32.834 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.834 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.834 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.834 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.834 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.834 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:203 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-20T01:47:32.835 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:19.165916+0000", 2024-04-20T01:47:32.835 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 13.590678642, 2024-04-20T01:47:32.835 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0066340219999999998, 2024-04-20T01:47:32.835 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:32.835 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:32.835 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:32.835 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:32.835 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:32.835 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 203 2024-04-20T01:47:32.835 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.835 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:32.835 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.835 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:32.835 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.165916+0000", 2024-04-20T01:47:32.835 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.836 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.836 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.836 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:32.836 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.165916+0000", 2024-04-20T01:47:32.836 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.836 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.836 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.836 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:32.836 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.165916+0000", 2024-04-20T01:47:32.836 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:32.836 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.836 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.836 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:32.836 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166073+0000", 2024-04-20T01:47:32.836 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00015743500000000001 2024-04-20T01:47:32.837 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.837 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.837 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:32.837 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166085+0000", 2024-04-20T01:47:32.837 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1708999999999999e-05 2024-04-20T01:47:32.837 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.837 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.837 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:32.837 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166095+0000", 2024-04-20T01:47:32.837 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.9159999999999996e-06 2024-04-20T01:47:32.837 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.837 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.837 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.837 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166738+0000", 2024-04-20T01:47:32.837 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00064347500000000001 2024-04-20T01:47:32.838 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.838 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.838 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:32.838 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166784+0000", 2024-04-20T01:47:32.838 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.6217999999999997e-05 2024-04-20T01:47:32.838 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.838 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.838 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:32.838 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166860+0000", 2024-04-20T01:47:32.838 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.6106999999999995e-05 2024-04-20T01:47:32.838 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.838 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.838 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:32.838 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.170548+0000", 2024-04-20T01:47:32.839 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0036874080000000001 2024-04-20T01:47:32.839 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.839 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.839 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:32.839 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.172489+0000", 2024-04-20T01:47:32.839 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0019416150000000001 2024-04-20T01:47:32.839 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.839 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.839 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:32.839 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.172521+0000", 2024-04-20T01:47:32.839 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.1346000000000001e-05 2024-04-20T01:47:32.839 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.839 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.839 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:32.839 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.172550+0000", 2024-04-20T01:47:32.840 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9342000000000001e-05 2024-04-20T01:47:32.840 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.840 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.840 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.840 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.840 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.840 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:205 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-20T01:47:32.840 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:19.166125+0000", 2024-04-20T01:47:32.840 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 13.590469689000001, 2024-04-20T01:47:32.840 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0065369790000000001, 2024-04-20T01:47:32.840 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:32.840 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:32.840 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:32.840 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:32.840 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:32.841 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 205 2024-04-20T01:47:32.841 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.841 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:32.841 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.841 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:32.841 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166125+0000", 2024-04-20T01:47:32.841 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.841 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.841 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.841 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:32.841 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166125+0000", 2024-04-20T01:47:32.841 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.841 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.841 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.842 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:32.842 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166125+0000", 2024-04-20T01:47:32.842 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:32.842 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.842 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.842 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:32.842 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166128+0000", 2024-04-20T01:47:32.842 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.4529999999999998e-06 2024-04-20T01:47:32.842 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.842 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.842 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:32.842 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166134+0000", 2024-04-20T01:47:32.842 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.8810000000000001e-06 2024-04-20T01:47:32.842 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.842 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.843 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:32.843 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166149+0000", 2024-04-20T01:47:32.843 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5068999999999999e-05 2024-04-20T01:47:32.843 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.843 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.843 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.843 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.167641+0000", 2024-04-20T01:47:32.843 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0014923219999999999 2024-04-20T01:47:32.843 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.843 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.843 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:32.843 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.167687+0000", 2024-04-20T01:47:32.843 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5408999999999998e-05 2024-04-20T01:47:32.843 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.843 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.843 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:32.844 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.167761+0000", 2024-04-20T01:47:32.844 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.4561999999999995e-05 2024-04-20T01:47:32.844 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.844 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.844 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:32.844 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.170771+0000", 2024-04-20T01:47:32.844 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.003009243 2024-04-20T01:47:32.844 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.844 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.844 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:32.844 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.172611+0000", 2024-04-20T01:47:32.844 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0018405610000000001 2024-04-20T01:47:32.844 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.844 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.844 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:32.845 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.172634+0000", 2024-04-20T01:47:32.845 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2651e-05 2024-04-20T01:47:32.845 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.845 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.845 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:32.845 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.172662+0000", 2024-04-20T01:47:32.845 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8262000000000001e-05 2024-04-20T01:47:32.845 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.845 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.845 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.845 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.845 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.845 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:206 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-20T01:47:32.845 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:19.166155+0000", 2024-04-20T01:47:32.845 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 13.590440072, 2024-04-20T01:47:32.846 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0065613249999999998, 2024-04-20T01:47:32.846 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:32.846 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:32.846 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:32.846 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:32.846 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:32.846 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 206 2024-04-20T01:47:32.846 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.846 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:32.846 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.846 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:32.846 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166155+0000", 2024-04-20T01:47:32.846 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.846 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.847 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.847 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:32.847 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166155+0000", 2024-04-20T01:47:32.847 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.847 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.847 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.847 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:32.847 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166154+0000", 2024-04-20T01:47:32.847 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:32.847 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.847 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.847 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:32.847 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166223+0000", 2024-04-20T01:47:32.847 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.8498999999999996e-05 2024-04-20T01:47:32.847 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.847 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.848 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:32.848 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166231+0000", 2024-04-20T01:47:32.848 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.2300000000000008e-06 2024-04-20T01:47:32.848 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.848 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.848 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:32.848 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166238+0000", 2024-04-20T01:47:32.848 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.1219999999999997e-06 2024-04-20T01:47:32.848 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.848 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.848 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.848 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.167979+0000", 2024-04-20T01:47:32.848 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0017406640000000001 2024-04-20T01:47:32.848 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.849 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.849 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:32.849 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.168028+0000", 2024-04-20T01:47:32.849 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.8779000000000002e-05 2024-04-20T01:47:32.849 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.849 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.849 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:32.849 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.168102+0000", 2024-04-20T01:47:32.849 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.4703000000000006e-05 2024-04-20T01:47:32.849 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.849 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.849 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:32.849 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.170793+0000", 2024-04-20T01:47:32.849 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0026903209999999999 2024-04-20T01:47:32.849 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.849 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.850 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:32.850 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.172666+0000", 2024-04-20T01:47:32.850 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00187382 2024-04-20T01:47:32.850 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.850 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.850 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:32.850 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.172688+0000", 2024-04-20T01:47:32.850 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1359e-05 2024-04-20T01:47:32.850 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.850 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.850 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:32.850 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.172716+0000", 2024-04-20T01:47:32.850 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8238999999999998e-05 2024-04-20T01:47:32.850 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.851 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.851 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.851 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.851 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.851 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:207 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-04-20T01:47:32.851 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:19.166245+0000", 2024-04-20T01:47:32.851 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 13.590349940999999, 2024-04-20T01:47:32.851 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0065250339999999999, 2024-04-20T01:47:32.851 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:32.851 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:32.851 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:32.851 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:32.851 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:32.851 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 207 2024-04-20T01:47:32.851 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.852 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:32.852 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.852 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:32.852 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166245+0000", 2024-04-20T01:47:32.852 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.852 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.852 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.852 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:32.852 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166245+0000", 2024-04-20T01:47:32.852 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:32.852 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.852 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.852 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:32.852 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166244+0000", 2024-04-20T01:47:32.852 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:32.852 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.853 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.853 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:32.853 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166248+0000", 2024-04-20T01:47:32.853 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.6480000000000001e-06 2024-04-20T01:47:32.853 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.853 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.853 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:32.853 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166255+0000", 2024-04-20T01:47:32.853 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.1640000000000004e-06 2024-04-20T01:47:32.853 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.853 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.853 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:32.853 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.166261+0000", 2024-04-20T01:47:32.853 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.7559999999999996e-06 2024-04-20T01:47:32.853 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.854 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.854 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:32.854 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.168606+0000", 2024-04-20T01:47:32.854 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.002345055 2024-04-20T01:47:32.854 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.854 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.854 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:32.854 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.168655+0000", 2024-04-20T01:47:32.854 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.8937000000000003e-05 2024-04-20T01:47:32.854 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.854 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.854 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:32.854 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.168730+0000", 2024-04-20T01:47:32.854 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.5012000000000006e-05 2024-04-20T01:47:32.854 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.855 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.855 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:32.855 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.170809+0000", 2024-04-20T01:47:32.855 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0020786860000000002 2024-04-20T01:47:32.855 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.855 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.855 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:32.855 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.172719+0000", 2024-04-20T01:47:32.855 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0019108619999999999 2024-04-20T01:47:32.855 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.855 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.855 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:32.855 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.172740+0000", 2024-04-20T01:47:32.855 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0696e-05 2024-04-20T01:47:32.855 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:32.856 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:32.856 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:32.856 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:19.172770+0000", 2024-04-20T01:47:32.856 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9757999999999999e-05 2024-04-20T01:47:32.856 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.856 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.856 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.856 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:32.856 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:32.856 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:32.858 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-20T01:47:32.858 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:47:32.871+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:47:32.958 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:32.973+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:47:32.997 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:47:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:47:32.998 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:33.008 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-20T01:47:33.060 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:33.074+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:47:33.135 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:33.136 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:47:33.136 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 30, 2024-04-20T01:47:33.136 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-20T01:47:33.136 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:33.146 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-20T01:47:33.189 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:33.203+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:33.260 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:47:33.275+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:47:33.272 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:33.273 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-20T01:47:33.273 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-20T01:47:33.273 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [] 2024-04-20T01:47:33.273 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:33.282 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-20T01:47:33.370 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:33.384+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:33.454 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:33.454 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:47:33.454 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:47:33.454 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:33.462 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:33.477+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:33.464 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-20T01:47:33.563 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:33.577+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:33.631 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:33.631 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:47:33.632 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 30, 2024-04-20T01:47:33.632 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-20T01:47:33.632 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:33.643 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-20T01:47:33.663 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:33.678+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:33.708 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-20T01:47:33.708 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-20T01:47:33.722 DEBUG:teuthology.orchestra.run.smithi165:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0 2024-04-20T01:47:33.758 INFO:tasks.rados.rados.0.smithi165.stdout:adding op weight read -> 100 2024-04-20T01:47:33.758 INFO:tasks.rados.rados.0.smithi165.stdout:adding op weight delete -> 50 2024-04-20T01:47:33.758 INFO:tasks.rados.rados.0.smithi165.stdout:adding op weight snap_create -> 50 2024-04-20T01:47:33.758 INFO:tasks.rados.rados.0.smithi165.stdout:adding op weight snap_remove -> 50 2024-04-20T01:47:33.758 INFO:tasks.rados.rados.0.smithi165.stdout:adding op weight rollback -> 50 2024-04-20T01:47:33.759 INFO:tasks.rados.rados.0.smithi165.stdout:adding op weight setattr -> 25 2024-04-20T01:47:33.759 INFO:tasks.rados.rados.0.smithi165.stdout:adding op weight rmattr -> 25 2024-04-20T01:47:33.759 INFO:tasks.rados.rados.0.smithi165.stdout:adding op weight copy_from -> 50 2024-04-20T01:47:33.759 INFO:tasks.rados.rados.0.smithi165.stdout:adding op weight append -> 100 2024-04-20T01:47:33.759 INFO:tasks.rados.rados.0.smithi165.stdout:ceph version 18.2.2-1252-g15eecde2 (15eecde2bab9803889d929c7e423d07970d8cbed) reef (stable) 2024-04-20T01:47:33.759 INFO:tasks.rados.rados.0.smithi165.stdout:Configuration: 2024-04-20T01:47:33.759 INFO:tasks.rados.rados.0.smithi165.stdout: Number of operations: 400 2024-04-20T01:47:33.759 INFO:tasks.rados.rados.0.smithi165.stdout: Number of objects: 50 2024-04-20T01:47:33.759 INFO:tasks.rados.rados.0.smithi165.stdout: Max in flight operations: 16 2024-04-20T01:47:33.759 INFO:tasks.rados.rados.0.smithi165.stdout: Object size (in bytes): 4000000 2024-04-20T01:47:33.759 INFO:tasks.rados.rados.0.smithi165.stdout: Write stride min: 400000 2024-04-20T01:47:33.759 INFO:tasks.rados.rados.0.smithi165.stdout: Write stride max: 800000 2024-04-20T01:47:33.764 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:47:33.778+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:47:33.773 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:33.773 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-20T01:47:33.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-20T01:47:33.773 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [] 2024-04-20T01:47:33.773 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:33.784 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-20T01:47:33.868 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:33.882+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:47:33.953 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:33.953 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:47:33.953 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:47:33.953 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:33.963 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-20T01:47:33.966 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:33.981+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:34.068 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:34.082+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:47:34.150 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:34.150 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:47:34.150 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 30, 2024-04-20T01:47:34.150 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-20T01:47:34.151 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:34.151 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-20T01:47:34.195 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:34.209+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:34.270 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:34.284+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:47:34.279 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:34.280 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-20T01:47:34.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-20T01:47:34.280 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [] 2024-04-20T01:47:34.280 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:34.290 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-20T01:47:34.371 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:47:34.385+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:47:34.395 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-20T01:47:34.457 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:34.458 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:47:34.458 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:47:34.458 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:34.467 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-20T01:47:34.512 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:34.526+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:34.565 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:47:34.573 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:34.587+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:34.586 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:47:34.596 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:34.596 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:47:34.596 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 30, 2024-04-20T01:47:34.596 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-20T01:47:34.596 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:34.606 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-20T01:47:34.606 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:47:34.628 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:47:34.647 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:47:34.668 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:47:34.674 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:34.689+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:47:34.688 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:47:34.709 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:47:34.722 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-20T01:47:34.729 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:47:34.736 INFO:tasks.rados.rados.0.smithi165.stdout:1: write initial oid 1 2024-04-20T01:47:34.736 INFO:tasks.rados.rados.0.smithi165.stdout:1: seq_num 0 ranges {0=32768} 2024-04-20T01:47:34.736 INFO:tasks.rados.rados.0.smithi165.stdout:1: writing smithi16579500-1 from 0 to 32768 tid 1 2024-04-20T01:47:34.736 INFO:tasks.rados.rados.0.smithi165.stdout:2: write initial oid 2 2024-04-20T01:47:34.736 INFO:tasks.rados.rados.0.smithi165.stdout:2: seq_num 1 ranges {0=32768} 2024-04-20T01:47:34.736 INFO:tasks.rados.rados.0.smithi165.stdout:2: writing smithi16579500-2 from 0 to 32768 tid 1 2024-04-20T01:47:34.736 INFO:tasks.rados.rados.0.smithi165.stdout:3: write initial oid 3 2024-04-20T01:47:34.736 INFO:tasks.rados.rados.0.smithi165.stdout:3: seq_num 2 ranges {0=49152} 2024-04-20T01:47:34.736 INFO:tasks.rados.rados.0.smithi165.stdout:3: writing smithi16579500-3 from 0 to 49152 tid 1 2024-04-20T01:47:34.736 INFO:tasks.rados.rados.0.smithi165.stdout:4: write initial oid 4 2024-04-20T01:47:34.736 INFO:tasks.rados.rados.0.smithi165.stdout:4: seq_num 3 ranges {0=65536} 2024-04-20T01:47:34.737 INFO:tasks.rados.rados.0.smithi165.stdout:4: writing smithi16579500-4 from 0 to 65536 tid 1 2024-04-20T01:47:34.737 INFO:tasks.rados.rados.0.smithi165.stdout:5: write initial oid 5 2024-04-20T01:47:34.737 INFO:tasks.rados.rados.0.smithi165.stdout:5: seq_num 4 ranges {0=81920} 2024-04-20T01:47:34.737 INFO:tasks.rados.rados.0.smithi165.stdout:5: writing smithi16579500-5 from 0 to 81920 tid 1 2024-04-20T01:47:34.737 INFO:tasks.rados.rados.0.smithi165.stdout:6: write initial oid 6 2024-04-20T01:47:34.737 INFO:tasks.rados.rados.0.smithi165.stdout:6: seq_num 5 ranges {0=98304} 2024-04-20T01:47:34.738 INFO:tasks.rados.rados.0.smithi165.stdout:6: writing smithi16579500-6 from 0 to 98304 tid 1 2024-04-20T01:47:34.738 INFO:tasks.rados.rados.0.smithi165.stdout:7: write initial oid 7 2024-04-20T01:47:34.738 INFO:tasks.rados.rados.0.smithi165.stdout:7: seq_num 6 ranges {0=114688} 2024-04-20T01:47:34.738 INFO:tasks.rados.rados.0.smithi165.stdout:7: writing smithi16579500-7 from 0 to 114688 tid 1 2024-04-20T01:47:34.739 INFO:tasks.rados.rados.0.smithi165.stdout:8: write initial oid 8 2024-04-20T01:47:34.739 INFO:tasks.rados.rados.0.smithi165.stdout:8: seq_num 7 ranges {0=131072} 2024-04-20T01:47:34.739 INFO:tasks.rados.rados.0.smithi165.stdout:8: writing smithi16579500-8 from 0 to 131072 tid 1 2024-04-20T01:47:34.740 INFO:tasks.rados.rados.0.smithi165.stdout:9: write initial oid 9 2024-04-20T01:47:34.740 INFO:tasks.rados.rados.0.smithi165.stdout:9: seq_num 8 ranges {0=147456} 2024-04-20T01:47:34.740 INFO:tasks.rados.rados.0.smithi165.stdout:9: writing smithi16579500-9 from 0 to 147456 tid 1 2024-04-20T01:47:34.742 INFO:tasks.rados.rados.0.smithi165.stdout:10: write initial oid 10 2024-04-20T01:47:34.742 INFO:tasks.rados.rados.0.smithi165.stdout:10: seq_num 9 ranges {0=163840} 2024-04-20T01:47:34.742 INFO:tasks.rados.rados.0.smithi165.stdout:10: writing smithi16579500-10 from 0 to 163840 tid 1 2024-04-20T01:47:34.742 INFO:tasks.rados.rados.0.smithi165.stdout:11: write initial oid 11 2024-04-20T01:47:34.742 INFO:tasks.rados.rados.0.smithi165.stdout:11: seq_num 10 ranges {0=180224} 2024-04-20T01:47:34.743 INFO:tasks.rados.rados.0.smithi165.stdout:11: writing smithi16579500-11 from 0 to 180224 tid 1 2024-04-20T01:47:34.744 INFO:tasks.rados.rados.0.smithi165.stdout:12: write initial oid 12 2024-04-20T01:47:34.744 INFO:tasks.rados.rados.0.smithi165.stdout:12: seq_num 11 ranges {0=196608} 2024-04-20T01:47:34.745 INFO:tasks.rados.rados.0.smithi165.stdout:12: writing smithi16579500-12 from 0 to 196608 tid 1 2024-04-20T01:47:34.745 INFO:tasks.rados.rados.0.smithi165.stdout:13: write initial oid 13 2024-04-20T01:47:34.745 INFO:tasks.rados.rados.0.smithi165.stdout:13: seq_num 12 ranges {0=212992} 2024-04-20T01:47:34.746 INFO:tasks.rados.rados.0.smithi165.stdout:13: writing smithi16579500-13 from 0 to 212992 tid 1 2024-04-20T01:47:34.746 INFO:tasks.rados.rados.0.smithi165.stdout:14: write initial oid 14 2024-04-20T01:47:34.746 INFO:tasks.rados.rados.0.smithi165.stdout:14: seq_num 13 ranges {0=229376} 2024-04-20T01:47:34.752 INFO:tasks.rados.rados.0.smithi165.stdout:14: writing smithi16579500-14 from 0 to 229376 tid 1 2024-04-20T01:47:34.752 INFO:tasks.rados.rados.0.smithi165.stdout:15: write initial oid 15 2024-04-20T01:47:34.752 INFO:tasks.rados.rados.0.smithi165.stdout:15: seq_num 14 ranges {0=245760} 2024-04-20T01:47:34.752 INFO:tasks.rados.rados.0.smithi165.stdout:15: writing smithi16579500-15 from 0 to 245760 tid 1 2024-04-20T01:47:34.752 INFO:tasks.rados.rados.0.smithi165.stdout:16: write initial oid 16 2024-04-20T01:47:34.752 INFO:tasks.rados.rados.0.smithi165.stdout:16: seq_num 15 ranges {0=262144} 2024-04-20T01:47:34.752 INFO:tasks.rados.rados.0.smithi165.stdout:16: writing smithi16579500-16 from 0 to 262144 tid 1 2024-04-20T01:47:34.752 INFO:tasks.rados.rados.0.smithi165.stdout: waiting on 16 2024-04-20T01:47:34.752 INFO:tasks.rados.rados.0.smithi165.stdout:1: finishing write tid 1 to smithi16579500-1 2024-04-20T01:47:34.752 INFO:tasks.rados.rados.0.smithi165.stdout:7: finishing write tid 1 to smithi16579500-7 2024-04-20T01:47:34.752 INFO:tasks.rados.rados.0.smithi165.stdout:7: finishing write tid 2 to smithi16579500-7 2024-04-20T01:47:34.753 INFO:tasks.rados.rados.0.smithi165.stdout:4: finishing write tid 1 to smithi16579500-4 2024-04-20T01:47:34.753 INFO:tasks.rados.rados.0.smithi165.stdout:4: finishing write tid 2 to smithi16579500-4 2024-04-20T01:47:34.753 INFO:tasks.rados.rados.0.smithi165.stdout:1: finishing write tid 2 to smithi16579500-1 2024-04-20T01:47:34.753 INFO:tasks.rados.rados.0.smithi165.stdout:7: finishing write tid 3 to smithi16579500-7 2024-04-20T01:47:34.753 INFO:tasks.rados.rados.0.smithi165.stdout:7: oid 7 updating version 0 to 1 2024-04-20T01:47:34.753 INFO:tasks.rados.rados.0.smithi165.stdout:7: oid 7 updating version 1 to 2 2024-04-20T01:47:34.753 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 7 v 2 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-04-20T01:47:34.753 INFO:tasks.rados.rados.0.smithi165.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2024-04-20T01:47:34.753 INFO:tasks.rados.rados.0.smithi165.stdout:6: finishing write tid 1 to smithi16579500-6 2024-04-20T01:47:34.754 INFO:tasks.rados.rados.0.smithi165.stdout:6: finishing write tid 2 to smithi16579500-6 2024-04-20T01:47:34.754 INFO:tasks.rados.rados.0.smithi165.stdout:4: finishing write tid 3 to smithi16579500-4 2024-04-20T01:47:34.754 INFO:tasks.rados.rados.0.smithi165.stdout:4: oid 4 updating version 0 to 1 2024-04-20T01:47:34.754 INFO:tasks.rados.rados.0.smithi165.stdout:4: oid 4 updating version 1 to 2 2024-04-20T01:47:34.754 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 4 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-04-20T01:47:34.754 INFO:tasks.rados.rados.0.smithi165.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2024-04-20T01:47:34.754 INFO:tasks.rados.rados.0.smithi165.stdout:1: finishing write tid 3 to smithi16579500-1 2024-04-20T01:47:34.754 INFO:tasks.rados.rados.0.smithi165.stdout:1: oid 1 updating version 0 to 1 2024-04-20T01:47:34.754 INFO:tasks.rados.rados.0.smithi165.stdout:1: oid 1 updating version 1 to 2 2024-04-20T01:47:34.754 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 1 v 2 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-04-20T01:47:34.754 INFO:tasks.rados.rados.0.smithi165.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2024-04-20T01:47:34.754 INFO:tasks.rados.rados.0.smithi165.stdout:1: done (15 left) 2024-04-20T01:47:34.754 INFO:tasks.rados.rados.0.smithi165.stdout:4: done (14 left) 2024-04-20T01:47:34.755 INFO:tasks.rados.rados.0.smithi165.stdout:7: done (13 left) 2024-04-20T01:47:34.755 INFO:tasks.rados.rados.0.smithi165.stdout:17: write initial oid 17 2024-04-20T01:47:34.755 INFO:tasks.rados.rados.0.smithi165.stdout:17: seq_num 16 ranges {0=278528} 2024-04-20T01:47:34.755 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.0 to 1 (10000) 2024-04-20T01:47:34.756 INFO:tasks.rados.rados.0.smithi165.stdout:17: writing smithi16579500-17 from 0 to 278528 tid 1 2024-04-20T01:47:34.756 INFO:tasks.rados.rados.0.smithi165.stdout:18: write initial oid 18 2024-04-20T01:47:34.756 INFO:tasks.rados.rados.0.smithi165.stdout:18: seq_num 17 ranges {0=294912} 2024-04-20T01:47:34.756 INFO:tasks.rados.rados.0.smithi165.stdout:18: writing smithi16579500-18 from 0 to 294912 tid 1 2024-04-20T01:47:34.756 INFO:tasks.rados.rados.0.smithi165.stdout:19: write initial oid 19 2024-04-20T01:47:34.756 INFO:tasks.rados.rados.0.smithi165.stdout:19: seq_num 18 ranges {0=311296} 2024-04-20T01:47:34.762 INFO:tasks.rados.rados.0.smithi165.stdout:19: writing smithi16579500-19 from 0 to 311296 tid 1 2024-04-20T01:47:34.762 INFO:tasks.rados.rados.0.smithi165.stdout: waiting on 16 2024-04-20T01:47:34.762 INFO:tasks.rados.rados.0.smithi165.stdout:8: finishing write tid 1 to smithi16579500-8 2024-04-20T01:47:34.762 INFO:tasks.rados.rados.0.smithi165.stdout:3: finishing write tid 1 to smithi16579500-3 2024-04-20T01:47:34.762 INFO:tasks.rados.rados.0.smithi165.stdout:3: finishing write tid 2 to smithi16579500-3 2024-04-20T01:47:34.762 INFO:tasks.rados.rados.0.smithi165.stdout:2: finishing write tid 1 to smithi16579500-2 2024-04-20T01:47:34.762 INFO:tasks.rados.rados.0.smithi165.stdout:8: finishing write tid 2 to smithi16579500-8 2024-04-20T01:47:34.763 INFO:tasks.rados.rados.0.smithi165.stdout:2: finishing write tid 2 to smithi16579500-2 2024-04-20T01:47:34.763 INFO:tasks.rados.rados.0.smithi165.stdout:6: finishing write tid 3 to smithi16579500-6 2024-04-20T01:47:34.763 INFO:tasks.rados.rados.0.smithi165.stdout:6: oid 6 updating version 0 to 1 2024-04-20T01:47:34.763 INFO:tasks.rados.rados.0.smithi165.stdout:6: oid 6 updating version 1 to 2 2024-04-20T01:47:34.763 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 6 v 2 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-04-20T01:47:34.763 INFO:tasks.rados.rados.0.smithi165.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2024-04-20T01:47:34.763 INFO:tasks.rados.rados.0.smithi165.stdout:9: finishing write tid 1 to smithi16579500-9 2024-04-20T01:47:34.763 INFO:tasks.rados.rados.0.smithi165.stdout:9: finishing write tid 2 to smithi16579500-9 2024-04-20T01:47:34.763 INFO:tasks.rados.rados.0.smithi165.stdout:11: finishing write tid 1 to smithi16579500-11 2024-04-20T01:47:34.763 INFO:tasks.rados.rados.0.smithi165.stdout:11: finishing write tid 2 to smithi16579500-11 2024-04-20T01:47:34.763 INFO:tasks.rados.rados.0.smithi165.stdout:3: finishing write tid 3 to smithi16579500-3 2024-04-20T01:47:34.763 INFO:tasks.rados.rados.0.smithi165.stdout:3: oid 3 updating version 0 to 3 2024-04-20T01:47:34.763 INFO:tasks.rados.rados.0.smithi165.stdout:3: oid 3 updating version 3 to 4 2024-04-20T01:47:34.763 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 3 v 4 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-04-20T01:47:34.764 INFO:tasks.rados.rados.0.smithi165.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2024-04-20T01:47:34.764 INFO:tasks.rados.rados.0.smithi165.stdout:2: finishing write tid 3 to smithi16579500-2 2024-04-20T01:47:34.764 INFO:tasks.rados.rados.0.smithi165.stdout:2: oid 2 updating version 0 to 1 2024-04-20T01:47:34.764 INFO:tasks.rados.rados.0.smithi165.stdout:2: oid 2 updating version 1 to 2 2024-04-20T01:47:34.764 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 2 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-04-20T01:47:34.764 INFO:tasks.rados.rados.0.smithi165.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2024-04-20T01:47:34.764 INFO:tasks.rados.rados.0.smithi165.stdout:10: finishing write tid 1 to smithi16579500-10 2024-04-20T01:47:34.764 INFO:tasks.rados.rados.0.smithi165.stdout:10: finishing write tid 2 to smithi16579500-10 2024-04-20T01:47:34.764 INFO:tasks.rados.rados.0.smithi165.stdout:2: done (15 left) 2024-04-20T01:47:34.764 INFO:tasks.rados.rados.0.smithi165.stdout:3: done (14 left) 2024-04-20T01:47:34.764 INFO:tasks.rados.rados.0.smithi165.stdout:6: done (13 left) 2024-04-20T01:47:34.765 INFO:tasks.rados.rados.0.smithi165.stdout:20: write initial oid 20 2024-04-20T01:47:34.765 INFO:tasks.rados.rados.0.smithi165.stdout:20: seq_num 19 ranges {0=327680} 2024-04-20T01:47:34.765 INFO:tasks.rados.rados.0.smithi165.stdout:20: writing smithi16579500-20 from 0 to 327680 tid 1 2024-04-20T01:47:34.765 INFO:tasks.rados.rados.0.smithi165.stdout:21: write initial oid 21 2024-04-20T01:47:34.765 INFO:tasks.rados.rados.0.smithi165.stdout:21: seq_num 20 ranges {0=344064} 2024-04-20T01:47:34.766 INFO:tasks.rados.rados.0.smithi165.stdout:21: writing smithi16579500-21 from 0 to 344064 tid 1 2024-04-20T01:47:34.766 INFO:tasks.rados.rados.0.smithi165.stdout:22: write initial oid 22 2024-04-20T01:47:34.766 INFO:tasks.rados.rados.0.smithi165.stdout:22: seq_num 21 ranges {0=360448} 2024-04-20T01:47:34.766 INFO:tasks.rados.rados.0.smithi165.stdout:22: writing smithi16579500-22 from 0 to 360448 tid 1 2024-04-20T01:47:34.766 INFO:tasks.rados.rados.0.smithi165.stdout: waiting on 16 2024-04-20T01:47:34.767 INFO:tasks.rados.rados.0.smithi165.stdout:8: finishing write tid 3 to smithi16579500-8 2024-04-20T01:47:34.767 INFO:tasks.rados.rados.0.smithi165.stdout:8: oid 8 updating version 0 to 1 2024-04-20T01:47:34.767 INFO:tasks.rados.rados.0.smithi165.stdout:8: oid 8 updating version 1 to 2 2024-04-20T01:47:34.767 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 8 v 2 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-20T01:47:34.767 INFO:tasks.rados.rados.0.smithi165.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2024-04-20T01:47:34.767 INFO:tasks.rados.rados.0.smithi165.stdout:9: finishing write tid 3 to smithi16579500-9 2024-04-20T01:47:34.767 INFO:tasks.rados.rados.0.smithi165.stdout:9: oid 9 updating version 0 to 1 2024-04-20T01:47:34.767 INFO:tasks.rados.rados.0.smithi165.stdout:9: oid 9 updating version 1 to 2 2024-04-20T01:47:34.767 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 9 v 2 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-04-20T01:47:34.767 INFO:tasks.rados.rados.0.smithi165.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2024-04-20T01:47:34.767 INFO:tasks.rados.rados.0.smithi165.stdout:5: finishing write tid 1 to smithi16579500-5 2024-04-20T01:47:34.767 INFO:tasks.rados.rados.0.smithi165.stdout:8: done (15 left) 2024-04-20T01:47:34.767 INFO:tasks.rados.rados.0.smithi165.stdout:9: done (14 left) 2024-04-20T01:47:34.767 INFO:tasks.rados.rados.0.smithi165.stdout:23: write initial oid 23 2024-04-20T01:47:34.767 INFO:tasks.rados.rados.0.smithi165.stdout:23: seq_num 22 ranges {0=376832} 2024-04-20T01:47:34.768 INFO:tasks.rados.rados.0.smithi165.stdout:23: writing smithi16579500-23 from 0 to 376832 tid 1 2024-04-20T01:47:34.768 INFO:tasks.rados.rados.0.smithi165.stdout:24: write initial oid 24 2024-04-20T01:47:34.768 INFO:tasks.rados.rados.0.smithi165.stdout:24: seq_num 23 ranges {0=393216} 2024-04-20T01:47:34.769 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:47:34.771 INFO:tasks.rados.rados.0.smithi165.stdout:24: writing smithi16579500-24 from 0 to 393216 tid 1 2024-04-20T01:47:34.771 INFO:tasks.rados.rados.0.smithi165.stdout: waiting on 16 2024-04-20T01:47:34.771 INFO:tasks.rados.rados.0.smithi165.stdout:5: finishing write tid 2 to smithi16579500-5 2024-04-20T01:47:34.772 INFO:tasks.rados.rados.0.smithi165.stdout:16: finishing write tid 1 to smithi16579500-16 2024-04-20T01:47:34.772 INFO:tasks.rados.rados.0.smithi165.stdout:16: finishing write tid 2 to smithi16579500-16 2024-04-20T01:47:34.772 INFO:tasks.rados.rados.0.smithi165.stdout:13: finishing write tid 1 to smithi16579500-13 2024-04-20T01:47:34.772 INFO:tasks.rados.rados.0.smithi165.stdout:11: finishing write tid 3 to smithi16579500-11 2024-04-20T01:47:34.772 INFO:tasks.rados.rados.0.smithi165.stdout:11: oid 11 updating version 0 to 4 2024-04-20T01:47:34.772 INFO:tasks.rados.rados.0.smithi165.stdout:11: oid 11 version 4 is already newer than 3 2024-04-20T01:47:34.773 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 11 v 4 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-04-20T01:47:34.773 INFO:tasks.rados.rados.0.smithi165.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2024-04-20T01:47:34.773 INFO:tasks.rados.rados.0.smithi165.stdout:14: finishing write tid 1 to smithi16579500-14 2024-04-20T01:47:34.773 INFO:tasks.rados.rados.0.smithi165.stdout:17: finishing write tid 1 to smithi16579500-17 2024-04-20T01:47:34.773 INFO:tasks.rados.rados.0.smithi165.stdout:14: finishing write tid 2 to smithi16579500-14 2024-04-20T01:47:34.773 INFO:tasks.rados.rados.0.smithi165.stdout:17: finishing write tid 2 to smithi16579500-17 2024-04-20T01:47:34.773 INFO:tasks.rados.rados.0.smithi165.stdout:13: finishing write tid 2 to smithi16579500-13 2024-04-20T01:47:34.773 INFO:tasks.rados.rados.0.smithi165.stdout:16: finishing write tid 3 to smithi16579500-16 2024-04-20T01:47:34.774 INFO:tasks.rados.rados.0.smithi165.stdout:16: oid 16 updating version 0 to 2 2024-04-20T01:47:34.774 INFO:tasks.rados.rados.0.smithi165.stdout:16: oid 16 version 2 is already newer than 1 2024-04-20T01:47:34.774 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 16 v 2 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-04-20T01:47:34.774 INFO:tasks.rados.rados.0.smithi165.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2024-04-20T01:47:34.774 INFO:tasks.rados.rados.0.smithi165.stdout:14: finishing write tid 3 to smithi16579500-14 2024-04-20T01:47:34.774 INFO:tasks.rados.rados.0.smithi165.stdout:14: oid 14 updating version 0 to 5 2024-04-20T01:47:34.774 INFO:tasks.rados.rados.0.smithi165.stdout:14: oid 14 updating version 5 to 6 2024-04-20T01:47:34.774 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 14 v 6 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-04-20T01:47:34.774 INFO:tasks.rados.rados.0.smithi165.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2024-04-20T01:47:34.774 INFO:tasks.rados.rados.0.smithi165.stdout:5: finishing write tid 3 to smithi16579500-5 2024-04-20T01:47:34.774 INFO:tasks.rados.rados.0.smithi165.stdout:5: oid 5 updating version 0 to 1 2024-04-20T01:47:34.774 INFO:tasks.rados.rados.0.smithi165.stdout:5: oid 5 updating version 1 to 2 2024-04-20T01:47:34.774 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 5 v 2 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-04-20T01:47:34.774 INFO:tasks.rados.rados.0.smithi165.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2024-04-20T01:47:34.774 INFO:tasks.rados.rados.0.smithi165.stdout:17: finishing write tid 3 to smithi16579500-17 2024-04-20T01:47:34.775 INFO:tasks.rados.rados.0.smithi165.stdout:17: oid 17 updating version 0 to 3 2024-04-20T01:47:34.775 INFO:tasks.rados.rados.0.smithi165.stdout:17: oid 17 updating version 3 to 4 2024-04-20T01:47:34.775 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 17 v 4 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-04-20T01:47:34.775 INFO:tasks.rados.rados.0.smithi165.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2024-04-20T01:47:34.775 INFO:tasks.rados.rados.0.smithi165.stdout:10: finishing write tid 3 to smithi16579500-10 2024-04-20T01:47:34.775 INFO:tasks.rados.rados.0.smithi165.stdout:10: oid 10 updating version 0 to 3 2024-04-20T01:47:34.775 INFO:tasks.rados.rados.0.smithi165.stdout:10: oid 10 updating version 3 to 4 2024-04-20T01:47:34.775 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 10 v 4 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-04-20T01:47:34.775 INFO:tasks.rados.rados.0.smithi165.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2024-04-20T01:47:34.775 INFO:tasks.rados.rados.0.smithi165.stdout:15: finishing write tid 1 to smithi16579500-15 2024-04-20T01:47:34.775 INFO:tasks.rados.rados.0.smithi165.stdout:5: done (15 left) 2024-04-20T01:47:34.775 INFO:tasks.rados.rados.0.smithi165.stdout:10: done (14 left) 2024-04-20T01:47:34.775 INFO:tasks.rados.rados.0.smithi165.stdout:11: done (13 left) 2024-04-20T01:47:34.775 INFO:tasks.rados.rados.0.smithi165.stdout:14: done (12 left) 2024-04-20T01:47:34.775 INFO:tasks.rados.rados.0.smithi165.stdout:16: done (11 left) 2024-04-20T01:47:34.776 INFO:tasks.rados.rados.0.smithi165.stdout:17: done (10 left) 2024-04-20T01:47:34.776 INFO:tasks.rados.rados.0.smithi165.stdout:25: write initial oid 25 2024-04-20T01:47:34.776 INFO:tasks.rados.rados.0.smithi165.stdout:25: seq_num 24 ranges {0=409600} 2024-04-20T01:47:34.777 INFO:tasks.rados.rados.0.smithi165.stdout:25: writing smithi16579500-25 from 0 to 409600 tid 1 2024-04-20T01:47:34.777 INFO:tasks.rados.rados.0.smithi165.stdout:26: write initial oid 26 2024-04-20T01:47:34.777 INFO:tasks.rados.rados.0.smithi165.stdout:26: seq_num 25 ranges {0=425984} 2024-04-20T01:47:34.777 INFO:tasks.rados.rados.0.smithi165.stdout:26: writing smithi16579500-26 from 0 to 425984 tid 1 2024-04-20T01:47:34.777 INFO:tasks.rados.rados.0.smithi165.stdout:27: write initial oid 27 2024-04-20T01:47:34.777 INFO:tasks.rados.rados.0.smithi165.stdout:27: seq_num 26 ranges {0=442368} 2024-04-20T01:47:34.778 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:34.791+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:34.779 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-20T01:47:34.779 INFO:tasks.rados.rados.0.smithi165.stdout:27: writing smithi16579500-27 from 0 to 442368 tid 1 2024-04-20T01:47:34.779 INFO:tasks.rados.rados.0.smithi165.stdout:28: write initial oid 28 2024-04-20T01:47:34.779 INFO:tasks.rados.rados.0.smithi165.stdout:28: seq_num 27 ranges {0=458752} 2024-04-20T01:47:34.781 INFO:tasks.rados.rados.0.smithi165.stdout:28: writing smithi16579500-28 from 0 to 458752 tid 1 2024-04-20T01:47:34.781 INFO:tasks.rados.rados.0.smithi165.stdout:29: write initial oid 29 2024-04-20T01:47:34.782 INFO:tasks.rados.rados.0.smithi165.stdout:29: seq_num 28 ranges {0=475136} 2024-04-20T01:47:34.784 INFO:tasks.rados.rados.0.smithi165.stdout:29: writing smithi16579500-29 from 0 to 475136 tid 1 2024-04-20T01:47:34.784 INFO:tasks.rados.rados.0.smithi165.stdout:30: write initial oid 30 2024-04-20T01:47:34.784 INFO:tasks.rados.rados.0.smithi165.stdout:30: seq_num 29 ranges {0=491520} 2024-04-20T01:47:34.787 INFO:tasks.rados.rados.0.smithi165.stdout:30: writing smithi16579500-30 from 0 to 491520 tid 1 2024-04-20T01:47:34.788 INFO:tasks.rados.rados.0.smithi165.stdout: waiting on 16 2024-04-20T01:47:34.788 INFO:tasks.rados.rados.0.smithi165.stdout:15: finishing write tid 2 to smithi16579500-15 2024-04-20T01:47:34.788 INFO:tasks.rados.rados.0.smithi165.stdout:15: finishing write tid 3 to smithi16579500-15 2024-04-20T01:47:34.788 INFO:tasks.rados.rados.0.smithi165.stdout:15: oid 15 updating version 0 to 6 2024-04-20T01:47:34.788 INFO:tasks.rados.rados.0.smithi165.stdout:15: oid 15 version 6 is already newer than 5 2024-04-20T01:47:34.788 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 15 v 6 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-04-20T01:47:34.788 INFO:tasks.rados.rados.0.smithi165.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2024-04-20T01:47:34.788 INFO:tasks.rados.rados.0.smithi165.stdout:13: finishing write tid 3 to smithi16579500-13 2024-04-20T01:47:34.788 INFO:tasks.rados.rados.0.smithi165.stdout:13: oid 13 updating version 0 to 2 2024-04-20T01:47:34.788 INFO:tasks.rados.rados.0.smithi165.stdout:13: oid 13 version 2 is already newer than 1 2024-04-20T01:47:34.788 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 13 v 2 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-04-20T01:47:34.788 INFO:tasks.rados.rados.0.smithi165.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2024-04-20T01:47:34.788 INFO:tasks.rados.rados.0.smithi165.stdout:18: finishing write tid 1 to smithi16579500-18 2024-04-20T01:47:34.788 INFO:tasks.rados.rados.0.smithi165.stdout:13: done (15 left) 2024-04-20T01:47:34.789 INFO:tasks.rados.rados.0.smithi165.stdout:15: done (14 left) 2024-04-20T01:47:34.789 INFO:tasks.rados.rados.0.smithi165.stdout:31: write initial oid 31 2024-04-20T01:47:34.789 INFO:tasks.rados.rados.0.smithi165.stdout:31: seq_num 30 ranges {0=507904} 2024-04-20T01:47:34.790 INFO:tasks.rados.rados.0.smithi165.stdout:31: writing smithi16579500-31 from 0 to 507904 tid 1 2024-04-20T01:47:34.791 INFO:tasks.rados.rados.0.smithi165.stdout:32: write initial oid 32 2024-04-20T01:47:34.791 INFO:tasks.rados.rados.0.smithi165.stdout:32: seq_num 31 ranges {0=524288} 2024-04-20T01:47:34.794 INFO:tasks.rados.rados.0.smithi165.stdout:32: writing smithi16579500-32 from 0 to 524288 tid 1 2024-04-20T01:47:34.794 INFO:tasks.rados.rados.0.smithi165.stdout: waiting on 16 2024-04-20T01:47:34.794 INFO:tasks.rados.rados.0.smithi165.stdout:18: finishing write tid 2 to smithi16579500-18 2024-04-20T01:47:34.794 INFO:tasks.rados.rados.0.smithi165.stdout:18: finishing write tid 3 to smithi16579500-18 2024-04-20T01:47:34.794 INFO:tasks.rados.rados.0.smithi165.stdout:18: oid 18 updating version 0 to 1 2024-04-20T01:47:34.794 INFO:tasks.rados.rados.0.smithi165.stdout:18: oid 18 updating version 1 to 2 2024-04-20T01:47:34.795 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 18 v 2 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-04-20T01:47:34.795 INFO:tasks.rados.rados.0.smithi165.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2024-04-20T01:47:34.795 INFO:tasks.rados.rados.0.smithi165.stdout:19: finishing write tid 1 to smithi16579500-19 2024-04-20T01:47:34.795 INFO:tasks.rados.rados.0.smithi165.stdout:18: done (15 left) 2024-04-20T01:47:34.795 INFO:tasks.rados.rados.0.smithi165.stdout:33: write initial oid 33 2024-04-20T01:47:34.795 INFO:tasks.rados.rados.0.smithi165.stdout:33: seq_num 32 ranges {0=540672} 2024-04-20T01:47:34.796 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:34.796 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-20T01:47:34.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-20T01:47:34.796 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-20T01:47:34.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.796 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.5s3 32/30 ECSubWrite(tid=0, reqid=client.4643.0:10, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:34.796 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.753689+0000", 2024-04-20T01:47:34.796 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.053883367000000001, 2024-04-20T01:47:34.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053390069999999998, 2024-04-20T01:47:34.796 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:34.796 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:34.797 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:34.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:34.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753689+0000", 2024-04-20T01:47:34.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:34.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753689+0000", 2024-04-20T01:47:34.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:34.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753688+0000", 2024-04-20T01:47:34.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:34.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:34.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753715+0000", 2024-04-20T01:47:34.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7231000000000001e-05 2024-04-20T01:47:34.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:34.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753726+0000", 2024-04-20T01:47:34.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0781000000000001e-05 2024-04-20T01:47:34.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:34.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753735+0000", 2024-04-20T01:47:34.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5380000000000002e-06 2024-04-20T01:47:34.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:34.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753772+0000", 2024-04-20T01:47:34.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7092000000000003e-05 2024-04-20T01:47:34.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:34.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753777+0000", 2024-04-20T01:47:34.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9189999999999998e-06 2024-04-20T01:47:34.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:34.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.759003+0000", 2024-04-20T01:47:34.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052264690000000001 2024-04-20T01:47:34.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:34.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.759028+0000", 2024-04-20T01:47:34.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4576e-05 2024-04-20T01:47:34.800 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.800 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.800 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.800 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.5s3 32/30 ECSubWrite(tid=1, reqid=client.4643.0:11, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:34.800 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.754397+0000", 2024-04-20T01:47:34.800 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.053175148999999998, 2024-04-20T01:47:34.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01012041, 2024-04-20T01:47:34.801 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:34.801 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:34.801 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:34.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:34.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754397+0000", 2024-04-20T01:47:34.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:34.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754397+0000", 2024-04-20T01:47:34.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:34.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754396+0000", 2024-04-20T01:47:34.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:34.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:34.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754423+0000", 2024-04-20T01:47:34.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7032000000000001e-05 2024-04-20T01:47:34.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:34.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754438+0000", 2024-04-20T01:47:34.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4946999999999999e-05 2024-04-20T01:47:34.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.803 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:34.803 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754447+0000", 2024-04-20T01:47:34.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8550000000000008e-06 2024-04-20T01:47:34.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.804 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:34.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754496+0000", 2024-04-20T01:47:34.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9018999999999999e-05 2024-04-20T01:47:34.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.804 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:34.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754506+0000", 2024-04-20T01:47:34.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7079999999999992e-06 2024-04-20T01:47:34.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.804 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:34.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764493+0000", 2024-04-20T01:47:34.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0099871460000000006 2024-04-20T01:47:34.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:34.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764517+0000", 2024-04-20T01:47:34.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4467000000000001e-05 2024-04-20T01:47:34.805 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.805 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.805 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.805 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.9s4 32/30 ECSubWrite(tid=2, reqid=client.4643.0:7, at_version=32'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:34.805 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.755700+0000", 2024-04-20T01:47:34.805 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.051871767999999999, 2024-04-20T01:47:34.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0088510410000000005, 2024-04-20T01:47:34.806 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:34.806 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:34.806 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:34.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:34.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755700+0000", 2024-04-20T01:47:34.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:34.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755700+0000", 2024-04-20T01:47:34.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:34.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755700+0000", 2024-04-20T01:47:34.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:34.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:34.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755735+0000", 2024-04-20T01:47:34.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5524000000000001e-05 2024-04-20T01:47:34.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:34.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755750+0000", 2024-04-20T01:47:34.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4609e-05 2024-04-20T01:47:34.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:34.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755759+0000", 2024-04-20T01:47:34.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3700000000000001e-06 2024-04-20T01:47:34.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:34.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755799+0000", 2024-04-20T01:47:34.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9560000000000001e-05 2024-04-20T01:47:34.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:34.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755804+0000", 2024-04-20T01:47:34.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3149999999999999e-06 2024-04-20T01:47:34.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.809 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:34.809 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764532+0000", 2024-04-20T01:47:34.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087277770000000008 2024-04-20T01:47:34.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.809 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:34.809 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764551+0000", 2024-04-20T01:47:34.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9582e-05 2024-04-20T01:47:34.809 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.809 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.809 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.809 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.9s4 32/30 ECSubWrite(tid=3, reqid=client.4643.0:8, at_version=32'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:34.810 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.755948+0000", 2024-04-20T01:47:34.810 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.051624591999999997, 2024-04-20T01:47:34.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0086164850000000001, 2024-04-20T01:47:34.810 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:34.810 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:34.810 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:34.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.810 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:34.810 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755948+0000", 2024-04-20T01:47:34.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:34.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755948+0000", 2024-04-20T01:47:34.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:34.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755947+0000", 2024-04-20T01:47:34.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:34.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:34.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755963+0000", 2024-04-20T01:47:34.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6013000000000001e-05 2024-04-20T01:47:34.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:34.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755975+0000", 2024-04-20T01:47:34.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2347000000000001e-05 2024-04-20T01:47:34.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:34.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755985+0000", 2024-04-20T01:47:34.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8730000000000002e-06 2024-04-20T01:47:34.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:34.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.756022+0000", 2024-04-20T01:47:34.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7039000000000002e-05 2024-04-20T01:47:34.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:34.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.756027+0000", 2024-04-20T01:47:34.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.006e-06 2024-04-20T01:47:34.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:34.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764554+0000", 2024-04-20T01:47:34.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0085269609999999996 2024-04-20T01:47:34.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:34.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764564+0000", 2024-04-20T01:47:34.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8740000000000007e-06 2024-04-20T01:47:34.814 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.814 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.814 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.814 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s3 33/30 ECSubWrite(tid=157, reqid=client.4643.0:16, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:34.814 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.757106+0000", 2024-04-20T01:47:34.814 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.050465909000000003, 2024-04-20T01:47:34.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0086461160000000006, 2024-04-20T01:47:34.814 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:34.814 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:34.814 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:34.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.814 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:34.815 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757106+0000", 2024-04-20T01:47:34.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.815 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:34.815 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757106+0000", 2024-04-20T01:47:34.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.815 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:34.815 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757106+0000", 2024-04-20T01:47:34.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:34.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.815 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:34.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757173+0000", 2024-04-20T01:47:34.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6734000000000002e-05 2024-04-20T01:47:34.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:34.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757184+0000", 2024-04-20T01:47:34.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0896999999999999e-05 2024-04-20T01:47:34.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:34.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757192+0000", 2024-04-20T01:47:34.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4460000000000001e-06 2024-04-20T01:47:34.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:34.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757248+0000", 2024-04-20T01:47:34.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5949999999999998e-05 2024-04-20T01:47:34.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:34.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757257+0000", 2024-04-20T01:47:34.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.935e-06 2024-04-20T01:47:34.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:34.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.765715+0000", 2024-04-20T01:47:34.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0084578169999999994 2024-04-20T01:47:34.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:34.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.765752+0000", 2024-04-20T01:47:34.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7608000000000003e-05 2024-04-20T01:47:34.818 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.818 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.818 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.818 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s3 33/30 ECSubWrite(tid=158, reqid=client.4643.0:17, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:34.818 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.757999+0000", 2024-04-20T01:47:34.818 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.049573659999999999, 2024-04-20T01:47:34.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077685380000000002, 2024-04-20T01:47:34.819 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:34.819 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:34.819 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:34.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:34.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757999+0000", 2024-04-20T01:47:34.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:34.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757999+0000", 2024-04-20T01:47:34.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.820 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:34.820 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757998+0000", 2024-04-20T01:47:34.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:34.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.820 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:34.820 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.758012+0000", 2024-04-20T01:47:34.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4440000000000001e-05 2024-04-20T01:47:34.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.820 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:34.820 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.758023+0000", 2024-04-20T01:47:34.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0059e-05 2024-04-20T01:47:34.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:34.821 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.758029+0000", 2024-04-20T01:47:34.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2169999999999999e-06 2024-04-20T01:47:34.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:34.821 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.759046+0000", 2024-04-20T01:47:34.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010175799999999999 2024-04-20T01:47:34.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:34.822 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.759055+0000", 2024-04-20T01:47:34.822 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9789999999999999e-06 2024-04-20T01:47:34.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.822 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:34.822 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.765756+0000", 2024-04-20T01:47:34.822 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0067008099999999998 2024-04-20T01:47:34.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.822 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:34.822 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.765767+0000", 2024-04-20T01:47:34.822 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0998999999999999e-05 2024-04-20T01:47:34.822 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.822 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.822 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.823 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s4 33/30 ECSubWrite(tid=0, reqid=client.4643.0:25, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:34.823 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.760223+0000", 2024-04-20T01:47:34.823 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.047349721999999997, 2024-04-20T01:47:34.823 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061353559999999998, 2024-04-20T01:47:34.823 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:34.823 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:34.823 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:34.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.824 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:34.824 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.760223+0000", 2024-04-20T01:47:34.824 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:34.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.760223+0000", 2024-04-20T01:47:34.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:34.826 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.760222+0000", 2024-04-20T01:47:34.826 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:34.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.826 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:34.827 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.760367+0000", 2024-04-20T01:47:34.827 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014520399999999999 2024-04-20T01:47:34.827 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.827 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:34.827 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.760378+0000", 2024-04-20T01:47:34.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0305e-05 2024-04-20T01:47:34.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:34.828 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.760386+0000", 2024-04-20T01:47:34.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3329999999999997e-06 2024-04-20T01:47:34.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:34.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.760437+0000", 2024-04-20T01:47:34.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1446000000000002e-05 2024-04-20T01:47:34.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:34.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.760446+0000", 2024-04-20T01:47:34.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9840000000000007e-06 2024-04-20T01:47:34.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:34.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.766335+0000", 2024-04-20T01:47:34.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058887979999999998 2024-04-20T01:47:34.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.831 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:34.831 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.766358+0000", 2024-04-20T01:47:34.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2637999999999999e-05 2024-04-20T01:47:34.831 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.831 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.831 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.831 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s4 33/30 ECSubWrite(tid=1, reqid=client.4643.0:26, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:34.831 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.761328+0000", 2024-04-20T01:47:34.831 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.046244319999999998, 2024-04-20T01:47:34.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050453329999999999, 2024-04-20T01:47:34.831 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:34.832 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:34.832 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:34.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:34.832 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.761328+0000", 2024-04-20T01:47:34.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:34.832 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.761328+0000", 2024-04-20T01:47:34.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:34.833 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.761327+0000", 2024-04-20T01:47:34.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:34.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.833 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:34.833 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.761346+0000", 2024-04-20T01:47:34.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8521000000000001e-05 2024-04-20T01:47:34.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.833 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:34.833 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.761359+0000", 2024-04-20T01:47:34.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3023999999999999e-05 2024-04-20T01:47:34.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:34.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.761376+0000", 2024-04-20T01:47:34.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7085999999999999e-05 2024-04-20T01:47:34.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:34.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.761732+0000", 2024-04-20T01:47:34.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00035644499999999999 2024-04-20T01:47:34.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:34.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.761739+0000", 2024-04-20T01:47:34.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4570000000000002e-06 2024-04-20T01:47:34.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:34.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.766362+0000", 2024-04-20T01:47:34.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046236239999999998 2024-04-20T01:47:34.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:34.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.766373+0000", 2024-04-20T01:47:34.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0793e-05 2024-04-20T01:47:34.835 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.835 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.835 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.836 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.1s1 33/30 ECSubWrite(tid=6, reqid=client.4643.0:61, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:34.836 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.783938+0000", 2024-04-20T01:47:34.836 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.023634473, 2024-04-20T01:47:34.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004475181, 2024-04-20T01:47:34.836 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:34.836 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:34.836 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:34.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.836 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:34.836 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.783938+0000", 2024-04-20T01:47:34.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.837 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:34.837 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.783938+0000", 2024-04-20T01:47:34.837 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.837 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:34.837 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.783937+0000", 2024-04-20T01:47:34.837 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:34.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.837 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:34.837 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784090+0000", 2024-04-20T01:47:34.838 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000152807 2024-04-20T01:47:34.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:34.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784103+0000", 2024-04-20T01:47:34.838 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.234e-05 2024-04-20T01:47:34.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:34.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784111+0000", 2024-04-20T01:47:34.838 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8300000000000002e-06 2024-04-20T01:47:34.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:34.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784167+0000", 2024-04-20T01:47:34.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5454999999999997e-05 2024-04-20T01:47:34.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:34.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784177+0000", 2024-04-20T01:47:34.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0209e-05 2024-04-20T01:47:34.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:34.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788392+0000", 2024-04-20T01:47:34.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042144629999999999 2024-04-20T01:47:34.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.840 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:34.840 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788413+0000", 2024-04-20T01:47:34.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1347000000000001e-05 2024-04-20T01:47:34.840 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.840 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.840 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.840 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s3 33/30 ECSubWrite(tid=168, reqid=client.4643.0:67, at_version=33'3, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:47:34.840 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.784794+0000", 2024-04-20T01:47:34.840 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.022778073999999999, 2024-04-20T01:47:34.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011714966, 2024-04-20T01:47:34.840 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:34.840 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:34.841 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:34.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:34.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784794+0000", 2024-04-20T01:47:34.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:34.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784794+0000", 2024-04-20T01:47:34.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:34.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784794+0000", 2024-04-20T01:47:34.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:34.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.842 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:34.842 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784957+0000", 2024-04-20T01:47:34.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016303800000000001 2024-04-20T01:47:34.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.842 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:34.842 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784970+0000", 2024-04-20T01:47:34.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3305000000000001e-05 2024-04-20T01:47:34.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.842 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:34.842 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784979+0000", 2024-04-20T01:47:34.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7129999999999994e-06 2024-04-20T01:47:34.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:34.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785046+0000", 2024-04-20T01:47:34.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7108999999999997e-05 2024-04-20T01:47:34.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:34.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785055+0000", 2024-04-20T01:47:34.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.234e-06 2024-04-20T01:47:34.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:34.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796482+0000", 2024-04-20T01:47:34.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011427196000000001 2024-04-20T01:47:34.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.844 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:34.844 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796509+0000", 2024-04-20T01:47:34.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6999e-05 2024-04-20T01:47:34.844 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.844 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.844 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.844 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s3 33/30 ECSubWrite(tid=169, reqid=client.4643.0:68, at_version=33'4, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:47:34.844 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.785553+0000", 2024-04-20T01:47:34.844 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.022019237000000001, 2024-04-20T01:47:34.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010979604, 2024-04-20T01:47:34.844 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:34.845 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:34.845 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:34.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:34.845 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785553+0000", 2024-04-20T01:47:34.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:34.845 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785553+0000", 2024-04-20T01:47:34.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:34.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785552+0000", 2024-04-20T01:47:34.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:34.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.846 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:34.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785579+0000", 2024-04-20T01:47:34.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6492999999999999e-05 2024-04-20T01:47:34.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.846 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:34.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785589+0000", 2024-04-20T01:47:34.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0353000000000001e-05 2024-04-20T01:47:34.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:34.847 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785598+0000", 2024-04-20T01:47:34.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8130000000000002e-06 2024-04-20T01:47:34.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:34.847 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788372+0000", 2024-04-20T01:47:34.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027740410000000001 2024-04-20T01:47:34.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:34.847 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788381+0000", 2024-04-20T01:47:34.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5930000000000006e-06 2024-04-20T01:47:34.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.848 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:34.848 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796518+0000", 2024-04-20T01:47:34.848 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0081374710000000003 2024-04-20T01:47:34.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.848 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:34.848 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796533+0000", 2024-04-20T01:47:34.848 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.451e-05 2024-04-20T01:47:34.848 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.848 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.848 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.849 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.849 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.1s1 33/30 ECSubWrite(tid=7, reqid=client.4643.0:62, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:34.849 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.786554+0000", 2024-04-20T01:47:34.849 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.021017813, 2024-04-20T01:47:34.849 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007049273, 2024-04-20T01:47:34.849 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:34.849 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:34.849 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:34.849 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.849 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:34.849 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.786554+0000", 2024-04-20T01:47:34.849 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.849 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.849 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:34.850 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.786554+0000", 2024-04-20T01:47:34.850 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.850 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.850 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:34.850 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.786554+0000", 2024-04-20T01:47:34.850 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:34.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.850 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.850 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:34.850 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.786572+0000", 2024-04-20T01:47:34.850 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8541000000000001e-05 2024-04-20T01:47:34.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.850 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.851 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:34.851 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.786584+0000", 2024-04-20T01:47:34.851 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1292e-05 2024-04-20T01:47:34.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.851 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:34.851 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.786590+0000", 2024-04-20T01:47:34.851 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4960000000000001e-06 2024-04-20T01:47:34.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.851 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:34.851 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.786637+0000", 2024-04-20T01:47:34.851 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6483999999999997e-05 2024-04-20T01:47:34.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.852 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:34.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.786647+0000", 2024-04-20T01:47:34.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0638e-05 2024-04-20T01:47:34.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.852 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.852 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:34.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793582+0000", 2024-04-20T01:47:34.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069347100000000002 2024-04-20T01:47:34.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.852 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.852 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:34.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793604+0000", 2024-04-20T01:47:34.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1753999999999999e-05 2024-04-20T01:47:34.852 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.852 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.853 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.853 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.853 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s4 33/30 ECSubWrite(tid=10, reqid=client.4643.0:70, at_version=33'7, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:34.853 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.787329+0000", 2024-04-20T01:47:34.853 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.020243687, 2024-04-20T01:47:34.853 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092166409999999994, 2024-04-20T01:47:34.853 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:34.853 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:34.853 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:34.853 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.853 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:34.853 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787329+0000", 2024-04-20T01:47:34.853 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.853 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.854 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:34.854 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787329+0000", 2024-04-20T01:47:34.854 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.854 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:34.854 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787328+0000", 2024-04-20T01:47:34.854 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:34.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.854 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:34.854 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787448+0000", 2024-04-20T01:47:34.854 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011992199999999999 2024-04-20T01:47:34.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.855 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:34.855 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787458+0000", 2024-04-20T01:47:34.855 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0037000000000001e-05 2024-04-20T01:47:34.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.855 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:34.855 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787466+0000", 2024-04-20T01:47:34.855 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6270000000000006e-06 2024-04-20T01:47:34.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.855 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:34.855 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787517+0000", 2024-04-20T01:47:34.855 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0890000000000002e-05 2024-04-20T01:47:34.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:34.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787528+0000", 2024-04-20T01:47:34.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0363000000000001e-05 2024-04-20T01:47:34.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:34.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796522+0000", 2024-04-20T01:47:34.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0089942619999999994 2024-04-20T01:47:34.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:34.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796545+0000", 2024-04-20T01:47:34.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3213999999999998e-05 2024-04-20T01:47:34.857 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.857 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.857 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.858 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s4 33/30 ECSubWrite(tid=11, reqid=client.4643.0:71, at_version=33'8, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:34.858 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.788214+0000", 2024-04-20T01:47:34.858 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.019358205999999999, 2024-04-20T01:47:34.858 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083448129999999995, 2024-04-20T01:47:34.858 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:34.858 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:34.858 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:34.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.858 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:34.858 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788214+0000", 2024-04-20T01:47:34.858 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.859 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:34.859 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788214+0000", 2024-04-20T01:47:34.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.859 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:34.859 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788214+0000", 2024-04-20T01:47:34.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:34.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.859 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:34.859 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788224+0000", 2024-04-20T01:47:34.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0013000000000001e-05 2024-04-20T01:47:34.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.860 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:34.860 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788231+0000", 2024-04-20T01:47:34.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0990000000000001e-06 2024-04-20T01:47:34.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.860 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:34.860 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788245+0000", 2024-04-20T01:47:34.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4166e-05 2024-04-20T01:47:34.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:34.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788446+0000", 2024-04-20T01:47:34.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00020135800000000001 2024-04-20T01:47:34.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:34.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788455+0000", 2024-04-20T01:47:34.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5099999999999998e-06 2024-04-20T01:47:34.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:34.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796549+0000", 2024-04-20T01:47:34.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080940860000000003 2024-04-20T01:47:34.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:34.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796559+0000", 2024-04-20T01:47:34.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9340000000000001e-06 2024-04-20T01:47:34.862 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.862 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.862 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.862 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s4 33/30 ECSubWrite(tid=170, reqid=client.4643.0:73, at_version=33'5, trim_to=0'0, roll_forward_to=33'4))", 2024-04-20T01:47:34.862 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.789806+0000", 2024-04-20T01:47:34.862 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.017766391999999999, 2024-04-20T01:47:34.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006772882, 2024-04-20T01:47:34.862 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:34.863 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:34.863 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:34.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.863 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:34.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.789806+0000", 2024-04-20T01:47:34.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.863 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:34.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.789806+0000", 2024-04-20T01:47:34.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:34.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.789805+0000", 2024-04-20T01:47:34.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:34.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:34.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.790025+0000", 2024-04-20T01:47:34.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021945299999999999 2024-04-20T01:47:34.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:34.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.790040+0000", 2024-04-20T01:47:34.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.504e-05 2024-04-20T01:47:34.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:34.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.790049+0000", 2024-04-20T01:47:34.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1770000000000004e-06 2024-04-20T01:47:34.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:34.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.790116+0000", 2024-04-20T01:47:34.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6960999999999998e-05 2024-04-20T01:47:34.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:34.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.790126+0000", 2024-04-20T01:47:34.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0614999999999999e-05 2024-04-20T01:47:34.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:34.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796559+0000", 2024-04-20T01:47:34.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064323160000000004 2024-04-20T01:47:34.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:34.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796579+0000", 2024-04-20T01:47:34.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9925999999999999e-05 2024-04-20T01:47:34.867 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.867 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.867 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.867 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s4 33/30 ECSubWrite(tid=171, reqid=client.4643.0:74, at_version=33'6, trim_to=0'0, roll_forward_to=33'4))", 2024-04-20T01:47:34.868 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.790763+0000", 2024-04-20T01:47:34.868 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.016809398999999999, 2024-04-20T01:47:34.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058345990000000002, 2024-04-20T01:47:34.868 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:34.868 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:34.868 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:34.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.868 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:34.868 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.790763+0000", 2024-04-20T01:47:34.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.868 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:34.868 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.790763+0000", 2024-04-20T01:47:34.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:34.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.790762+0000", 2024-04-20T01:47:34.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:34.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:34.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.790778+0000", 2024-04-20T01:47:34.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6263e-05 2024-04-20T01:47:34.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:34.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.790792+0000", 2024-04-20T01:47:34.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3536999999999999e-05 2024-04-20T01:47:34.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:34.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.790800+0000", 2024-04-20T01:47:34.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2840000000000006e-06 2024-04-20T01:47:34.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:34.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.790919+0000", 2024-04-20T01:47:34.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000119245 2024-04-20T01:47:34.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:34.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.790928+0000", 2024-04-20T01:47:34.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4889999999999995e-06 2024-04-20T01:47:34.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:34.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796584+0000", 2024-04-20T01:47:34.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005656047 2024-04-20T01:47:34.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:34.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796597+0000", 2024-04-20T01:47:34.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3419e-05 2024-04-20T01:47:34.872 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.872 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.872 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.872 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.872 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.5s3 33/30 ECSubWrite(tid=6, reqid=client.4643.0:65, at_version=0'0, trim_to=0'0, roll_forward_to=33'4))", 2024-04-20T01:47:34.872 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.793486+0000", 2024-04-20T01:47:34.872 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.014086397000000001, 2024-04-20T01:47:34.872 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040802499999999997, 2024-04-20T01:47:34.872 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:34.872 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:34.872 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:34.872 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.872 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:34.873 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793486+0000", 2024-04-20T01:47:34.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.873 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:34.873 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793486+0000", 2024-04-20T01:47:34.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.873 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:34.873 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793485+0000", 2024-04-20T01:47:34.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:34.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:34.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793494+0000", 2024-04-20T01:47:34.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6950000000000006e-06 2024-04-20T01:47:34.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:34.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793504+0000", 2024-04-20T01:47:34.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0513000000000001e-05 2024-04-20T01:47:34.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:34.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793513+0000", 2024-04-20T01:47:34.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1759999999999993e-06 2024-04-20T01:47:34.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:34.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793578+0000", 2024-04-20T01:47:34.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4883999999999997e-05 2024-04-20T01:47:34.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:34.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793587+0000", 2024-04-20T01:47:34.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1929999999999999e-06 2024-04-20T01:47:34.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:34.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797553+0000", 2024-04-20T01:47:34.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039659939999999996 2024-04-20T01:47:34.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:34.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797566+0000", 2024-04-20T01:47:34.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3360000000000001e-05 2024-04-20T01:47:34.876 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.876 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.876 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.877 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s4 33/30 ECSubWrite(tid=172, reqid=client.4643.0:74, at_version=0'0, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:34.877 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.797669+0000", 2024-04-20T01:47:34.877 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.0099030609999999995, 2024-04-20T01:47:34.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047862850000000004, 2024-04-20T01:47:34.877 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:34.877 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:34.877 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:34.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.877 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:34.877 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797669+0000", 2024-04-20T01:47:34.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:34.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797669+0000", 2024-04-20T01:47:34.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:34.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797669+0000", 2024-04-20T01:47:34.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:34.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:34.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797679+0000", 2024-04-20T01:47:34.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0903999999999999e-05 2024-04-20T01:47:34.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:34.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797692+0000", 2024-04-20T01:47:34.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2431e-05 2024-04-20T01:47:34.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:34.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797701+0000", 2024-04-20T01:47:34.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9479999999999997e-06 2024-04-20T01:47:34.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:34.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797780+0000", 2024-04-20T01:47:34.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9320000000000006e-05 2024-04-20T01:47:34.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:34.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797790+0000", 2024-04-20T01:47:34.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4560000000000006e-06 2024-04-20T01:47:34.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:34.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.802440+0000", 2024-04-20T01:47:34.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046507700000000003 2024-04-20T01:47:34.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:34.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.802455+0000", 2024-04-20T01:47:34.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5028000000000001e-05 2024-04-20T01:47:34.881 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.881 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.881 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.881 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s3 33/30 ECSubWrite(tid=173, reqid=client.4643.0:68, at_version=0'0, trim_to=0'0, roll_forward_to=33'4))", 2024-04-20T01:47:34.881 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.797711+0000", 2024-04-20T01:47:34.881 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.0098611380000000002, 2024-04-20T01:47:34.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046087649999999999, 2024-04-20T01:47:34.881 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:34.881 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:34.882 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:34.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:34.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797711+0000", 2024-04-20T01:47:34.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:34.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797711+0000", 2024-04-20T01:47:34.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:34.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797711+0000", 2024-04-20T01:47:34.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:47:34.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.883 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:34.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797718+0000", 2024-04-20T01:47:34.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4900000000000003e-06 2024-04-20T01:47:34.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.883 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:34.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797725+0000", 2024-04-20T01:47:34.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7710000000000001e-06 2024-04-20T01:47:34.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.883 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:34.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797731+0000", 2024-04-20T01:47:34.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7810000000000002e-06 2024-04-20T01:47:34.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.884 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:34.884 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797780+0000", 2024-04-20T01:47:34.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9450999999999998e-05 2024-04-20T01:47:34.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.884 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:34.884 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797790+0000", 2024-04-20T01:47:34.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2490000000000007e-06 2024-04-20T01:47:34.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:34.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.802299+0000", 2024-04-20T01:47:34.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045091200000000001 2024-04-20T01:47:34.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:34.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.802320+0000", 2024-04-20T01:47:34.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1138e-05 2024-04-20T01:47:34.885 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.885 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.885 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.886 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.1s1 33/30 ECSubWrite(tid=15, reqid=client.4643.0:80, at_version=33'4, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:47:34.886 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.797832+0000", 2024-04-20T01:47:34.886 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.0097405870000000002, 2024-04-20T01:47:34.886 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044688059999999996, 2024-04-20T01:47:34.886 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:34.886 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:34.886 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:34.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.886 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:34.886 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797832+0000", 2024-04-20T01:47:34.886 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:34.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797832+0000", 2024-04-20T01:47:34.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:34.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:34.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797831+0000", 2024-04-20T01:47:34.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:34.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.888 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:34.888 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797845+0000", 2024-04-20T01:47:34.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4255e-05 2024-04-20T01:47:34.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.888 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:34.888 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797854+0000", 2024-04-20T01:47:34.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8810000000000002e-06 2024-04-20T01:47:34.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.888 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:34.888 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797861+0000", 2024-04-20T01:47:34.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2200000000000003e-06 2024-04-20T01:47:34.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.889 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:34.889 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797912+0000", 2024-04-20T01:47:34.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0723999999999997e-05 2024-04-20T01:47:34.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.889 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:34.889 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797920+0000", 2024-04-20T01:47:34.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2840000000000006e-06 2024-04-20T01:47:34.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.889 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:34.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.802273+0000", 2024-04-20T01:47:34.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043524569999999997 2024-04-20T01:47:34.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:34.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:34.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:34.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.802300+0000", 2024-04-20T01:47:34.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7551000000000001e-05 2024-04-20T01:47:34.890 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.890 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.890 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.891 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:34.891 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:34.891 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:34.892 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:47:34.893 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:34.906+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:47:34.895 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-20T01:47:34.910 INFO:tasks.rados.rados.0.smithi165.stdout:33: writing smithi16579500-33 from 0 to 540672 tid 1 2024-04-20T01:47:34.910 INFO:tasks.rados.rados.0.smithi165.stdout: waiting on 16 2024-04-20T01:47:34.910 INFO:tasks.rados.rados.0.smithi165.stdout:19: finishing write tid 2 to smithi16579500-19 2024-04-20T01:47:34.910 INFO:tasks.rados.rados.0.smithi165.stdout:20: finishing write tid 1 to smithi16579500-20 2024-04-20T01:47:34.910 INFO:tasks.rados.rados.0.smithi165.stdout:20: finishing write tid 2 to smithi16579500-20 2024-04-20T01:47:34.910 INFO:tasks.rados.rados.0.smithi165.stdout:21: finishing write tid 1 to smithi16579500-21 2024-04-20T01:47:34.910 INFO:tasks.rados.rados.0.smithi165.stdout:22: finishing write tid 1 to smithi16579500-22 2024-04-20T01:47:34.910 INFO:tasks.rados.rados.0.smithi165.stdout:22: finishing write tid 2 to smithi16579500-22 2024-04-20T01:47:34.911 INFO:tasks.rados.rados.0.smithi165.stdout:19: finishing write tid 3 to smithi16579500-19 2024-04-20T01:47:34.911 INFO:tasks.rados.rados.0.smithi165.stdout:19: oid 19 updating version 0 to 7 2024-04-20T01:47:34.911 INFO:tasks.rados.rados.0.smithi165.stdout:19: oid 19 updating version 7 to 8 2024-04-20T01:47:34.911 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 19 v 8 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-04-20T01:47:34.911 INFO:tasks.rados.rados.0.smithi165.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2024-04-20T01:47:34.911 INFO:tasks.rados.rados.0.smithi165.stdout:21: finishing write tid 2 to smithi16579500-21 2024-04-20T01:47:34.911 INFO:tasks.rados.rados.0.smithi165.stdout:22: finishing write tid 3 to smithi16579500-22 2024-04-20T01:47:34.911 INFO:tasks.rados.rados.0.smithi165.stdout:22: oid 22 updating version 0 to 4 2024-04-20T01:47:34.911 INFO:tasks.rados.rados.0.smithi165.stdout:22: oid 22 version 4 is already newer than 3 2024-04-20T01:47:34.911 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 22 v 4 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-04-20T01:47:34.911 INFO:tasks.rados.rados.0.smithi165.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2024-04-20T01:47:34.911 INFO:tasks.rados.rados.0.smithi165.stdout:20: finishing write tid 3 to smithi16579500-20 2024-04-20T01:47:34.911 INFO:tasks.rados.rados.0.smithi165.stdout:20: oid 20 updating version 0 to 3 2024-04-20T01:47:34.911 INFO:tasks.rados.rados.0.smithi165.stdout:20: oid 20 updating version 3 to 4 2024-04-20T01:47:34.912 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 20 v 4 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-04-20T01:47:34.912 INFO:tasks.rados.rados.0.smithi165.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2024-04-20T01:47:34.912 INFO:tasks.rados.rados.0.smithi165.stdout:25: finishing write tid 1 to smithi16579500-25 2024-04-20T01:47:34.912 INFO:tasks.rados.rados.0.smithi165.stdout:23: finishing write tid 1 to smithi16579500-23 2024-04-20T01:47:34.912 INFO:tasks.rados.rados.0.smithi165.stdout:25: finishing write tid 2 to smithi16579500-25 2024-04-20T01:47:34.912 INFO:tasks.rados.rados.0.smithi165.stdout:23: finishing write tid 2 to smithi16579500-23 2024-04-20T01:47:34.912 INFO:tasks.rados.rados.0.smithi165.stdout:24: finishing write tid 1 to smithi16579500-24 2024-04-20T01:47:34.912 INFO:tasks.rados.rados.0.smithi165.stdout:24: finishing write tid 2 to smithi16579500-24 2024-04-20T01:47:34.912 INFO:tasks.rados.rados.0.smithi165.stdout:21: finishing write tid 3 to smithi16579500-21 2024-04-20T01:47:34.912 INFO:tasks.rados.rados.0.smithi165.stdout:21: oid 21 updating version 0 to 1 2024-04-20T01:47:34.912 INFO:tasks.rados.rados.0.smithi165.stdout:21: oid 21 updating version 1 to 2 2024-04-20T01:47:34.912 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 21 v 2 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-04-20T01:47:34.912 INFO:tasks.rados.rados.0.smithi165.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2024-04-20T01:47:34.912 INFO:tasks.rados.rados.0.smithi165.stdout:23: finishing write tid 3 to smithi16579500-23 2024-04-20T01:47:34.913 INFO:tasks.rados.rados.0.smithi165.stdout:23: oid 23 updating version 0 to 3 2024-04-20T01:47:34.913 INFO:tasks.rados.rados.0.smithi165.stdout:23: oid 23 updating version 3 to 4 2024-04-20T01:47:34.913 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 23 v 4 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-04-20T01:47:34.913 INFO:tasks.rados.rados.0.smithi165.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2024-04-20T01:47:34.913 INFO:tasks.rados.rados.0.smithi165.stdout:24: finishing write tid 3 to smithi16579500-24 2024-04-20T01:47:34.913 INFO:tasks.rados.rados.0.smithi165.stdout:24: oid 24 updating version 0 to 7 2024-04-20T01:47:34.913 INFO:tasks.rados.rados.0.smithi165.stdout:24: oid 24 updating version 7 to 8 2024-04-20T01:47:34.913 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 24 v 8 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-04-20T01:47:34.913 INFO:tasks.rados.rados.0.smithi165.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2024-04-20T01:47:34.913 INFO:tasks.rados.rados.0.smithi165.stdout:27: finishing write tid 1 to smithi16579500-27 2024-04-20T01:47:34.913 INFO:tasks.rados.rados.0.smithi165.stdout:25: finishing write tid 3 to smithi16579500-25 2024-04-20T01:47:34.913 INFO:tasks.rados.rados.0.smithi165.stdout:25: oid 25 updating version 0 to 5 2024-04-20T01:47:34.914 INFO:tasks.rados.rados.0.smithi165.stdout:25: oid 25 updating version 5 to 6 2024-04-20T01:47:34.914 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 25 v 6 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-04-20T01:47:34.914 INFO:tasks.rados.rados.0.smithi165.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2024-04-20T01:47:34.914 INFO:tasks.rados.rados.0.smithi165.stdout:26: finishing write tid 1 to smithi16579500-26 2024-04-20T01:47:34.914 INFO:tasks.rados.rados.0.smithi165.stdout:26: finishing write tid 2 to smithi16579500-26 2024-04-20T01:47:34.914 INFO:tasks.rados.rados.0.smithi165.stdout:27: finishing write tid 2 to smithi16579500-27 2024-04-20T01:47:34.914 INFO:tasks.rados.rados.0.smithi165.stdout:26: finishing write tid 3 to smithi16579500-26 2024-04-20T01:47:34.914 INFO:tasks.rados.rados.0.smithi165.stdout:26: oid 26 updating version 0 to 5 2024-04-20T01:47:34.914 INFO:tasks.rados.rados.0.smithi165.stdout:26: oid 26 updating version 5 to 6 2024-04-20T01:47:34.914 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 26 v 6 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-04-20T01:47:34.914 INFO:tasks.rados.rados.0.smithi165.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2024-04-20T01:47:34.914 INFO:tasks.rados.rados.0.smithi165.stdout:29: finishing write tid 1 to smithi16579500-29 2024-04-20T01:47:34.915 INFO:tasks.rados.rados.0.smithi165.stdout:28: finishing write tid 1 to smithi16579500-28 2024-04-20T01:47:34.915 INFO:tasks.rados.rados.0.smithi165.stdout:28: finishing write tid 2 to smithi16579500-28 2024-04-20T01:47:34.915 INFO:tasks.rados.rados.0.smithi165.stdout:29: finishing write tid 2 to smithi16579500-29 2024-04-20T01:47:34.915 INFO:tasks.rados.rados.0.smithi165.stdout:27: finishing write tid 3 to smithi16579500-27 2024-04-20T01:47:34.915 INFO:tasks.rados.rados.0.smithi165.stdout:27: oid 27 updating version 0 to 3 2024-04-20T01:47:34.915 INFO:tasks.rados.rados.0.smithi165.stdout:27: oid 27 updating version 3 to 4 2024-04-20T01:47:34.915 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 27 v 4 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-04-20T01:47:34.915 INFO:tasks.rados.rados.0.smithi165.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2024-04-20T01:47:34.915 INFO:tasks.rados.rados.0.smithi165.stdout:30: finishing write tid 1 to smithi16579500-30 2024-04-20T01:47:34.915 INFO:tasks.rados.rados.0.smithi165.stdout:30: finishing write tid 2 to smithi16579500-30 2024-04-20T01:47:34.915 INFO:tasks.rados.rados.0.smithi165.stdout:28: finishing write tid 3 to smithi16579500-28 2024-04-20T01:47:34.915 INFO:tasks.rados.rados.0.smithi165.stdout:28: oid 28 updating version 0 to 6 2024-04-20T01:47:34.915 INFO:tasks.rados.rados.0.smithi165.stdout:28: oid 28 version 6 is already newer than 5 2024-04-20T01:47:34.915 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 28 v 6 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-04-20T01:47:34.916 INFO:tasks.rados.rados.0.smithi165.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2024-04-20T01:47:34.916 INFO:tasks.rados.rados.0.smithi165.stdout:29: finishing write tid 3 to smithi16579500-29 2024-04-20T01:47:34.916 INFO:tasks.rados.rados.0.smithi165.stdout:29: oid 29 updating version 0 to 8 2024-04-20T01:47:34.916 INFO:tasks.rados.rados.0.smithi165.stdout:29: oid 29 version 8 is already newer than 7 2024-04-20T01:47:34.916 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 29 v 8 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-04-20T01:47:34.916 INFO:tasks.rados.rados.0.smithi165.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2024-04-20T01:47:34.916 INFO:tasks.rados.rados.0.smithi165.stdout:19: done (15 left) 2024-04-20T01:47:34.916 INFO:tasks.rados.rados.0.smithi165.stdout:20: done (14 left) 2024-04-20T01:47:34.916 INFO:tasks.rados.rados.0.smithi165.stdout:21: done (13 left) 2024-04-20T01:47:34.916 INFO:tasks.rados.rados.0.smithi165.stdout:22: done (12 left) 2024-04-20T01:47:34.916 INFO:tasks.rados.rados.0.smithi165.stdout:23: done (11 left) 2024-04-20T01:47:34.916 INFO:tasks.rados.rados.0.smithi165.stdout:24: done (10 left) 2024-04-20T01:47:34.916 INFO:tasks.rados.rados.0.smithi165.stdout:25: done (9 left) 2024-04-20T01:47:34.916 INFO:tasks.rados.rados.0.smithi165.stdout:26: done (8 left) 2024-04-20T01:47:34.916 INFO:tasks.rados.rados.0.smithi165.stdout:27: done (7 left) 2024-04-20T01:47:34.917 INFO:tasks.rados.rados.0.smithi165.stdout:28: done (6 left) 2024-04-20T01:47:34.917 INFO:tasks.rados.rados.0.smithi165.stdout:29: done (5 left) 2024-04-20T01:47:34.917 INFO:tasks.rados.rados.0.smithi165.stdout:34: write initial oid 34 2024-04-20T01:47:34.917 INFO:tasks.rados.rados.0.smithi165.stdout:34: seq_num 33 ranges {0=557056} 2024-04-20T01:47:34.917 INFO:tasks.rados.rados.0.smithi165.stdout:34: writing smithi16579500-34 from 0 to 557056 tid 1 2024-04-20T01:47:34.917 INFO:tasks.rados.rados.0.smithi165.stdout:35: write initial oid 35 2024-04-20T01:47:34.917 INFO:tasks.rados.rados.0.smithi165.stdout:35: seq_num 34 ranges {0=573440} 2024-04-20T01:47:34.917 INFO:tasks.rados.rados.0.smithi165.stdout:35: writing smithi16579500-35 from 0 to 573440 tid 1 2024-04-20T01:47:34.917 INFO:tasks.rados.rados.0.smithi165.stdout:36: write initial oid 36 2024-04-20T01:47:34.917 INFO:tasks.rados.rados.0.smithi165.stdout:36: seq_num 35 ranges {0=589824} 2024-04-20T01:47:34.917 INFO:tasks.rados.rados.0.smithi165.stdout:36: writing smithi16579500-36 from 0 to 589824 tid 1 2024-04-20T01:47:34.917 INFO:tasks.rados.rados.0.smithi165.stdout:37: write initial oid 37 2024-04-20T01:47:34.917 INFO:tasks.rados.rados.0.smithi165.stdout:37: seq_num 36 ranges {0=606208} 2024-04-20T01:47:34.918 INFO:tasks.rados.rados.0.smithi165.stdout:37: writing smithi16579500-37 from 0 to 606208 tid 1 2024-04-20T01:47:34.918 INFO:tasks.rados.rados.0.smithi165.stdout:38: write initial oid 38 2024-04-20T01:47:34.918 INFO:tasks.rados.rados.0.smithi165.stdout:38: seq_num 37 ranges {0=622592} 2024-04-20T01:47:34.918 INFO:tasks.rados.rados.0.smithi165.stdout:38: writing smithi16579500-38 from 0 to 622592 tid 1 2024-04-20T01:47:34.918 INFO:tasks.rados.rados.0.smithi165.stdout:39: write initial oid 39 2024-04-20T01:47:34.918 INFO:tasks.rados.rados.0.smithi165.stdout:39: seq_num 38 ranges {0=638976} 2024-04-20T01:47:34.918 INFO:tasks.rados.rados.0.smithi165.stdout:39: writing smithi16579500-39 from 0 to 638976 tid 1 2024-04-20T01:47:34.918 INFO:tasks.rados.rados.0.smithi165.stdout:40: write initial oid 40 2024-04-20T01:47:34.918 INFO:tasks.rados.rados.0.smithi165.stdout:40: seq_num 39 ranges {0=671744} 2024-04-20T01:47:34.918 INFO:tasks.rados.rados.0.smithi165.stdout:40: writing smithi16579500-40 from 0 to 671744 tid 1 2024-04-20T01:47:34.918 INFO:tasks.rados.rados.0.smithi165.stdout:41: write initial oid 41 2024-04-20T01:47:34.918 INFO:tasks.rados.rados.0.smithi165.stdout:41: seq_num 40 ranges {0=688128} 2024-04-20T01:47:34.918 INFO:tasks.rados.rados.0.smithi165.stdout:41: writing smithi16579500-41 from 0 to 688128 tid 1 2024-04-20T01:47:34.918 INFO:tasks.rados.rados.0.smithi165.stdout:42: write initial oid 42 2024-04-20T01:47:34.919 INFO:tasks.rados.rados.0.smithi165.stdout:42: seq_num 41 ranges {0=704512} 2024-04-20T01:47:34.919 INFO:tasks.rados.rados.0.smithi165.stdout:42: writing smithi16579500-42 from 0 to 704512 tid 1 2024-04-20T01:47:34.919 INFO:tasks.rados.rados.0.smithi165.stdout:43: write initial oid 43 2024-04-20T01:47:34.919 INFO:tasks.rados.rados.0.smithi165.stdout:43: seq_num 42 ranges {0=720896} 2024-04-20T01:47:34.919 INFO:tasks.rados.rados.0.smithi165.stdout:43: writing smithi16579500-43 from 0 to 720896 tid 1 2024-04-20T01:47:34.919 INFO:tasks.rados.rados.0.smithi165.stdout:44: write initial oid 44 2024-04-20T01:47:34.919 INFO:tasks.rados.rados.0.smithi165.stdout:44: seq_num 43 ranges {0=737280} 2024-04-20T01:47:34.919 INFO:tasks.rados.rados.0.smithi165.stdout:44: writing smithi16579500-44 from 0 to 737280 tid 1 2024-04-20T01:47:34.919 INFO:tasks.rados.rados.0.smithi165.stdout: waiting on 16 2024-04-20T01:47:34.919 INFO:tasks.rados.rados.0.smithi165.stdout:30: finishing write tid 3 to smithi16579500-30 2024-04-20T01:47:34.919 INFO:tasks.rados.rados.0.smithi165.stdout:30: oid 30 updating version 0 to 3 2024-04-20T01:47:34.919 INFO:tasks.rados.rados.0.smithi165.stdout:30: oid 30 updating version 3 to 4 2024-04-20T01:47:34.919 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 30 v 4 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-04-20T01:47:34.920 INFO:tasks.rados.rados.0.smithi165.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2024-04-20T01:47:34.920 INFO:tasks.rados.rados.0.smithi165.stdout:30: done (15 left) 2024-04-20T01:47:34.920 INFO:tasks.rados.rados.0.smithi165.stdout:45: write initial oid 45 2024-04-20T01:47:34.920 INFO:tasks.rados.rados.0.smithi165.stdout:45: seq_num 44 ranges {0=753664} 2024-04-20T01:47:34.920 INFO:tasks.rados.rados.0.smithi165.stdout:45: writing smithi16579500-45 from 0 to 753664 tid 1 2024-04-20T01:47:34.920 INFO:tasks.rados.rados.0.smithi165.stdout: waiting on 16 2024-04-20T01:47:34.920 INFO:tasks.rados.rados.0.smithi165.stdout:31: finishing write tid 1 to smithi16579500-31 2024-04-20T01:47:34.920 INFO:tasks.rados.rados.0.smithi165.stdout:31: finishing write tid 2 to smithi16579500-31 2024-04-20T01:47:34.920 INFO:tasks.rados.rados.0.smithi165.stdout:31: finishing write tid 3 to smithi16579500-31 2024-04-20T01:47:34.920 INFO:tasks.rados.rados.0.smithi165.stdout:31: oid 31 updating version 0 to 7 2024-04-20T01:47:34.920 INFO:tasks.rados.rados.0.smithi165.stdout:31: oid 31 updating version 7 to 8 2024-04-20T01:47:34.920 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 31 v 8 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-04-20T01:47:34.920 INFO:tasks.rados.rados.0.smithi165.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2024-04-20T01:47:34.921 INFO:tasks.rados.rados.0.smithi165.stdout:33: finishing write tid 1 to smithi16579500-33 2024-04-20T01:47:34.921 INFO:tasks.rados.rados.0.smithi165.stdout:32: finishing write tid 1 to smithi16579500-32 2024-04-20T01:47:34.921 INFO:tasks.rados.rados.0.smithi165.stdout:33: finishing write tid 2 to smithi16579500-33 2024-04-20T01:47:34.921 INFO:tasks.rados.rados.0.smithi165.stdout:32: finishing write tid 2 to smithi16579500-32 2024-04-20T01:47:34.921 INFO:tasks.rados.rados.0.smithi165.stdout:33: finishing write tid 3 to smithi16579500-33 2024-04-20T01:47:34.921 INFO:tasks.rados.rados.0.smithi165.stdout:33: oid 33 updating version 0 to 10 2024-04-20T01:47:34.921 INFO:tasks.rados.rados.0.smithi165.stdout:33: oid 33 version 10 is already newer than 9 2024-04-20T01:47:34.921 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 33 v 10 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-04-20T01:47:34.921 INFO:tasks.rados.rados.0.smithi165.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2024-04-20T01:47:34.921 INFO:tasks.rados.rados.0.smithi165.stdout:32: finishing write tid 3 to smithi16579500-32 2024-04-20T01:47:34.921 INFO:tasks.rados.rados.0.smithi165.stdout:32: oid 32 updating version 0 to 3 2024-04-20T01:47:34.921 INFO:tasks.rados.rados.0.smithi165.stdout:32: oid 32 updating version 3 to 4 2024-04-20T01:47:34.922 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 32 v 4 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-04-20T01:47:34.922 INFO:tasks.rados.rados.0.smithi165.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2024-04-20T01:47:34.922 INFO:tasks.rados.rados.0.smithi165.stdout:34: finishing write tid 1 to smithi16579500-34 2024-04-20T01:47:34.922 INFO:tasks.rados.rados.0.smithi165.stdout:34: finishing write tid 2 to smithi16579500-34 2024-04-20T01:47:34.922 INFO:tasks.rados.rados.0.smithi165.stdout:34: finishing write tid 3 to smithi16579500-34 2024-04-20T01:47:34.922 INFO:tasks.rados.rados.0.smithi165.stdout:34: oid 34 updating version 0 to 10 2024-04-20T01:47:34.922 INFO:tasks.rados.rados.0.smithi165.stdout:34: oid 34 version 10 is already newer than 9 2024-04-20T01:47:34.922 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 34 v 10 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-04-20T01:47:34.922 INFO:tasks.rados.rados.0.smithi165.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2024-04-20T01:47:34.922 INFO:tasks.rados.rados.0.smithi165.stdout:36: finishing write tid 1 to smithi16579500-36 2024-04-20T01:47:34.922 INFO:tasks.rados.rados.0.smithi165.stdout:36: finishing write tid 2 to smithi16579500-36 2024-04-20T01:47:34.922 INFO:tasks.rados.rados.0.smithi165.stdout:36: finishing write tid 3 to smithi16579500-36 2024-04-20T01:47:34.922 INFO:tasks.rados.rados.0.smithi165.stdout:36: oid 36 updating version 0 to 1 2024-04-20T01:47:34.922 INFO:tasks.rados.rados.0.smithi165.stdout:36: oid 36 updating version 1 to 2 2024-04-20T01:47:34.923 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 36 v 2 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-04-20T01:47:34.923 INFO:tasks.rados.rados.0.smithi165.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2024-04-20T01:47:34.923 INFO:tasks.rados.rados.0.smithi165.stdout:38: finishing write tid 1 to smithi16579500-38 2024-04-20T01:47:34.923 INFO:tasks.rados.rados.0.smithi165.stdout:38: finishing write tid 2 to smithi16579500-38 2024-04-20T01:47:34.923 INFO:tasks.rados.rados.0.smithi165.stdout:38: finishing write tid 3 to smithi16579500-38 2024-04-20T01:47:34.923 INFO:tasks.rados.rados.0.smithi165.stdout:38: oid 38 updating version 0 to 11 2024-04-20T01:47:34.923 INFO:tasks.rados.rados.0.smithi165.stdout:38: oid 38 updating version 11 to 12 2024-04-20T01:47:34.923 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 38 v 12 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-04-20T01:47:34.923 INFO:tasks.rados.rados.0.smithi165.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2024-04-20T01:47:34.923 INFO:tasks.rados.rados.0.smithi165.stdout:39: finishing write tid 1 to smithi16579500-39 2024-04-20T01:47:34.923 INFO:tasks.rados.rados.0.smithi165.stdout:40: finishing write tid 1 to smithi16579500-40 2024-04-20T01:47:34.923 INFO:tasks.rados.rados.0.smithi165.stdout:40: finishing write tid 2 to smithi16579500-40 2024-04-20T01:47:34.923 INFO:tasks.rados.rados.0.smithi165.stdout:39: finishing write tid 2 to smithi16579500-39 2024-04-20T01:47:34.924 INFO:tasks.rados.rados.0.smithi165.stdout:40: finishing write tid 3 to smithi16579500-40 2024-04-20T01:47:34.924 INFO:tasks.rados.rados.0.smithi165.stdout:40: oid 40 updating version 0 to 2 2024-04-20T01:47:34.924 INFO:tasks.rados.rados.0.smithi165.stdout:40: oid 40 version 2 is already newer than 1 2024-04-20T01:47:34.924 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 40 v 2 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-04-20T01:47:34.924 INFO:tasks.rados.rados.0.smithi165.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2024-04-20T01:47:34.924 INFO:tasks.rados.rados.0.smithi165.stdout:39: finishing write tid 3 to smithi16579500-39 2024-04-20T01:47:34.924 INFO:tasks.rados.rados.0.smithi165.stdout:39: oid 39 updating version 0 to 10 2024-04-20T01:47:34.924 INFO:tasks.rados.rados.0.smithi165.stdout:39: oid 39 version 10 is already newer than 9 2024-04-20T01:47:34.924 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 39 v 10 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-04-20T01:47:34.924 INFO:tasks.rados.rados.0.smithi165.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2024-04-20T01:47:34.924 INFO:tasks.rados.rados.0.smithi165.stdout:41: finishing write tid 1 to smithi16579500-41 2024-04-20T01:47:34.924 INFO:tasks.rados.rados.0.smithi165.stdout:42: finishing write tid 1 to smithi16579500-42 2024-04-20T01:47:34.924 INFO:tasks.rados.rados.0.smithi165.stdout:42: finishing write tid 2 to smithi16579500-42 2024-04-20T01:47:34.924 INFO:tasks.rados.rados.0.smithi165.stdout:41: finishing write tid 2 to smithi16579500-41 2024-04-20T01:47:34.925 INFO:tasks.rados.rados.0.smithi165.stdout:41: finishing write tid 3 to smithi16579500-41 2024-04-20T01:47:34.925 INFO:tasks.rados.rados.0.smithi165.stdout:41: oid 41 updating version 0 to 12 2024-04-20T01:47:34.925 INFO:tasks.rados.rados.0.smithi165.stdout:41: oid 41 version 12 is already newer than 11 2024-04-20T01:47:34.925 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 41 v 12 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-04-20T01:47:34.925 INFO:tasks.rados.rados.0.smithi165.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2024-04-20T01:47:34.925 INFO:tasks.rados.rados.0.smithi165.stdout:43: finishing write tid 1 to smithi16579500-43 2024-04-20T01:47:34.925 INFO:tasks.rados.rados.0.smithi165.stdout:42: finishing write tid 3 to smithi16579500-42 2024-04-20T01:47:34.925 INFO:tasks.rados.rados.0.smithi165.stdout:42: oid 42 updating version 0 to 5 2024-04-20T01:47:34.925 INFO:tasks.rados.rados.0.smithi165.stdout:42: oid 42 updating version 5 to 6 2024-04-20T01:47:34.925 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 42 v 6 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-04-20T01:47:34.925 INFO:tasks.rados.rados.0.smithi165.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2024-04-20T01:47:34.925 INFO:tasks.rados.rados.0.smithi165.stdout:43: finishing write tid 2 to smithi16579500-43 2024-04-20T01:47:34.925 INFO:tasks.rados.rados.0.smithi165.stdout:31: done (15 left) 2024-04-20T01:47:34.925 INFO:tasks.rados.rados.0.smithi165.stdout:32: done (14 left) 2024-04-20T01:47:34.925 INFO:tasks.rados.rados.0.smithi165.stdout:33: done (13 left) 2024-04-20T01:47:34.926 INFO:tasks.rados.rados.0.smithi165.stdout:34: done (12 left) 2024-04-20T01:47:34.926 INFO:tasks.rados.rados.0.smithi165.stdout:36: done (11 left) 2024-04-20T01:47:34.926 INFO:tasks.rados.rados.0.smithi165.stdout:38: done (10 left) 2024-04-20T01:47:34.926 INFO:tasks.rados.rados.0.smithi165.stdout:39: done (9 left) 2024-04-20T01:47:34.926 INFO:tasks.rados.rados.0.smithi165.stdout:40: done (8 left) 2024-04-20T01:47:34.926 INFO:tasks.rados.rados.0.smithi165.stdout:41: done (7 left) 2024-04-20T01:47:34.926 INFO:tasks.rados.rados.0.smithi165.stdout:42: done (6 left) 2024-04-20T01:47:34.926 INFO:tasks.rados.rados.0.smithi165.stdout:46: write initial oid 46 2024-04-20T01:47:34.927 INFO:tasks.rados.rados.0.smithi165.stdout:46: seq_num 45 ranges {0=770048} 2024-04-20T01:47:34.927 INFO:tasks.rados.rados.0.smithi165.stdout:46: writing smithi16579500-46 from 0 to 770048 tid 1 2024-04-20T01:47:34.927 INFO:tasks.rados.rados.0.smithi165.stdout:47: write initial oid 47 2024-04-20T01:47:34.927 INFO:tasks.rados.rados.0.smithi165.stdout:47: seq_num 46 ranges {0=786432} 2024-04-20T01:47:34.927 INFO:tasks.rados.rados.0.smithi165.stdout:47: writing smithi16579500-47 from 0 to 786432 tid 1 2024-04-20T01:47:34.927 INFO:tasks.rados.rados.0.smithi165.stdout:48: write initial oid 48 2024-04-20T01:47:34.928 INFO:tasks.rados.rados.0.smithi165.stdout:48: seq_num 47 ranges {0=425984,425984=376832} 2024-04-20T01:47:34.928 INFO:tasks.rados.rados.0.smithi165.stdout:48: writing smithi16579500-48 from 0 to 425984 tid 1 2024-04-20T01:47:34.928 INFO:tasks.rados.rados.0.smithi165.stdout:48: writing smithi16579500-48 from 425984 to 802816 tid 2 2024-04-20T01:47:34.928 INFO:tasks.rados.rados.0.smithi165.stdout:49: write initial oid 49 2024-04-20T01:47:34.928 INFO:tasks.rados.rados.0.smithi165.stdout:49: seq_num 48 ranges {0=442368,442368=376832} 2024-04-20T01:47:34.928 INFO:tasks.rados.rados.0.smithi165.stdout:49: writing smithi16579500-49 from 0 to 442368 tid 1 2024-04-20T01:47:34.928 INFO:tasks.rados.rados.0.smithi165.stdout:49: writing smithi16579500-49 from 442368 to 819200 tid 2 2024-04-20T01:47:34.928 INFO:tasks.rados.rados.0.smithi165.stdout:50: write initial oid 50 2024-04-20T01:47:34.928 INFO:tasks.rados.rados.0.smithi165.stdout:50: seq_num 49 ranges {0=458752,458752=376832} 2024-04-20T01:47:34.929 INFO:tasks.rados.rados.0.smithi165.stdout:50: writing smithi16579500-50 from 0 to 458752 tid 1 2024-04-20T01:47:34.929 INFO:tasks.rados.rados.0.smithi165.stdout:50: writing smithi16579500-50 from 458752 to 835584 tid 2 2024-04-20T01:47:34.929 INFO:tasks.rados.rados.0.smithi165.stdout:51: copy_from oid 6 from oid 33 current snap is 0 2024-04-20T01:47:34.929 INFO:tasks.rados.rados.0.smithi165.stdout:52: rollback oid 24 current snap is 0 2024-04-20T01:47:34.929 INFO:tasks.rados.rados.0.smithi165.stdout:52: done (12 left) 2024-04-20T01:47:34.929 INFO:tasks.rados.rados.0.smithi165.stdout:53: read oid 34 snap -1 2024-04-20T01:47:34.929 INFO:tasks.rados.rados.0.smithi165.stdout:53: expect (ObjNum 33 snap 0 seq_num 33) 2024-04-20T01:47:34.929 INFO:tasks.rados.rados.0.smithi165.stdout:44: finishing write tid 1 to smithi16579500-44 2024-04-20T01:47:34.929 INFO:tasks.rados.rados.0.smithi165.stdout:43: finishing write tid 3 to smithi16579500-43 2024-04-20T01:47:34.929 INFO:tasks.rados.rados.0.smithi165.stdout:43: oid 43 updating version 0 to 14 2024-04-20T01:47:34.929 INFO:tasks.rados.rados.0.smithi165.stdout:43: oid 43 version 14 is already newer than 13 2024-04-20T01:47:34.929 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 43 v 14 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-04-20T01:47:34.929 INFO:tasks.rados.rados.0.smithi165.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2024-04-20T01:47:34.929 INFO:tasks.rados.rados.0.smithi165.stdout:44: finishing write tid 2 to smithi16579500-44 2024-04-20T01:47:34.930 INFO:tasks.rados.rados.0.smithi165.stdout:43: done (12 left) 2024-04-20T01:47:34.930 INFO:tasks.rados.rados.0.smithi165.stdout:54: read oid 5 snap -1 2024-04-20T01:47:34.930 INFO:tasks.rados.rados.0.smithi165.stdout:54: expect (ObjNum 4 snap 0 seq_num 4) 2024-04-20T01:47:34.930 INFO:tasks.rados.rados.0.smithi165.stdout:55: read oid 13 snap -1 2024-04-20T01:47:34.930 INFO:tasks.rados.rados.0.smithi165.stdout:55: expect (ObjNum 12 snap 0 seq_num 12) 2024-04-20T01:47:34.930 INFO:tasks.rados.rados.0.smithi165.stdout:56: setattr oid 28 current snap is 0 2024-04-20T01:47:34.930 INFO:tasks.rados.rados.0.smithi165.stdout:44: finishing write tid 3 to smithi16579500-44 2024-04-20T01:47:34.930 INFO:tasks.rados.rados.0.smithi165.stdout:44: oid 44 updating version 0 to 1 2024-04-20T01:47:34.930 INFO:tasks.rados.rados.0.smithi165.stdout:44: oid 44 updating version 1 to 2 2024-04-20T01:47:34.930 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 44 v 2 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-04-20T01:47:34.930 INFO:tasks.rados.rados.0.smithi165.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2024-04-20T01:47:34.930 INFO:tasks.rados.rados.0.smithi165.stdout:45: finishing write tid 1 to smithi16579500-45 2024-04-20T01:47:34.930 INFO:tasks.rados.rados.0.smithi165.stdout:45: finishing write tid 2 to smithi16579500-45 2024-04-20T01:47:34.930 INFO:tasks.rados.rados.0.smithi165.stdout:46: finishing write tid 1 to smithi16579500-46 2024-04-20T01:47:34.931 INFO:tasks.rados.rados.0.smithi165.stdout:46: finishing write tid 2 to smithi16579500-46 2024-04-20T01:47:34.931 INFO:tasks.rados.rados.0.smithi165.stdout:45: finishing write tid 3 to smithi16579500-45 2024-04-20T01:47:34.931 INFO:tasks.rados.rados.0.smithi165.stdout:45: oid 45 updating version 0 to 12 2024-04-20T01:47:34.931 INFO:tasks.rados.rados.0.smithi165.stdout:45: oid 45 version 12 is already newer than 11 2024-04-20T01:47:34.931 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 45 v 12 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-04-20T01:47:34.931 INFO:tasks.rados.rados.0.smithi165.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2024-04-20T01:47:34.931 INFO:tasks.rados.rados.0.smithi165.stdout:46: finishing write tid 3 to smithi16579500-46 2024-04-20T01:47:34.931 INFO:tasks.rados.rados.0.smithi165.stdout:46: oid 46 updating version 0 to 5 2024-04-20T01:47:34.931 INFO:tasks.rados.rados.0.smithi165.stdout:46: oid 46 updating version 5 to 6 2024-04-20T01:47:34.931 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 46 v 6 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-04-20T01:47:34.931 INFO:tasks.rados.rados.0.smithi165.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2024-04-20T01:47:34.931 INFO:tasks.rados.rados.0.smithi165.stdout:47: finishing write tid 1 to smithi16579500-47 2024-04-20T01:47:34.931 INFO:tasks.rados.rados.0.smithi165.stdout:47: finishing write tid 2 to smithi16579500-47 2024-04-20T01:47:34.931 INFO:tasks.rados.rados.0.smithi165.stdout:47: finishing write tid 3 to smithi16579500-47 2024-04-20T01:47:34.931 INFO:tasks.rados.rados.0.smithi165.stdout:47: oid 47 updating version 0 to 4 2024-04-20T01:47:34.932 INFO:tasks.rados.rados.0.smithi165.stdout:47: oid 47 version 4 is already newer than 3 2024-04-20T01:47:34.932 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 47 v 4 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-04-20T01:47:34.932 INFO:tasks.rados.rados.0.smithi165.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2024-04-20T01:47:34.932 INFO:tasks.rados.rados.0.smithi165.stdout:49: finishing write tid 1 to smithi16579500-49 2024-04-20T01:47:34.932 INFO:tasks.rados.rados.0.smithi165.stdout:49: finishing write tid 2 to smithi16579500-49 2024-04-20T01:47:34.932 INFO:tasks.rados.rados.0.smithi165.stdout:49: finishing write tid 3 to smithi16579500-49 2024-04-20T01:47:34.932 INFO:tasks.rados.rados.0.smithi165.stdout:50: finishing write tid 1 to smithi16579500-50 2024-04-20T01:47:34.932 INFO:tasks.rados.rados.0.smithi165.stdout:54: expect (ObjNum 4 snap 0 seq_num 4) 2024-04-20T01:47:34.932 INFO:tasks.rados.rados.0.smithi165.stdout:49: finishing write tid 4 to smithi16579500-49 2024-04-20T01:47:34.932 INFO:tasks.rados.rados.0.smithi165.stdout:49: oid 49 updating version 0 to 3 2024-04-20T01:47:34.932 INFO:tasks.rados.rados.0.smithi165.stdout:49: oid 49 updating version 3 to 4 2024-04-20T01:47:34.932 INFO:tasks.rados.rados.0.smithi165.stdout:49: oid 49 updating version 4 to 5 2024-04-20T01:47:34.932 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 49 v 5 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-04-20T01:47:34.932 INFO:tasks.rados.rados.0.smithi165.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2024-04-20T01:47:34.933 INFO:tasks.rados.rados.0.smithi165.stdout:44: done (14 left) 2024-04-20T01:47:34.933 INFO:tasks.rados.rados.0.smithi165.stdout:45: done (13 left) 2024-04-20T01:47:34.933 INFO:tasks.rados.rados.0.smithi165.stdout:46: done (12 left) 2024-04-20T01:47:34.933 INFO:tasks.rados.rados.0.smithi165.stdout:47: done (11 left) 2024-04-20T01:47:34.933 INFO:tasks.rados.rados.0.smithi165.stdout:49: done (10 left) 2024-04-20T01:47:34.933 INFO:tasks.rados.rados.0.smithi165.stdout:54: done (9 left) 2024-04-20T01:47:34.933 INFO:tasks.rados.rados.0.smithi165.stdout:57: snap_create 2024-04-20T01:47:34.933 INFO:tasks.rados.rados.0.smithi165.stdout:50: finishing write tid 2 to smithi16579500-50 2024-04-20T01:47:34.933 INFO:tasks.rados.rados.0.smithi165.stdout:50: finishing write tid 3 to smithi16579500-50 2024-04-20T01:47:34.933 INFO:tasks.rados.rados.0.smithi165.stdout:51: finishing copy_from racing read to smithi16579500-6 2024-04-20T01:47:34.933 INFO:tasks.rados.rados.0.smithi165.stdout:50: finishing write tid 4 to smithi16579500-50 2024-04-20T01:47:34.933 INFO:tasks.rados.rados.0.smithi165.stdout:50: oid 50 updating version 0 to 6 2024-04-20T01:47:34.933 INFO:tasks.rados.rados.0.smithi165.stdout:50: oid 50 version 6 is already newer than 5 2024-04-20T01:47:34.933 INFO:tasks.rados.rados.0.smithi165.stdout:50: oid 50 updating version 6 to 7 2024-04-20T01:47:34.934 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 50 v 7 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-04-20T01:47:34.934 INFO:tasks.rados.rados.0.smithi165.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2024-04-20T01:47:34.934 INFO:tasks.rados.rados.0.smithi165.stdout:53: expect (ObjNum 33 snap 0 seq_num 33) 2024-04-20T01:47:34.934 INFO:tasks.rados.rados.0.smithi165.stdout:51: finishing copy_from to smithi16579500-6 2024-04-20T01:47:34.934 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 6 v 10 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-04-20T01:47:34.934 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 28 v 11 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-04-20T01:47:34.934 INFO:tasks.rados.rados.0.smithi165.stdout:55: expect (ObjNum 12 snap 0 seq_num 12) 2024-04-20T01:47:34.951 INFO:tasks.rados.rados.0.smithi165.stdout:12: finishing write tid 1 to smithi16579500-12 2024-04-20T01:47:34.954 INFO:tasks.rados.rados.0.smithi165.stdout:12: finishing write tid 2 to smithi16579500-12 2024-04-20T01:47:34.956 INFO:tasks.rados.rados.0.smithi165.stdout:35: finishing write tid 1 to smithi16579500-35 2024-04-20T01:47:34.956 INFO:tasks.rados.rados.0.smithi165.stdout:12: finishing write tid 3 to smithi16579500-12 2024-04-20T01:47:34.956 INFO:tasks.rados.rados.0.smithi165.stdout:12: oid 12 updating version 0 to 2 2024-04-20T01:47:34.956 INFO:tasks.rados.rados.0.smithi165.stdout:12: oid 12 version 2 is already newer than 1 2024-04-20T01:47:34.956 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 12 v 2 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-04-20T01:47:34.956 INFO:tasks.rados.rados.0.smithi165.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2024-04-20T01:47:34.959 INFO:tasks.rados.rados.0.smithi165.stdout:35: finishing write tid 2 to smithi16579500-35 2024-04-20T01:47:34.960 INFO:tasks.rados.rados.0.smithi165.stdout:37: finishing write tid 1 to smithi16579500-37 2024-04-20T01:47:34.960 INFO:tasks.rados.rados.0.smithi165.stdout:37: finishing write tid 2 to smithi16579500-37 2024-04-20T01:47:34.962 INFO:tasks.rados.rados.0.smithi165.stdout:35: finishing write tid 3 to smithi16579500-35 2024-04-20T01:47:34.962 INFO:tasks.rados.rados.0.smithi165.stdout:35: oid 35 updating version 0 to 4 2024-04-20T01:47:34.962 INFO:tasks.rados.rados.0.smithi165.stdout:35: oid 35 version 4 is already newer than 3 2024-04-20T01:47:34.962 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 35 v 4 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-04-20T01:47:34.962 INFO:tasks.rados.rados.0.smithi165.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2024-04-20T01:47:34.962 INFO:tasks.rados.rados.0.smithi165.stdout:48: finishing write tid 1 to smithi16579500-48 2024-04-20T01:47:34.962 INFO:tasks.rados.rados.0.smithi165.stdout:48: finishing write tid 2 to smithi16579500-48 2024-04-20T01:47:34.962 INFO:tasks.rados.rados.0.smithi165.stdout:37: finishing write tid 3 to smithi16579500-37 2024-04-20T01:47:34.962 INFO:tasks.rados.rados.0.smithi165.stdout:37: oid 37 updating version 0 to 5 2024-04-20T01:47:34.962 INFO:tasks.rados.rados.0.smithi165.stdout:37: oid 37 updating version 5 to 6 2024-04-20T01:47:34.962 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 37 v 6 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-04-20T01:47:34.962 INFO:tasks.rados.rados.0.smithi165.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2024-04-20T01:47:34.963 INFO:tasks.rados.rados.0.smithi165.stdout:48: finishing write tid 3 to smithi16579500-48 2024-04-20T01:47:34.964 INFO:tasks.rados.rados.0.smithi165.stdout:48: finishing write tid 4 to smithi16579500-48 2024-04-20T01:47:34.965 INFO:tasks.rados.rados.0.smithi165.stdout:48: oid 48 updating version 0 to 7 2024-04-20T01:47:34.965 INFO:tasks.rados.rados.0.smithi165.stdout:48: oid 48 updating version 7 to 9 2024-04-20T01:47:34.965 INFO:tasks.rados.rados.0.smithi165.stdout:48: oid 48 version 9 is already newer than 8 2024-04-20T01:47:34.965 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 48 v 9 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-04-20T01:47:34.965 INFO:tasks.rados.rados.0.smithi165.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2024-04-20T01:47:34.993 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:47:35.007+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:47:35.041 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:47:35.069 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:35.069 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:35.070 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:47:35.080 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-20T01:47:35.091 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:47:35.094 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:35.108+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:47:35.111 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:47:35.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:47:35.125+0000 7fd275cdc700 0 -- 172.21.15.44:0/3916084304 >> [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] conn(0x7fd27021f460 msgr2=0x7fd270221850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-20T01:47:35.195 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:35.209+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:35.249 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:35.249 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:35.259 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-20T01:47:35.295 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:47:35.310+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:47:35.333 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:47:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:47:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:47:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:47:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:47:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:47:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:47:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:47:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:47:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:47:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:47:35.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:47:35.336 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:47:35.336 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:47:35.336 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:47:35.336 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:47:35.336 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:47:35.389 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:35.390 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:35.396 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:47:35.410+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:47:35.400 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-20T01:47:35.497 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:47:35.511+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:47:35.530 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:35.530 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:35.540 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-20T01:47:35.598 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:35.612+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:47:35.669 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:35.669 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:35.679 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-20T01:47:35.699 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:47:35.713+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:47:35.740 INFO:tasks.rados.rados.0.smithi165.stdout:12: done (9 left) 2024-04-20T01:47:35.740 INFO:tasks.rados.rados.0.smithi165.stdout:35: done (8 left) 2024-04-20T01:47:35.740 INFO:tasks.rados.rados.0.smithi165.stdout:37: done (7 left) 2024-04-20T01:47:35.740 INFO:tasks.rados.rados.0.smithi165.stdout:48: done (6 left) 2024-04-20T01:47:35.740 INFO:tasks.rados.rados.0.smithi165.stdout:50: done (5 left) 2024-04-20T01:47:35.740 INFO:tasks.rados.rados.0.smithi165.stdout:51: done (4 left) 2024-04-20T01:47:35.740 INFO:tasks.rados.rados.0.smithi165.stdout:53: done (3 left) 2024-04-20T01:47:35.740 INFO:tasks.rados.rados.0.smithi165.stdout:55: done (2 left) 2024-04-20T01:47:35.741 INFO:tasks.rados.rados.0.smithi165.stdout:56: done (1 left) 2024-04-20T01:47:35.741 INFO:tasks.rados.rados.0.smithi165.stdout:57: done (0 left) 2024-04-20T01:47:35.741 INFO:tasks.rados.rados.0.smithi165.stdout:58: read oid 30 snap -1 2024-04-20T01:47:35.741 INFO:tasks.rados.rados.0.smithi165.stdout:58: expect (ObjNum 29 snap 0 seq_num 29) 2024-04-20T01:47:35.741 INFO:tasks.rados.rados.0.smithi165.stdout:59: copy_from oid 17 from oid 47 current snap is 1 2024-04-20T01:47:35.741 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 38 current snap is 1 2024-04-20T01:47:35.741 INFO:tasks.rados.rados.0.smithi165.stdout:60: seq_num 50 ranges {622592=475136,1097728=376832} 2024-04-20T01:47:35.744 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.1 to 1 (10000) 2024-04-20T01:47:35.744 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is unset 2024-04-20T01:47:35.746 INFO:tasks.rados.rados.0.smithi165.stdout:60: writing smithi16579500-38 from 622592 to 1097728 tid 1 2024-04-20T01:47:35.748 INFO:tasks.rados.rados.0.smithi165.stdout:60: writing smithi16579500-38 from 1097728 to 1474560 tid 2 2024-04-20T01:47:35.748 INFO:tasks.rados.rados.0.smithi165.stdout:58: expect (ObjNum 29 snap 0 seq_num 29) 2024-04-20T01:47:35.759 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-20T01:47:35.763 INFO:tasks.rados.rados.0.smithi165.stdout:59: finishing copy_from racing read to smithi16579500-17 2024-04-20T01:47:35.763 INFO:tasks.rados.rados.0.smithi165.stdout:59: finishing copy_from to smithi16579500-17 2024-04-20T01:47:35.763 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 17 v 4 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-04-20T01:47:35.763 INFO:tasks.rados.rados.0.smithi165.stdout:60: finishing write tid 1 to smithi16579500-38 2024-04-20T01:47:35.763 INFO:tasks.rados.rados.0.smithi165.stdout:60: finishing write tid 2 to smithi16579500-38 2024-04-20T01:47:35.763 INFO:tasks.rados.rados.0.smithi165.stdout:58: done (2 left) 2024-04-20T01:47:35.763 INFO:tasks.rados.rados.0.smithi165.stdout:59: done (1 left) 2024-04-20T01:47:35.763 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 7 current snap is 1 2024-04-20T01:47:35.763 INFO:tasks.rados.rados.0.smithi165.stdout:60: finishing write tid 3 to smithi16579500-38 2024-04-20T01:47:35.763 INFO:tasks.rados.rados.0.smithi165.stdout:61: seq_num 51 ranges {114688=491520,606208=376832} 2024-04-20T01:47:35.766 INFO:tasks.rados.rados.0.smithi165.stdout:61: writing smithi16579500-7 from 114688 to 606208 tid 1 2024-04-20T01:47:35.768 INFO:tasks.rados.rados.0.smithi165.stdout:61: writing smithi16579500-7 from 606208 to 983040 tid 2 2024-04-20T01:47:35.768 INFO:tasks.rados.rados.0.smithi165.stdout:62: read oid 32 snap -1 2024-04-20T01:47:35.769 INFO:tasks.rados.rados.0.smithi165.stdout:62: expect (ObjNum 31 snap 0 seq_num 31) 2024-04-20T01:47:35.769 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 47 current snap is 1 2024-04-20T01:47:35.769 INFO:tasks.rados.rados.0.smithi165.stdout:60: finishing write tid 4 to smithi16579500-38 2024-04-20T01:47:35.769 INFO:tasks.rados.rados.0.smithi165.stdout:60: oid 38 updating version 0 to 17 2024-04-20T01:47:35.769 INFO:tasks.rados.rados.0.smithi165.stdout:60: oid 38 updating version 17 to 18 2024-04-20T01:47:35.769 INFO:tasks.rados.rados.0.smithi165.stdout:60: oid 38 version 18 is already newer than 16 2024-04-20T01:47:35.769 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 38 v 18 (ObjNum 50 snap 1 seq_num 50) dirty exists 2024-04-20T01:47:35.769 INFO:tasks.rados.rados.0.smithi165.stdout:60: left oid 38 (ObjNum 50 snap 1 seq_num 50) 2024-04-20T01:47:35.769 INFO:tasks.rados.rados.0.smithi165.stdout:63: seq_num 52 ranges {786432=507904,1294336=376832} 2024-04-20T01:47:35.776 INFO:tasks.rados.rados.0.smithi165.stdout:63: writing smithi16579500-47 from 786432 to 1294336 tid 1 2024-04-20T01:47:35.778 INFO:tasks.rados.rados.0.smithi165.stdout:63: writing smithi16579500-47 from 1294336 to 1671168 tid 2 2024-04-20T01:47:35.778 INFO:tasks.rados.rados.0.smithi165.stdout:60: done (3 left) 2024-04-20T01:47:35.779 INFO:tasks.rados.rados.0.smithi165.stdout:64: delete oid 18 current snap is 1 2024-04-20T01:47:35.779 INFO:tasks.rados.rados.0.smithi165.stdout:61: finishing write tid 1 to smithi16579500-7 2024-04-20T01:47:35.779 INFO:tasks.rados.rados.0.smithi165.stdout:61: finishing write tid 2 to smithi16579500-7 2024-04-20T01:47:35.779 INFO:tasks.rados.rados.0.smithi165.stdout:61: finishing write tid 3 to smithi16579500-7 2024-04-20T01:47:35.779 INFO:tasks.rados.rados.0.smithi165.stdout:61: finishing write tid 4 to smithi16579500-7 2024-04-20T01:47:35.779 INFO:tasks.rados.rados.0.smithi165.stdout:61: oid 7 updating version 0 to 5 2024-04-20T01:47:35.779 INFO:tasks.rados.rados.0.smithi165.stdout:61: oid 7 updating version 5 to 6 2024-04-20T01:47:35.779 INFO:tasks.rados.rados.0.smithi165.stdout:61: oid 7 version 6 is already newer than 4 2024-04-20T01:47:35.779 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 7 v 6 (ObjNum 51 snap 1 seq_num 51) dirty exists 2024-04-20T01:47:35.779 INFO:tasks.rados.rados.0.smithi165.stdout:61: left oid 7 (ObjNum 51 snap 1 seq_num 51) 2024-04-20T01:47:35.779 INFO:tasks.rados.rados.0.smithi165.stdout:62: expect (ObjNum 31 snap 0 seq_num 31) 2024-04-20T01:47:35.794 INFO:tasks.rados.rados.0.smithi165.stdout:63: finishing write tid 1 to smithi16579500-47 2024-04-20T01:47:35.794 INFO:tasks.rados.rados.0.smithi165.stdout:63: finishing write tid 2 to smithi16579500-47 2024-04-20T01:47:35.794 INFO:tasks.rados.rados.0.smithi165.stdout:63: finishing write tid 3 to smithi16579500-47 2024-04-20T01:47:35.794 INFO:tasks.rados.rados.0.smithi165.stdout:63: finishing write tid 4 to smithi16579500-47 2024-04-20T01:47:35.794 INFO:tasks.rados.rados.0.smithi165.stdout:63: oid 47 updating version 0 to 10 2024-04-20T01:47:35.794 INFO:tasks.rados.rados.0.smithi165.stdout:63: oid 47 version 10 is already newer than 9 2024-04-20T01:47:35.795 INFO:tasks.rados.rados.0.smithi165.stdout:63: oid 47 updating version 10 to 11 2024-04-20T01:47:35.795 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 47 v 11 (ObjNum 52 snap 1 seq_num 52) dirty exists 2024-04-20T01:47:35.795 INFO:tasks.rados.rados.0.smithi165.stdout:63: left oid 47 (ObjNum 52 snap 1 seq_num 52) 2024-04-20T01:47:35.795 INFO:tasks.rados.rados.0.smithi165.stdout:61: done (3 left) 2024-04-20T01:47:35.795 INFO:tasks.rados.rados.0.smithi165.stdout:62: done (2 left) 2024-04-20T01:47:35.795 INFO:tasks.rados.rados.0.smithi165.stdout:63: done (1 left) 2024-04-20T01:47:35.795 INFO:tasks.rados.rados.0.smithi165.stdout:64: done (0 left) 2024-04-20T01:47:35.795 INFO:tasks.rados.rados.0.smithi165.stdout:65: copy_from oid 35 from oid 39 current snap is 1 2024-04-20T01:47:35.795 INFO:tasks.rados.rados.0.smithi165.stdout:66: read oid 31 snap 0 2024-04-20T01:47:35.795 INFO:tasks.rados.rados.0.smithi165.stdout:66: expect (ObjNum 30 snap 0 seq_num 30) 2024-04-20T01:47:35.795 INFO:tasks.rados.rados.0.smithi165.stdout:67: snap_create 2024-04-20T01:47:35.798 INFO:tasks.rados.rados.0.smithi165.stdout:66: expect (ObjNum 30 snap 0 seq_num 30) 2024-04-20T01:47:35.799 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:35.814+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:35.812 INFO:tasks.rados.rados.0.smithi165.stdout:65: finishing copy_from racing read to smithi16579500-35 2024-04-20T01:47:35.812 INFO:tasks.rados.rados.0.smithi165.stdout:65: finishing copy_from to smithi16579500-35 2024-04-20T01:47:35.812 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 35 v 10 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-04-20T01:47:35.859 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:35.859 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:35.870 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-20T01:47:35.900 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:35.914+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:36.001 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:36.015+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:36.039 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:36.039 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:36.050 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-20T01:47:36.102 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:36.117+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:47:36.177 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:36.178 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:36.189 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-20T01:47:36.203 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:36.217+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:36.304 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:47:36.318+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:47:36.316 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:36.316 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:36.326 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-20T01:47:36.404 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:47:36.419+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:47:36.455 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:36.455 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:36.465 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-20T01:47:36.506 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:47:36.520+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:47:36.606 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:36.620+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:36.634 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:36.634 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:36.644 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-20T01:47:36.707 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:36.721+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:36.749 INFO:tasks.rados.rados.0.smithi165.stdout:65: done (2 left) 2024-04-20T01:47:36.749 INFO:tasks.rados.rados.0.smithi165.stdout:66: done (1 left) 2024-04-20T01:47:36.749 INFO:tasks.rados.rados.0.smithi165.stdout:67: done (0 left) 2024-04-20T01:47:36.750 INFO:tasks.rados.rados.0.smithi165.stdout:68: read oid 40 snap 1 2024-04-20T01:47:36.750 INFO:tasks.rados.rados.0.smithi165.stdout:68: expect (ObjNum 39 snap 0 seq_num 39) 2024-04-20T01:47:36.750 INFO:tasks.rados.rados.0.smithi165.stdout:69: read oid 42 snap -1 2024-04-20T01:47:36.750 INFO:tasks.rados.rados.0.smithi165.stdout:69: expect (ObjNum 41 snap 0 seq_num 41) 2024-04-20T01:47:36.750 INFO:tasks.rados.rados.0.smithi165.stdout:70: delete oid 43 current snap is 2 2024-04-20T01:47:36.752 INFO:tasks.rados.rados.0.smithi165.stdout:70: done (2 left) 2024-04-20T01:47:36.752 INFO:tasks.rados.rados.0.smithi165.stdout:71: delete oid 27 current snap is 2 2024-04-20T01:47:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.2 to 1 (10000) 2024-04-20T01:47:36.757 INFO:tasks.rados.rados.0.smithi165.stdout:68: expect (ObjNum 39 snap 0 seq_num 39) 2024-04-20T01:47:36.769 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-20T01:47:36.774 INFO:tasks.rados.rados.0.smithi165.stdout:69: expect (ObjNum 41 snap 0 seq_num 41) 2024-04-20T01:47:36.792 INFO:tasks.rados.rados.0.smithi165.stdout:68: done (2 left) 2024-04-20T01:47:36.792 INFO:tasks.rados.rados.0.smithi165.stdout:69: done (1 left) 2024-04-20T01:47:36.792 INFO:tasks.rados.rados.0.smithi165.stdout:71: done (0 left) 2024-04-20T01:47:36.793 INFO:tasks.rados.rados.0.smithi165.stdout:72: read oid 21 snap 0 2024-04-20T01:47:36.793 INFO:tasks.rados.rados.0.smithi165.stdout:72: expect (ObjNum 20 snap 0 seq_num 20) 2024-04-20T01:47:36.793 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 19 current snap is 2 2024-04-20T01:47:36.793 INFO:tasks.rados.rados.0.smithi165.stdout:73: seq_num 56 ranges {311296=573440,884736=376832} 2024-04-20T01:47:36.797 INFO:tasks.rados.rados.0.smithi165.stdout:73: writing smithi16579500-19 from 311296 to 884736 tid 1 2024-04-20T01:47:36.799 INFO:tasks.rados.rados.0.smithi165.stdout:73: writing smithi16579500-19 from 884736 to 1261568 tid 2 2024-04-20T01:47:36.799 INFO:tasks.rados.rados.0.smithi165.stdout:74: snap_remove snap 1 2024-04-20T01:47:36.807 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:36.822+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:47:36.820 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:36.820 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:36.831 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-20T01:47:36.908 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:36.923+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:47:37.002 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:37.002 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:37.009 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:37.023+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:37.014 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-20T01:47:37.110 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:37.125+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:47:37.157 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:37.157 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:37.169 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-20T01:47:37.212 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:37.226+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:37.312 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:37.312 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:37.313 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:37.327+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:47:37.324 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-20T01:47:37.347 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-20T01:47:37.413 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:37.428+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:47:37.471 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:37.472 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:37.484 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-20T01:47:37.514 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:37.528+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:37.534 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:47:37.555 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:47:37.577 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:47:37.598 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:47:37.615 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:37.629+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:47:37.619 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:47:37.629 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:37.629 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:47:37.630 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-04-20T01:47:37.630 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-04-20T01:47:37.630 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:37.640 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:47:37.641 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-20T01:47:37.660 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:47:37.683 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:47:37.704 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:47:37.716 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:37.730+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:47:37.726 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:47:37.746 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:47:37.753 INFO:tasks.rados.rados.0.smithi165.stdout:74: done (2 left) 2024-04-20T01:47:37.753 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 44 current snap is 2 2024-04-20T01:47:37.753 INFO:tasks.rados.rados.0.smithi165.stdout:75: seq_num 57 ranges {737280=589824,1327104=376832} 2024-04-20T01:47:37.758 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.3 to 1 (10000) 2024-04-20T01:47:37.760 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-20T01:47:37.761 INFO:tasks.rados.rados.0.smithi165.stdout:75: writing smithi16579500-44 from 737280 to 1327104 tid 1 2024-04-20T01:47:37.763 INFO:tasks.rados.rados.0.smithi165.stdout:75: writing smithi16579500-44 from 1327104 to 1703936 tid 2 2024-04-20T01:47:37.763 INFO:tasks.rados.rados.0.smithi165.stdout:76: delete oid 13 current snap is 2 2024-04-20T01:47:37.764 INFO:tasks.rados.rados.0.smithi165.stdout:72: expect (ObjNum 20 snap 0 seq_num 20) 2024-04-20T01:47:37.772 INFO:tasks.rados.rados.0.smithi165.stdout:73: finishing write tid 1 to smithi16579500-19 2024-04-20T01:47:37.772 INFO:tasks.rados.rados.0.smithi165.stdout:73: finishing write tid 2 to smithi16579500-19 2024-04-20T01:47:37.772 INFO:tasks.rados.rados.0.smithi165.stdout:73: finishing write tid 3 to smithi16579500-19 2024-04-20T01:47:37.772 INFO:tasks.rados.rados.0.smithi165.stdout:73: finishing write tid 4 to smithi16579500-19 2024-04-20T01:47:37.772 INFO:tasks.rados.rados.0.smithi165.stdout:73: oid 19 updating version 0 to 19 2024-04-20T01:47:37.772 INFO:tasks.rados.rados.0.smithi165.stdout:73: oid 19 updating version 19 to 20 2024-04-20T01:47:37.772 INFO:tasks.rados.rados.0.smithi165.stdout:73: oid 19 version 20 is already newer than 18 2024-04-20T01:47:37.773 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 19 v 20 (ObjNum 56 snap 2 seq_num 56) dirty exists 2024-04-20T01:47:37.773 INFO:tasks.rados.rados.0.smithi165.stdout:73: left oid 19 (ObjNum 56 snap 2 seq_num 56) 2024-04-20T01:47:37.773 INFO:tasks.rados.rados.0.smithi165.stdout:75: finishing write tid 1 to smithi16579500-44 2024-04-20T01:47:37.777 INFO:tasks.rados.rados.0.smithi165.stdout:75: finishing write tid 2 to smithi16579500-44 2024-04-20T01:47:37.777 INFO:tasks.rados.rados.0.smithi165.stdout:75: finishing write tid 3 to smithi16579500-44 2024-04-20T01:47:37.781 INFO:tasks.rados.rados.0.smithi165.stdout:75: finishing write tid 4 to smithi16579500-44 2024-04-20T01:47:37.781 INFO:tasks.rados.rados.0.smithi165.stdout:75: oid 44 updating version 0 to 7 2024-04-20T01:47:37.781 INFO:tasks.rados.rados.0.smithi165.stdout:75: oid 44 updating version 7 to 9 2024-04-20T01:47:37.781 INFO:tasks.rados.rados.0.smithi165.stdout:75: oid 44 version 9 is already newer than 8 2024-04-20T01:47:37.781 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 44 v 9 (ObjNum 57 snap 2 seq_num 57) dirty exists 2024-04-20T01:47:37.781 INFO:tasks.rados.rados.0.smithi165.stdout:75: left oid 44 (ObjNum 57 snap 2 seq_num 57) 2024-04-20T01:47:37.782 INFO:tasks.rados.rados.0.smithi165.stdout:72: done (3 left) 2024-04-20T01:47:37.782 INFO:tasks.rados.rados.0.smithi165.stdout:73: done (2 left) 2024-04-20T01:47:37.782 INFO:tasks.rados.rados.0.smithi165.stdout:75: done (1 left) 2024-04-20T01:47:37.782 INFO:tasks.rados.rados.0.smithi165.stdout:76: done (0 left) 2024-04-20T01:47:37.782 INFO:tasks.rados.rados.0.smithi165.stdout:77: copy_from oid 39 from oid 3 current snap is 2 2024-04-20T01:47:37.782 INFO:tasks.rados.rados.0.smithi165.stdout:78: setattr oid 41 current snap is 2 2024-04-20T01:47:37.783 INFO:tasks.rados.rados.0.smithi165.stdout:79: rollback oid 50 current snap is 2 2024-04-20T01:47:37.783 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 50 to 0 2024-04-20T01:47:37.783 INFO:tasks.rados.rados.0.smithi165.stdout:80: read oid 2 snap -1 2024-04-20T01:47:37.784 INFO:tasks.rados.rados.0.smithi165.stdout:80: expect (ObjNum 1 snap 0 seq_num 1) 2024-04-20T01:47:37.784 INFO:tasks.rados.rados.0.smithi165.stdout:81: snap_create 2024-04-20T01:47:37.784 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-20T01:47:37.785 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:47:37.787 INFO:tasks.rados.rados.0.smithi165.stdout:79: finishing rollback tid 0 to smithi16579500-50 2024-04-20T01:47:37.789 INFO:tasks.rados.rados.0.smithi165.stdout:80: expect (ObjNum 1 snap 0 seq_num 1) 2024-04-20T01:47:37.790 INFO:tasks.rados.rados.0.smithi165.stdout:79: finishing rollback tid 1 to smithi16579500-50 2024-04-20T01:47:37.791 INFO:tasks.rados.rados.0.smithi165.stdout:79: finishing rollback tid 2 to smithi16579500-50 2024-04-20T01:47:37.791 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 50 v 15 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-04-20T01:47:37.792 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 41 v 26 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-04-20T01:47:37.794 INFO:tasks.rados.rados.0.smithi165.stdout:77: finishing copy_from racing read to smithi16579500-39 2024-04-20T01:47:37.802 INFO:tasks.rados.rados.0.smithi165.stdout:77: finishing copy_from to smithi16579500-39 2024-04-20T01:47:37.803 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 39 v 4 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-04-20T01:47:37.807 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:37.807 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-04-20T01:47:37.807 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-04-20T01:47:37.808 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [] 2024-04-20T01:47:37.808 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:37.812 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:47:37.817 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:37.831+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:37.822 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-20T01:47:37.834 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:47:37.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:47:37.849+0000 7f862dd96700 0 -- 172.21.15.44:0/4029027699 >> [v2:172.21.15.123:6824/43110,v1:172.21.15.123:6825/43110] conn(0x7f862821cfe0 msgr2=0x7f862821f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-20T01:47:37.918 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:37.932+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:47:37.992 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:37.992 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:47:37.992 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-04-20T01:47:37.993 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:38.005 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-20T01:47:38.049 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:47:38.063+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:47:38.057 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:47:38.079 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:47:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:47:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:47:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:47:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:47:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:47:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:47:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:47:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:47:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:47:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:47:38.081 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:47:38.082 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:47:38.082 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:47:38.082 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:47:38.082 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:47:38.082 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:47:38.120 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:38.134+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:47:38.157 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:38.157 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:47:38.157 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-04-20T01:47:38.157 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-04-20T01:47:38.157 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:38.169 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-20T01:47:38.221 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:38.235+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:38.314 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-04-20T01:47:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-04-20T01:47:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-04-20T01:47:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.7s4 36/30 ECSubWrite(tid=57, reqid=client.4643.0:210, at_version=36'7, trim_to=0'0, roll_forward_to=33'5))", 2024-04-20T01:47:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.778787+0000", 2024-04-20T01:47:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.54882028000000005, 2024-04-20T01:47:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0078786210000000006, 2024-04-20T01:47:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778787+0000", 2024-04-20T01:47:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778787+0000", 2024-04-20T01:47:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778787+0000", 2024-04-20T01:47:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-04-20T01:47:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778852+0000", 2024-04-20T01:47:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4950999999999995e-05 2024-04-20T01:47:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778875+0000", 2024-04-20T01:47:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.247e-05 2024-04-20T01:47:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778886+0000", 2024-04-20T01:47:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1756999999999999e-05 2024-04-20T01:47:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.779834+0000", 2024-04-20T01:47:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00094809500000000004 2024-04-20T01:47:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.779847+0000", 2024-04-20T01:47:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.24e-05 2024-04-20T01:47:38.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.786624+0000", 2024-04-20T01:47:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0067774719999999997 2024-04-20T01:47:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.786666+0000", 2024-04-20T01:47:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1705000000000001e-05 2024-04-20T01:47:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.319 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.7s4 36/30 ECSubWrite(tid=58, reqid=client.4643.0:211, at_version=36'8, trim_to=0'0, roll_forward_to=33'5))", 2024-04-20T01:47:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.782105+0000", 2024-04-20T01:47:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.54550222800000003, 2024-04-20T01:47:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096109860000000002, 2024-04-20T01:47:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782105+0000", 2024-04-20T01:47:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782105+0000", 2024-04-20T01:47:38.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782104+0000", 2024-04-20T01:47:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782192+0000", 2024-04-20T01:47:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7806000000000006e-05 2024-04-20T01:47:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782206+0000", 2024-04-20T01:47:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3767e-05 2024-04-20T01:47:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782217+0000", 2024-04-20T01:47:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0808999999999999e-05 2024-04-20T01:47:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782307+0000", 2024-04-20T01:47:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0904999999999998e-05 2024-04-20T01:47:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782319+0000", 2024-04-20T01:47:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1932e-05 2024-04-20T01:47:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.791680+0000", 2024-04-20T01:47:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.009360808 2024-04-20T01:47:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.791716+0000", 2024-04-20T01:47:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6186000000000002e-05 2024-04-20T01:47:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.7s4 36/30 ECSubWrite(tid=60, reqid=client.4643.0:212, at_version=36'9, trim_to=0'0, roll_forward_to=33'5))", 2024-04-20T01:47:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.785701+0000", 2024-04-20T01:47:38.324 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.54190691099999999, 2024-04-20T01:47:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0060346890000000002, 2024-04-20T01:47:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.785701+0000", 2024-04-20T01:47:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.785701+0000", 2024-04-20T01:47:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.785693+0000", 2024-04-20T01:47:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999919 2024-04-20T01:47:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.785739+0000", 2024-04-20T01:47:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6122000000000003e-05 2024-04-20T01:47:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.785765+0000", 2024-04-20T01:47:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5812999999999998e-05 2024-04-20T01:47:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.785777+0000", 2024-04-20T01:47:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2619999999999999e-05 2024-04-20T01:47:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.785867+0000", 2024-04-20T01:47:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9940999999999998e-05 2024-04-20T01:47:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.785880+0000", 2024-04-20T01:47:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2301e-05 2024-04-20T01:47:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.791723+0000", 2024-04-20T01:47:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0058432450000000004 2024-04-20T01:47:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.791735+0000", 2024-04-20T01:47:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2588999999999999e-05 2024-04-20T01:47:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 36/30 ECSubWriteReply(tid=32, last_complete=36'22, committed=1, applied=1))", 2024-04-20T01:47:38.328 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.789811+0000", 2024-04-20T01:47:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.53779618699999998, 2024-04-20T01:47:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0013553370000000001, 2024-04-20T01:47:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.789811+0000", 2024-04-20T01:47:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.789811+0000", 2024-04-20T01:47:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.789811+0000", 2024-04-20T01:47:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.789821+0000", 2024-04-20T01:47:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0205e-05 2024-04-20T01:47:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.789833+0000", 2024-04-20T01:47:38.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2427e-05 2024-04-20T01:47:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.789843+0000", 2024-04-20T01:47:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0025e-05 2024-04-20T01:47:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.791152+0000", 2024-04-20T01:47:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0013088279999999999 2024-04-20T01:47:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.791167+0000", 2024-04-20T01:47:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4785e-05 2024-04-20T01:47:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.7s4 36/30 ECSubWrite(tid=61, reqid=client.4643.0:212, at_version=0'0, trim_to=0'0, roll_forward_to=36'9))", 2024-04-20T01:47:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.793075+0000", 2024-04-20T01:47:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.53453308499999996, 2024-04-20T01:47:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0014375740000000001, 2024-04-20T01:47:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.793075+0000", 2024-04-20T01:47:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.793075+0000", 2024-04-20T01:47:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.793074+0000", 2024-04-20T01:47:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.793095+0000", 2024-04-20T01:47:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1033999999999999e-05 2024-04-20T01:47:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.793111+0000", 2024-04-20T01:47:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6259000000000001e-05 2024-04-20T01:47:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.793121+0000", 2024-04-20T01:47:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7329999999999998e-06 2024-04-20T01:47:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.793199+0000", 2024-04-20T01:47:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8393999999999995e-05 2024-04-20T01:47:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.793210+0000", 2024-04-20T01:47:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0981999999999999e-05 2024-04-20T01:47:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.794489+0000", 2024-04-20T01:47:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0012783250000000001 2024-04-20T01:47:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.794512+0000", 2024-04-20T01:47:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3507999999999999e-05 2024-04-20T01:47:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:215 3.4s0 3:28069c2b:::smithi16579500-39:head [copy-from ver 4] snapc 3=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:47:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.797216+0000", 2024-04-20T01:47:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.53039163300000003, 2024-04-20T01:47:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.020029964000000001, 2024-04-20T01:47:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:38.336 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 215 2024-04-20T01:47:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797216+0000", 2024-04-20T01:47:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797216+0000", 2024-04-20T01:47:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.337 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797214+0000", 2024-04-20T01:47:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999976 2024-04-20T01:47:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797223+0000", 2024-04-20T01:47:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9709999999999993e-06 2024-04-20T01:47:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797240+0000", 2024-04-20T01:47:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6872e-05 2024-04-20T01:47:38.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797249+0000", 2024-04-20T01:47:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1500000000000005e-06 2024-04-20T01:47:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797298+0000", 2024-04-20T01:47:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9715000000000001e-05 2024-04-20T01:47:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797408+0000", 2024-04-20T01:47:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000109883 2024-04-20T01:47:38.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808715+0000", 2024-04-20T01:47:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011306145 2024-04-20T01:47:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.812376+0000", 2024-04-20T01:47:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0036618029999999999 2024-04-20T01:47:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:47:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.817198+0000", 2024-04-20T01:47:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048218749999999998 2024-04-20T01:47:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.817246+0000", 2024-04-20T01:47:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7787000000000002e-05 2024-04-20T01:47:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:216 3.4s0 3:28069c2b:::smithi16579500-39:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:47:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.797261+0000", 2024-04-20T01:47:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.53034690500000004, 2024-04-20T01:47:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012431373000000001, 2024-04-20T01:47:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:47:38.341 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 216 2024-04-20T01:47:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797261+0000", 2024-04-20T01:47:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797261+0000", 2024-04-20T01:47:38.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797260+0000", 2024-04-20T01:47:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797266+0000", 2024-04-20T01:47:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2140000000000001e-06 2024-04-20T01:47:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797275+0000", 2024-04-20T01:47:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9309999999999997e-06 2024-04-20T01:47:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797282+0000", 2024-04-20T01:47:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5169999999999996e-06 2024-04-20T01:47:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797635+0000", 2024-04-20T01:47:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035326500000000001 2024-04-20T01:47:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for blocked object", 2024-04-20T01:47:38.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797689+0000", 2024-04-20T01:47:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3687999999999998e-05 2024-04-20T01:47:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.809488+0000", 2024-04-20T01:47:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011799417 2024-04-20T01:47:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.809591+0000", 2024-04-20T01:47:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102729 2024-04-20T01:47:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.809692+0000", 2024-04-20T01:47:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000101202 2024-04-20T01:47:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:217 3.4s0 3:27f742c3:::smithi16579500-41:head [create,setxattr STaaXsn_tt`[noPeao^kO]r]qhc]OnRdXWqXa]RQ (421) in=40b,setxattr QccpemYdNR_fUkaT_Zk (1613) in=19b,setxattr e]rWtr\\spa\\[`\\etXP]UVYl (1613) in=23b,setxattr ctjQdgZSs[oQ^YekRX]`P_kg`lYfeo\\]jOW\\Ncgj (105) in=40b,setxattr ]UUsac^QNTgnq\\c_amO]eQ^UmNmdPVNbpsche]nq (1366) in=40b,setxattr tsV]fVpQmWmtatroVb]qjW\\n\\^hTba^e (1613) in=32b,setxattr plUORnNkpnPqade]gn^TNqootWi^\\Ygrc[O[dZfX (105) in=40b,setxattr ]aq]PelqnUhOnhUcccRY_o`UqoXf`_^fRi`adYdm (630) in=40b,setxattr lobeSZbbYgcdsPag__QR]`_ih[ (1613) in=26b,setxattr cmbU]Vpc[onnejbfSlj]qNT]`Yspgre^bQ[go^so (138) in=40b,setxattr pkmgTn^hlhmYnjkbSslNYhkOctl_ctnq]^Wm (1613) in=36b,setxattr jrdhlkfStVpRdk]N[rjbfWgQ_RYkY^ZYdg]\\l[dV (1367) in=40b,setxattr `jS`f[NpneY_j`inTVdpWq`ntO_diaj[folX`ibo (1473) in=40b,setxattr aj_mddZOpYlRadmQjbSnSRVdY\\gTgZ]RNfWSSbbo (1858) in=40b,setxattr YegfaljqWjdPp\\]^ejiXpWObnZd]dQTlihpU_Q`q (1649) in=40b,setxattr UsbSqlQaZndaoThjTR\\i_ei]Zs[hUpZasNRfZkaO (1611) in=40b,setxattr oZqjdnYqTXfkcaTmagUaNpZTgXfVkphqmiUTlmOf (1612) in=40b,setxattr [PdlRk]`\\iiigl[fRo`obkeWlZ]W`eqS[nmRn[kg (841) in=40b,setxattr eQXfldiUZkeo^Sjbkb_shi[kNcVQe^W[loVl[g^d (1753) in=40b,setxattr aYNbp]cNSQRqR\\PWsdUPp[Qp]`Zl\\fdnSPeUOdn\\ (13964) in=40b] snapc 3=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:47:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.798958+0000", 2024-04-20T01:47:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.52864957099999998, 2024-04-20T01:47:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0078906770000000005, 2024-04-20T01:47:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:38.346 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 217 2024-04-20T01:47:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.798958+0000", 2024-04-20T01:47:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.798958+0000", 2024-04-20T01:47:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.347 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.798956+0000", 2024-04-20T01:47:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:47:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799150+0000", 2024-04-20T01:47:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000193735 2024-04-20T01:47:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799162+0000", 2024-04-20T01:47:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1705999999999999e-05 2024-04-20T01:47:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799170+0000", 2024-04-20T01:47:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1480000000000006e-06 2024-04-20T01:47:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799329+0000", 2024-04-20T01:47:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000158622 2024-04-20T01:47:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799445+0000", 2024-04-20T01:47:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116106 2024-04-20T01:47:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.800456+0000", 2024-04-20T01:47:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0010113190000000001 2024-04-20T01:47:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.804628+0000", 2024-04-20T01:47:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0041720530000000002 2024-04-20T01:47:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:47:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.806746+0000", 2024-04-20T01:47:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021178870000000002 2024-04-20T01:47:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.806849+0000", 2024-04-20T01:47:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102847 2024-04-20T01:47:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs1 36/30 ECSubWrite(tid=19, reqid=client.4643.0:218, at_version=36'13, trim_to=0'0, roll_forward_to=34'11))", 2024-04-20T01:47:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.799051+0000", 2024-04-20T01:47:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.52855618299999996, 2024-04-20T01:47:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002478128, 2024-04-20T01:47:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.351 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799051+0000", 2024-04-20T01:47:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799051+0000", 2024-04-20T01:47:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799051+0000", 2024-04-20T01:47:38.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799076+0000", 2024-04-20T01:47:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4352e-05 2024-04-20T01:47:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799088+0000", 2024-04-20T01:47:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.219e-05 2024-04-20T01:47:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799096+0000", 2024-04-20T01:47:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4549999999999995e-06 2024-04-20T01:47:38.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799153+0000", 2024-04-20T01:47:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6444999999999999e-05 2024-04-20T01:47:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799163+0000", 2024-04-20T01:47:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0645000000000001e-05 2024-04-20T01:47:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.801507+0000", 2024-04-20T01:47:38.354 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023441260000000002 2024-04-20T01:47:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.801530+0000", 2024-04-20T01:47:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2187e-05 2024-04-20T01:47:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:221 3.4s0 3:22cffa5d:::smithi16579500-2:head [read 0~32768,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:47:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.799176+0000", 2024-04-20T01:47:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.52843202199999995, 2024-04-20T01:47:38.355 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045659560000000004, 2024-04-20T01:47:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:47:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 221 2024-04-20T01:47:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799176+0000", 2024-04-20T01:47:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.356 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799176+0000", 2024-04-20T01:47:38.357 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.357 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.357 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.357 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799175+0000", 2024-04-20T01:47:38.357 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:38.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.357 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.357 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.357 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799179+0000", 2024-04-20T01:47:38.357 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0579999999999999e-06 2024-04-20T01:47:38.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.357 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.357 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.358 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799187+0000", 2024-04-20T01:47:38.358 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1140000000000006e-06 2024-04-20T01:47:38.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.358 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.358 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.358 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799193+0000", 2024-04-20T01:47:38.358 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8830000000000003e-06 2024-04-20T01:47:38.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.358 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.358 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.358 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.801172+0000", 2024-04-20T01:47:38.358 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019787419999999999 2024-04-20T01:47:38.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.358 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.359 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:38.359 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.801248+0000", 2024-04-20T01:47:38.359 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6237999999999998e-05 2024-04-20T01:47:38.359 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.359 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.359 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.359 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.803742+0000", 2024-04-20T01:47:38.359 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00249343 2024-04-20T01:47:38.359 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.359 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.359 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.359 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.359 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.360 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:222 3.4s0 3:22cffa5d:::smithi16579500-2:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:47:38.360 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.799198+0000", 2024-04-20T01:47:38.360 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.52840985299999998, 2024-04-20T01:47:38.360 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0049783730000000003, 2024-04-20T01:47:38.360 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.360 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:47:38.360 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:38.360 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:38.360 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:38.360 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 222 2024-04-20T01:47:38.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.360 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.360 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.360 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.360 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799198+0000", 2024-04-20T01:47:38.361 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.361 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.361 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799198+0000", 2024-04-20T01:47:38.361 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.361 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.361 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799197+0000", 2024-04-20T01:47:38.361 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:38.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.361 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.362 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799200+0000", 2024-04-20T01:47:38.362 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1159999999999999e-06 2024-04-20T01:47:38.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.362 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.362 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799208+0000", 2024-04-20T01:47:38.362 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4379999999999998e-06 2024-04-20T01:47:38.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.362 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.362 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799213+0000", 2024-04-20T01:47:38.362 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1449999999999999e-06 2024-04-20T01:47:38.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.362 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.363 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.801564+0000", 2024-04-20T01:47:38.363 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023512609999999999 2024-04-20T01:47:38.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.363 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:38.363 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.801655+0000", 2024-04-20T01:47:38.363 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1015999999999997e-05 2024-04-20T01:47:38.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.363 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.363 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.804176+0000", 2024-04-20T01:47:38.363 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025208330000000001 2024-04-20T01:47:38.363 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.363 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.364 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.364 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:223 3.4s0 3:22cffa5d:::smithi16579500-2:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:47:38.364 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.799217+0000", 2024-04-20T01:47:38.364 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.52839026499999997, 2024-04-20T01:47:38.364 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.005189679, 2024-04-20T01:47:38.364 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.364 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:47:38.364 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:38.364 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:38.364 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:38.364 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 223 2024-04-20T01:47:38.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.364 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.365 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.365 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.365 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799217+0000", 2024-04-20T01:47:38.365 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.365 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.365 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.365 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799217+0000", 2024-04-20T01:47:38.365 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.365 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.365 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.365 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799217+0000", 2024-04-20T01:47:38.365 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:38.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.366 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.366 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799220+0000", 2024-04-20T01:47:38.366 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.213e-06 2024-04-20T01:47:38.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.366 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.366 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799226+0000", 2024-04-20T01:47:38.366 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4029999999999998e-06 2024-04-20T01:47:38.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.366 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.366 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799230+0000", 2024-04-20T01:47:38.366 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2180000000000001e-06 2024-04-20T01:47:38.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.367 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.367 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.801991+0000", 2024-04-20T01:47:38.367 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027615579999999999 2024-04-20T01:47:38.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.367 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:38.367 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802053+0000", 2024-04-20T01:47:38.367 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1581000000000006e-05 2024-04-20T01:47:38.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.367 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.367 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.804407+0000", 2024-04-20T01:47:38.367 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002354119 2024-04-20T01:47:38.368 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.368 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.368 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.368 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs1 36/30 ECSubWrite(tid=20, reqid=client.4643.0:219, at_version=36'14, trim_to=0'0, roll_forward_to=34'11))", 2024-04-20T01:47:38.368 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.802063+0000", 2024-04-20T01:47:38.368 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.52554440899999999, 2024-04-20T01:47:38.368 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025932809999999998, 2024-04-20T01:47:38.368 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.368 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.368 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.369 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.369 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802063+0000", 2024-04-20T01:47:38.369 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.369 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.369 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802063+0000", 2024-04-20T01:47:38.369 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.369 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.369 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802054+0000", 2024-04-20T01:47:38.369 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999905 2024-04-20T01:47:38.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.370 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.370 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802090+0000", 2024-04-20T01:47:38.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6168000000000003e-05 2024-04-20T01:47:38.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.370 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.370 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802099+0000", 2024-04-20T01:47:38.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1710000000000008e-06 2024-04-20T01:47:38.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.370 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.370 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802106+0000", 2024-04-20T01:47:38.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6370000000000002e-06 2024-04-20T01:47:38.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.371 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.371 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802638+0000", 2024-04-20T01:47:38.371 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053191999999999996 2024-04-20T01:47:38.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.371 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.371 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802642+0000", 2024-04-20T01:47:38.371 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8579999999999999e-06 2024-04-20T01:47:38.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.371 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.371 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.804630+0000", 2024-04-20T01:47:38.371 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019875510000000002 2024-04-20T01:47:38.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.372 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.372 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.804657+0000", 2024-04-20T01:47:38.372 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6523e-05 2024-04-20T01:47:38.372 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.372 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.372 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.372 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs1 36/30 ECSubWrite(tid=21, reqid=client.4643.0:220, at_version=36'15, trim_to=0'0, roll_forward_to=34'11))", 2024-04-20T01:47:38.372 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.802113+0000", 2024-04-20T01:47:38.372 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.52549438800000003, 2024-04-20T01:47:38.372 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00255755, 2024-04-20T01:47:38.373 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.373 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.373 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.373 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.373 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802113+0000", 2024-04-20T01:47:38.373 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.373 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.373 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802113+0000", 2024-04-20T01:47:38.373 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.374 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.374 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802113+0000", 2024-04-20T01:47:38.374 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:38.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.374 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.374 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802130+0000", 2024-04-20T01:47:38.374 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6761000000000001e-05 2024-04-20T01:47:38.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.374 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.374 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802135+0000", 2024-04-20T01:47:38.374 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8730000000000004e-06 2024-04-20T01:47:38.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.375 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.375 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802140+0000", 2024-04-20T01:47:38.375 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.028e-06 2024-04-20T01:47:38.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.375 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.375 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802998+0000", 2024-04-20T01:47:38.375 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00085757800000000001 2024-04-20T01:47:38.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.375 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.375 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.803007+0000", 2024-04-20T01:47:38.375 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8650000000000007e-06 2024-04-20T01:47:38.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.375 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.376 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.804659+0000", 2024-04-20T01:47:38.376 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0016524420000000001 2024-04-20T01:47:38.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.376 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.376 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.804671+0000", 2024-04-20T01:47:38.376 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.145e-05 2024-04-20T01:47:38.376 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.376 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.376 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.376 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpReadReply(3.4s0 36/30 ECSubReadReply(tid=36, attrs_read=0))", 2024-04-20T01:47:38.376 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.802570+0000", 2024-04-20T01:47:38.376 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.52503792999999999, 2024-04-20T01:47:38.376 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001185623, 2024-04-20T01:47:38.376 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.377 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.377 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.377 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802570+0000", 2024-04-20T01:47:38.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.377 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802570+0000", 2024-04-20T01:47:38.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.377 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802570+0000", 2024-04-20T01:47:38.378 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.378 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.378 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802570+0000", 2024-04-20T01:47:38.378 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.378 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.378 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802592+0000", 2024-04-20T01:47:38.378 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1855999999999999e-05 2024-04-20T01:47:38.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.378 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.379 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802600+0000", 2024-04-20T01:47:38.379 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2970000000000004e-06 2024-04-20T01:47:38.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.379 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.379 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.379 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.803521+0000", 2024-04-20T01:47:38.379 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00092114699999999998 2024-04-20T01:47:38.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.379 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.379 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.379 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.803755+0000", 2024-04-20T01:47:38.379 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023432300000000001 2024-04-20T01:47:38.379 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.379 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.380 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.380 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.380 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.380 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpReadReply(3.4s0 36/30 ECSubReadReply(tid=37, attrs_read=0))", 2024-04-20T01:47:38.380 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.802999+0000", 2024-04-20T01:47:38.380 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.52460848500000001, 2024-04-20T01:47:38.380 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011940710000000001, 2024-04-20T01:47:38.380 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.380 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.380 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.380 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.380 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.380 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802999+0000", 2024-04-20T01:47:38.380 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.380 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.381 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.381 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802999+0000", 2024-04-20T01:47:38.381 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.381 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.381 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802999+0000", 2024-04-20T01:47:38.381 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.381 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.381 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.802999+0000", 2024-04-20T01:47:38.381 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.382 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.382 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.803012+0000", 2024-04-20T01:47:38.382 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2926e-05 2024-04-20T01:47:38.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.382 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.382 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.803017+0000", 2024-04-20T01:47:38.382 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4209999999999997e-06 2024-04-20T01:47:38.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.382 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.382 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.803974+0000", 2024-04-20T01:47:38.383 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00095719900000000005 2024-04-20T01:47:38.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.383 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.383 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.804193+0000", 2024-04-20T01:47:38.383 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00021952500000000001 2024-04-20T01:47:38.383 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.383 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.383 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.383 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs1 36/30 ECSubWrite(tid=22, reqid=client.4643.0:220, at_version=0'0, trim_to=0'0, roll_forward_to=36'15))", 2024-04-20T01:47:38.383 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.807135+0000", 2024-04-20T01:47:38.383 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.52047289699999999, 2024-04-20T01:47:38.384 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0036840359999999999, 2024-04-20T01:47:38.384 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.384 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.384 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.384 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.384 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.807135+0000", 2024-04-20T01:47:38.384 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.384 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.384 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.807135+0000", 2024-04-20T01:47:38.384 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.807134+0000", 2024-04-20T01:47:38.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:38.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.807146+0000", 2024-04-20T01:47:38.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2098e-05 2024-04-20T01:47:38.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.807157+0000", 2024-04-20T01:47:38.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0771000000000001e-05 2024-04-20T01:47:38.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.386 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.386 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.807180+0000", 2024-04-20T01:47:38.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3283999999999999e-05 2024-04-20T01:47:38.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.386 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.386 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.807346+0000", 2024-04-20T01:47:38.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000165439 2024-04-20T01:47:38.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.386 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.386 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.807356+0000", 2024-04-20T01:47:38.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9510000000000001e-06 2024-04-20T01:47:38.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.387 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.387 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.810794+0000", 2024-04-20T01:47:38.387 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034378500000000001 2024-04-20T01:47:38.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.387 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.387 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.810819+0000", 2024-04-20T01:47:38.387 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5174999999999999e-05 2024-04-20T01:47:38.387 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.387 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.387 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.387 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 36/30 ECSubWriteReply(tid=39, last_complete=36'26, committed=1, applied=1))", 2024-04-20T01:47:38.387 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.807862+0000", 2024-04-20T01:47:38.388 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.51974610499999996, 2024-04-20T01:47:38.388 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00188674, 2024-04-20T01:47:38.388 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.388 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.388 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.388 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.388 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.807862+0000", 2024-04-20T01:47:38.388 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.388 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.388 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.807862+0000", 2024-04-20T01:47:38.388 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.389 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.807861+0000", 2024-04-20T01:47:38.389 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:38.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.389 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.807870+0000", 2024-04-20T01:47:38.389 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3579999999999992e-06 2024-04-20T01:47:38.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.389 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.807883+0000", 2024-04-20T01:47:38.389 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3094e-05 2024-04-20T01:47:38.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.390 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.807891+0000", 2024-04-20T01:47:38.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3869999999999995e-06 2024-04-20T01:47:38.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.390 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.390 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.809720+0000", 2024-04-20T01:47:38.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0018284620000000001 2024-04-20T01:47:38.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.390 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.390 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.809748+0000", 2024-04-20T01:47:38.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8330000000000002e-05 2024-04-20T01:47:38.390 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.390 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.391 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.391 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 36/30 ECSubWriteReply(tid=39, last_complete=36'26, committed=1, applied=1))", 2024-04-20T01:47:38.391 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.808278+0000", 2024-04-20T01:47:38.391 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.51932979099999999, 2024-04-20T01:47:38.391 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001630238, 2024-04-20T01:47:38.391 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.391 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.391 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.391 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808278+0000", 2024-04-20T01:47:38.391 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.392 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.392 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808278+0000", 2024-04-20T01:47:38.392 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.392 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.392 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808277+0000", 2024-04-20T01:47:38.392 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:38.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.392 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.392 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808286+0000", 2024-04-20T01:47:38.392 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1759999999999999e-06 2024-04-20T01:47:38.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.393 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.393 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808300+0000", 2024-04-20T01:47:38.393 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3427999999999999e-05 2024-04-20T01:47:38.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.393 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.393 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808308+0000", 2024-04-20T01:47:38.393 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8740000000000001e-06 2024-04-20T01:47:38.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.393 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.393 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.809893+0000", 2024-04-20T01:47:38.393 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001584764 2024-04-20T01:47:38.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.394 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.394 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.394 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.809908+0000", 2024-04-20T01:47:38.394 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4907e-05 2024-04-20T01:47:38.394 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.394 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.394 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.394 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.394 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 36/30 ECSubWriteReply(tid=39, last_complete=36'26, committed=1, applied=1))", 2024-04-20T01:47:38.394 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.808551+0000", 2024-04-20T01:47:38.394 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.51905649899999995, 2024-04-20T01:47:38.394 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0012296060000000001, 2024-04-20T01:47:38.394 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.395 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.395 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.395 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.395 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808551+0000", 2024-04-20T01:47:38.395 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.395 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.395 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808551+0000", 2024-04-20T01:47:38.395 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.395 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.395 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808550+0000", 2024-04-20T01:47:38.396 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:38.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.396 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.396 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808556+0000", 2024-04-20T01:47:38.396 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5849999999999999e-06 2024-04-20T01:47:38.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.396 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.396 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808567+0000", 2024-04-20T01:47:38.396 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1573000000000001e-05 2024-04-20T01:47:38.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.396 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.396 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808576+0000", 2024-04-20T01:47:38.397 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1759999999999993e-06 2024-04-20T01:47:38.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.397 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.397 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.397 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.809771+0000", 2024-04-20T01:47:38.397 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001195439 2024-04-20T01:47:38.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.397 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.397 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.397 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.809781+0000", 2024-04-20T01:47:38.397 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6849999999999996e-06 2024-04-20T01:47:38.397 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.397 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.397 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.397 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.398 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.398 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:38.399 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-20T01:47:38.400 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:38.413+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:47:38.499 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:38.514+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:47:38.547 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:38.547 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:47:38.547 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-04-20T01:47:38.547 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:38.559 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-20T01:47:38.603 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:47:38.617+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:47:38.701 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:38.715+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:38.702 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:38.702 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:47:38.702 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-04-20T01:47:38.702 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-04-20T01:47:38.702 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:38.714 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-20T01:47:38.762 INFO:tasks.rados.rados.0.smithi165.stdout:77: done (4 left) 2024-04-20T01:47:38.762 INFO:tasks.rados.rados.0.smithi165.stdout:78: done (3 left) 2024-04-20T01:47:38.762 INFO:tasks.rados.rados.0.smithi165.stdout:79: done (2 left) 2024-04-20T01:47:38.762 INFO:tasks.rados.rados.0.smithi165.stdout:80: done (1 left) 2024-04-20T01:47:38.762 INFO:tasks.rados.rados.0.smithi165.stdout:81: done (0 left) 2024-04-20T01:47:38.763 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 45 current snap is 3 2024-04-20T01:47:38.763 INFO:tasks.rados.rados.0.smithi165.stdout:82: seq_num 59 ranges {753664=622592,1376256=376832} 2024-04-20T01:47:38.765 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is unset 2024-04-20T01:47:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.4 to 1 (10000) 2024-04-20T01:47:38.769 INFO:tasks.rados.rados.0.smithi165.stdout:82: writing smithi16579500-45 from 753664 to 1376256 tid 1 2024-04-20T01:47:38.772 INFO:tasks.rados.rados.0.smithi165.stdout:82: writing smithi16579500-45 from 1376256 to 1753088 tid 2 2024-04-20T01:47:38.772 INFO:tasks.rados.rados.0.smithi165.stdout:83: read oid 32 snap -1 2024-04-20T01:47:38.773 INFO:tasks.rados.rados.0.smithi165.stdout:83: expect (ObjNum 31 snap 0 seq_num 31) 2024-04-20T01:47:38.773 INFO:tasks.rados.rados.0.smithi165.stdout:84: copy_from oid 6 from oid 42 current snap is 3 2024-04-20T01:47:38.773 INFO:tasks.rados.rados.0.smithi165.stdout:85: rollback oid 46 current snap is 3 2024-04-20T01:47:38.773 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 46 to 2 2024-04-20T01:47:38.773 INFO:tasks.rados.rados.0.smithi165.stdout:86: read oid 34 snap -1 2024-04-20T01:47:38.773 INFO:tasks.rados.rados.0.smithi165.stdout:86: expect (ObjNum 33 snap 0 seq_num 33) 2024-04-20T01:47:38.773 INFO:tasks.rados.rados.0.smithi165.stdout:87: rollback oid 39 current snap is 3 2024-04-20T01:47:38.773 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 39 to 0 2024-04-20T01:47:38.773 INFO:tasks.rados.rados.0.smithi165.stdout:88: rmattr oid 48 current snap is 3 2024-04-20T01:47:38.773 INFO:tasks.rados.rados.0.smithi165.stdout:88: done (6 left) 2024-04-20T01:47:38.773 INFO:tasks.rados.rados.0.smithi165.stdout:89: rmattr oid 25 current snap is 3 2024-04-20T01:47:38.773 INFO:tasks.rados.rados.0.smithi165.stdout:89: done (6 left) 2024-04-20T01:47:38.773 INFO:tasks.rados.rados.0.smithi165.stdout:90: delete oid 7 current snap is 3 2024-04-20T01:47:38.783 INFO:tasks.rados.rados.0.smithi165.stdout:83: expect (ObjNum 31 snap 0 seq_num 31) 2024-04-20T01:47:38.788 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-20T01:47:38.796 INFO:tasks.rados.rados.0.smithi165.stdout:82: finishing write tid 1 to smithi16579500-45 2024-04-20T01:47:38.796 INFO:tasks.rados.rados.0.smithi165.stdout:84: finishing copy_from racing read to smithi16579500-6 2024-04-20T01:47:38.796 INFO:tasks.rados.rados.0.smithi165.stdout:82: finishing write tid 2 to smithi16579500-45 2024-04-20T01:47:38.796 INFO:tasks.rados.rados.0.smithi165.stdout:87: finishing rollback tid 0 to smithi16579500-39 2024-04-20T01:47:38.796 INFO:tasks.rados.rados.0.smithi165.stdout:87: finishing rollback tid 1 to smithi16579500-39 2024-04-20T01:47:38.796 INFO:tasks.rados.rados.0.smithi165.stdout:87: finishing rollback tid 2 to smithi16579500-39 2024-04-20T01:47:38.796 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 39 v 32 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-04-20T01:47:38.796 INFO:tasks.rados.rados.0.smithi165.stdout:86: expect (ObjNum 33 snap 0 seq_num 33) 2024-04-20T01:47:38.810 INFO:tasks.rados.rados.0.smithi165.stdout:82: finishing write tid 3 to smithi16579500-45 2024-04-20T01:47:38.810 INFO:tasks.rados.rados.0.smithi165.stdout:85: finishing rollback tid 0 to smithi16579500-46 2024-04-20T01:47:38.810 INFO:tasks.rados.rados.0.smithi165.stdout:82: finishing write tid 4 to smithi16579500-45 2024-04-20T01:47:38.810 INFO:tasks.rados.rados.0.smithi165.stdout:82: oid 45 updating version 0 to 17 2024-04-20T01:47:38.811 INFO:tasks.rados.rados.0.smithi165.stdout:82: oid 45 updating version 17 to 18 2024-04-20T01:47:38.811 INFO:tasks.rados.rados.0.smithi165.stdout:82: oid 45 version 18 is already newer than 16 2024-04-20T01:47:38.811 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 45 v 18 (ObjNum 59 snap 3 seq_num 59) dirty exists 2024-04-20T01:47:38.811 INFO:tasks.rados.rados.0.smithi165.stdout:82: left oid 45 (ObjNum 59 snap 3 seq_num 59) 2024-04-20T01:47:38.811 INFO:tasks.rados.rados.0.smithi165.stdout:85: finishing rollback tid 1 to smithi16579500-46 2024-04-20T01:47:38.811 INFO:tasks.rados.rados.0.smithi165.stdout:85: finishing rollback tid 2 to smithi16579500-46 2024-04-20T01:47:38.811 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 46 v 10 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-04-20T01:47:38.811 INFO:tasks.rados.rados.0.smithi165.stdout:84: finishing copy_from to smithi16579500-6 2024-04-20T01:47:38.811 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 6 v 6 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-04-20T01:47:38.811 INFO:tasks.rados.rados.0.smithi165.stdout:82: done (6 left) 2024-04-20T01:47:38.811 INFO:tasks.rados.rados.0.smithi165.stdout:83: done (5 left) 2024-04-20T01:47:38.811 INFO:tasks.rados.rados.0.smithi165.stdout:84: done (4 left) 2024-04-20T01:47:38.811 INFO:tasks.rados.rados.0.smithi165.stdout:85: done (3 left) 2024-04-20T01:47:38.811 INFO:tasks.rados.rados.0.smithi165.stdout:86: done (2 left) 2024-04-20T01:47:38.812 INFO:tasks.rados.rados.0.smithi165.stdout:87: done (1 left) 2024-04-20T01:47:38.812 INFO:tasks.rados.rados.0.smithi165.stdout:90: done (0 left) 2024-04-20T01:47:38.812 INFO:tasks.rados.rados.0.smithi165.stdout:91: snap_remove snap 2 2024-04-20T01:47:38.833 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:47:38.847+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:47:38.902 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:47:38.916+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:47:38.912 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:38.912 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-04-20T01:47:38.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-04-20T01:47:38.912 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-04-20T01:47:38.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.913 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.5s4 32/30 ECSubWrite(tid=0, reqid=client.4643.0:10, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:38.913 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.753709+0000", 2024-04-20T01:47:38.913 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.1720936799999997, 2024-04-20T01:47:38.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034126199999999999, 2024-04-20T01:47:38.913 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.913 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.913 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.753709+0000", 2024-04-20T01:47:38.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.753709+0000", 2024-04-20T01:47:38.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.753707+0000", 2024-04-20T01:47:38.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:38.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.753781+0000", 2024-04-20T01:47:38.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3194000000000004e-05 2024-04-20T01:47:38.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.753805+0000", 2024-04-20T01:47:38.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4584e-05 2024-04-20T01:47:38.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.753820+0000", 2024-04-20T01:47:38.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4525999999999999e-05 2024-04-20T01:47:38.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.753900+0000", 2024-04-20T01:47:38.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0116000000000003e-05 2024-04-20T01:47:38.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.753910+0000", 2024-04-20T01:47:38.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0258000000000001e-05 2024-04-20T01:47:38.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.757088+0000", 2024-04-20T01:47:38.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0031782619999999998 2024-04-20T01:47:38.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.757121+0000", 2024-04-20T01:47:38.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2997999999999998e-05 2024-04-20T01:47:38.917 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.917 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.917 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.917 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.5s4 32/30 ECSubWrite(tid=1, reqid=client.4643.0:11, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:38.917 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.754526+0000", 2024-04-20T01:47:38.917 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.17127596, 2024-04-20T01:47:38.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003640782, 2024-04-20T01:47:38.917 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.917 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.917 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.754526+0000", 2024-04-20T01:47:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.754526+0000", 2024-04-20T01:47:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.754525+0000", 2024-04-20T01:47:38.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:38.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.754547+0000", 2024-04-20T01:47:38.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1126000000000001e-05 2024-04-20T01:47:38.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.754560+0000", 2024-04-20T01:47:38.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3118999999999999e-05 2024-04-20T01:47:38.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.754569+0000", 2024-04-20T01:47:38.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5899999999999997e-06 2024-04-20T01:47:38.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.754620+0000", 2024-04-20T01:47:38.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1103999999999997e-05 2024-04-20T01:47:38.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.754631+0000", 2024-04-20T01:47:38.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0169999999999999e-05 2024-04-20T01:47:38.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.758140+0000", 2024-04-20T01:47:38.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0035096049999999998 2024-04-20T01:47:38.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.758167+0000", 2024-04-20T01:47:38.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7002999999999999e-05 2024-04-20T01:47:38.922 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.922 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.922 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.922 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.5s4 33/30 ECSubWrite(tid=2, reqid=client.4643.0:11, at_version=0'0, trim_to=0'0, roll_forward_to=32'2))", 2024-04-20T01:47:38.923 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.765877+0000", 2024-04-20T01:47:38.923 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.1599253389999999, 2024-04-20T01:47:38.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0075032049999999998, 2024-04-20T01:47:38.923 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.923 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.923 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765877+0000", 2024-04-20T01:47:38.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765877+0000", 2024-04-20T01:47:38.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765877+0000", 2024-04-20T01:47:38.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:38.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765904+0000", 2024-04-20T01:47:38.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7461e-05 2024-04-20T01:47:38.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765927+0000", 2024-04-20T01:47:38.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2697999999999999e-05 2024-04-20T01:47:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765939+0000", 2024-04-20T01:47:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2142e-05 2024-04-20T01:47:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.768205+0000", 2024-04-20T01:47:38.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022662239999999998 2024-04-20T01:47:38.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.768221+0000", 2024-04-20T01:47:38.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6161e-05 2024-04-20T01:47:38.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.773346+0000", 2024-04-20T01:47:38.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051250469999999998 2024-04-20T01:47:38.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.773380+0000", 2024-04-20T01:47:38.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3760999999999997e-05 2024-04-20T01:47:38.927 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.927 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.927 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.927 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/30 ECSubWrite(tid=1, reqid=client.4643.0:38, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:38.927 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.767259+0000", 2024-04-20T01:47:38.927 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.158543323, 2024-04-20T01:47:38.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0043950059999999999, 2024-04-20T01:47:38.927 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.927 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.927 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.767259+0000", 2024-04-20T01:47:38.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.767259+0000", 2024-04-20T01:47:38.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.767256+0000", 2024-04-20T01:47:38.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999971 2024-04-20T01:47:38.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.767281+0000", 2024-04-20T01:47:38.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5014000000000001e-05 2024-04-20T01:47:38.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.767298+0000", 2024-04-20T01:47:38.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7163000000000002e-05 2024-04-20T01:47:38.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.767309+0000", 2024-04-20T01:47:38.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0728e-05 2024-04-20T01:47:38.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.767404+0000", 2024-04-20T01:47:38.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5292999999999997e-05 2024-04-20T01:47:38.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.767415+0000", 2024-04-20T01:47:38.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0995999999999999e-05 2024-04-20T01:47:38.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.771619+0000", 2024-04-20T01:47:38.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042037719999999997 2024-04-20T01:47:38.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.771654+0000", 2024-04-20T01:47:38.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5064999999999997e-05 2024-04-20T01:47:38.932 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.932 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.932 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.933 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/30 ECSubWrite(tid=2, reqid=client.4643.0:38, at_version=0'0, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:47:38.933 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.777270+0000", 2024-04-20T01:47:38.933 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.148532436, 2024-04-20T01:47:38.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0057051339999999997, 2024-04-20T01:47:38.933 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.933 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.933 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.933 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.933 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777270+0000", 2024-04-20T01:47:38.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.934 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777270+0000", 2024-04-20T01:47:38.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.934 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777268+0000", 2024-04-20T01:47:38.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:47:38.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.934 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777287+0000", 2024-04-20T01:47:38.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9128999999999999e-05 2024-04-20T01:47:38.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777309+0000", 2024-04-20T01:47:38.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1475999999999999e-05 2024-04-20T01:47:38.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777323+0000", 2024-04-20T01:47:38.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4076e-05 2024-04-20T01:47:38.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777375+0000", 2024-04-20T01:47:38.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2685999999999997e-05 2024-04-20T01:47:38.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.936 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777389+0000", 2024-04-20T01:47:38.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3369e-05 2024-04-20T01:47:38.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.936 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.782943+0000", 2024-04-20T01:47:38.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055546110000000001 2024-04-20T01:47:38.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.936 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.782975+0000", 2024-04-20T01:47:38.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1569000000000002e-05 2024-04-20T01:47:38.937 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.937 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.937 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.937 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/30 ECSubWrite(tid=4, reqid=client.4643.0:58, at_version=33'3, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:47:38.937 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.780211+0000", 2024-04-20T01:47:38.937 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.1455917500000004, 2024-04-20T01:47:38.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0098815680000000003, 2024-04-20T01:47:38.937 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.937 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.937 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.937 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.937 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780211+0000", 2024-04-20T01:47:38.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.938 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.938 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780211+0000", 2024-04-20T01:47:38.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.938 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.938 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780209+0000", 2024-04-20T01:47:38.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:47:38.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.938 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.938 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780307+0000", 2024-04-20T01:47:38.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8575999999999999e-05 2024-04-20T01:47:38.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.939 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.939 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780328+0000", 2024-04-20T01:47:38.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0537999999999999e-05 2024-04-20T01:47:38.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.939 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.939 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780340+0000", 2024-04-20T01:47:38.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1997999999999999e-05 2024-04-20T01:47:38.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.939 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780423+0000", 2024-04-20T01:47:38.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2812999999999995e-05 2024-04-20T01:47:38.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780434+0000", 2024-04-20T01:47:38.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1028e-05 2024-04-20T01:47:38.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790048+0000", 2024-04-20T01:47:38.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096147339999999998 2024-04-20T01:47:38.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.941 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790092+0000", 2024-04-20T01:47:38.941 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3649999999999997e-05 2024-04-20T01:47:38.941 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.941 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.941 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.941 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/30 ECSubWrite(tid=5, reqid=client.4643.0:59, at_version=33'4, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:47:38.941 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.781201+0000", 2024-04-20T01:47:38.941 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.1446015120000004, 2024-04-20T01:47:38.941 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0089164740000000006, 2024-04-20T01:47:38.941 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.941 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.941 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.942 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.942 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781201+0000", 2024-04-20T01:47:38.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.942 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.942 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781201+0000", 2024-04-20T01:47:38.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.942 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.942 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781200+0000", 2024-04-20T01:47:38.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:38.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781219+0000", 2024-04-20T01:47:38.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8748000000000001e-05 2024-04-20T01:47:38.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781236+0000", 2024-04-20T01:47:38.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7251000000000001e-05 2024-04-20T01:47:38.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781246+0000", 2024-04-20T01:47:38.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0026e-05 2024-04-20T01:47:38.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.782742+0000", 2024-04-20T01:47:38.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001496079 2024-04-20T01:47:38.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.782753+0000", 2024-04-20T01:47:38.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0767999999999999e-05 2024-04-20T01:47:38.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790103+0000", 2024-04-20T01:47:38.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073499259999999997 2024-04-20T01:47:38.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790117+0000", 2024-04-20T01:47:38.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4729000000000001e-05 2024-04-20T01:47:38.945 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.945 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.945 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.945 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.5s4 33/30 ECSubWrite(tid=5, reqid=client.4643.0:65, at_version=33'4, trim_to=0'0, roll_forward_to=32'2))", 2024-04-20T01:47:38.945 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.783666+0000", 2024-04-20T01:47:38.945 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.1421364909999996, 2024-04-20T01:47:38.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0063804899999999999, 2024-04-20T01:47:38.946 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.946 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.946 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783666+0000", 2024-04-20T01:47:38.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783666+0000", 2024-04-20T01:47:38.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783665+0000", 2024-04-20T01:47:38.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:38.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783685+0000", 2024-04-20T01:47:38.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9735e-05 2024-04-20T01:47:38.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783701+0000", 2024-04-20T01:47:38.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6175e-05 2024-04-20T01:47:38.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783710+0000", 2024-04-20T01:47:38.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3649999999999993e-06 2024-04-20T01:47:38.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783800+0000", 2024-04-20T01:47:38.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0061000000000004e-05 2024-04-20T01:47:38.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783809+0000", 2024-04-20T01:47:38.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0750000000000004e-06 2024-04-20T01:47:38.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790003+0000", 2024-04-20T01:47:38.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0061941590000000003 2024-04-20T01:47:38.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790046+0000", 2024-04-20T01:47:38.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2831999999999999e-05 2024-04-20T01:47:38.949 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.949 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.950 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.950 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/30 ECSubWrite(tid=24, reqid=client.4643.0:100, at_version=33'9, trim_to=0'0, roll_forward_to=33'8))", 2024-04-20T01:47:38.950 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.818111+0000", 2024-04-20T01:47:38.950 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.1076909590000001, 2024-04-20T01:47:38.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0036699419999999998, 2024-04-20T01:47:38.950 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.950 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.950 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.818111+0000", 2024-04-20T01:47:38.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.818111+0000", 2024-04-20T01:47:38.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.818110+0000", 2024-04-20T01:47:38.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:38.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.818236+0000", 2024-04-20T01:47:38.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012526099999999999 2024-04-20T01:47:38.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.818253+0000", 2024-04-20T01:47:38.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7377e-05 2024-04-20T01:47:38.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.818260+0000", 2024-04-20T01:47:38.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.1999999999999997e-06 2024-04-20T01:47:38.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.818315+0000", 2024-04-20T01:47:38.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4258000000000001e-05 2024-04-20T01:47:38.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.818321+0000", 2024-04-20T01:47:38.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7909999999999999e-06 2024-04-20T01:47:38.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.821755+0000", 2024-04-20T01:47:38.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034333060000000001 2024-04-20T01:47:38.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.821781+0000", 2024-04-20T01:47:38.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6704e-05 2024-04-20T01:47:38.954 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.954 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.954 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.954 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/30 ECSubWrite(tid=25, reqid=client.4643.0:101, at_version=33'10, trim_to=0'0, roll_forward_to=33'8))", 2024-04-20T01:47:38.954 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.819138+0000", 2024-04-20T01:47:38.954 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.1066644999999999, 2024-04-20T01:47:38.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004933425, 2024-04-20T01:47:38.954 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.954 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.954 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.819138+0000", 2024-04-20T01:47:38.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.819138+0000", 2024-04-20T01:47:38.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.819137+0000", 2024-04-20T01:47:38.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:38.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.819158+0000", 2024-04-20T01:47:38.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048e-05 2024-04-20T01:47:38.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.819169+0000", 2024-04-20T01:47:38.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1922999999999999e-05 2024-04-20T01:47:38.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.819180+0000", 2024-04-20T01:47:38.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0995000000000001e-05 2024-04-20T01:47:38.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.819244+0000", 2024-04-20T01:47:38.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3212000000000004e-05 2024-04-20T01:47:38.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.819254+0000", 2024-04-20T01:47:38.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0198e-05 2024-04-20T01:47:38.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.824053+0000", 2024-04-20T01:47:38.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047987969999999996 2024-04-20T01:47:38.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.824071+0000", 2024-04-20T01:47:38.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8576999999999998e-05 2024-04-20T01:47:38.958 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.958 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.958 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.958 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs5 33/30 ECSubWrite(tid=0, reqid=client.4643.0:106, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:38.958 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.825844+0000", 2024-04-20T01:47:38.958 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.0999585290000002, 2024-04-20T01:47:38.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0062702069999999999, 2024-04-20T01:47:38.958 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.958 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.959 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.825844+0000", 2024-04-20T01:47:38.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.825844+0000", 2024-04-20T01:47:38.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.825843+0000", 2024-04-20T01:47:38.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:38.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826258+0000", 2024-04-20T01:47:38.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00041487099999999998 2024-04-20T01:47:38.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826269+0000", 2024-04-20T01:47:38.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0264e-05 2024-04-20T01:47:38.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826275+0000", 2024-04-20T01:47:38.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3010000000000003e-06 2024-04-20T01:47:38.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826329+0000", 2024-04-20T01:47:38.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4036999999999998e-05 2024-04-20T01:47:38.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826337+0000", 2024-04-20T01:47:38.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7470000000000005e-06 2024-04-20T01:47:38.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.832089+0000", 2024-04-20T01:47:38.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0057520339999999996 2024-04-20T01:47:38.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.832114+0000", 2024-04-20T01:47:38.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5267999999999999e-05 2024-04-20T01:47:38.962 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.962 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.962 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.962 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs5 33/30 ECSubWrite(tid=1, reqid=client.4643.0:107, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:38.962 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.826758+0000", 2024-04-20T01:47:38.963 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.0990443289999998, 2024-04-20T01:47:38.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0053877639999999997, 2024-04-20T01:47:38.963 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.963 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.963 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826758+0000", 2024-04-20T01:47:38.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826758+0000", 2024-04-20T01:47:38.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826757+0000", 2024-04-20T01:47:38.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:38.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826771+0000", 2024-04-20T01:47:38.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3735e-05 2024-04-20T01:47:38.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826782+0000", 2024-04-20T01:47:38.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1071e-05 2024-04-20T01:47:38.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826792+0000", 2024-04-20T01:47:38.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4970000000000007e-06 2024-04-20T01:47:38.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826933+0000", 2024-04-20T01:47:38.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014126699999999999 2024-04-20T01:47:38.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826942+0000", 2024-04-20T01:47:38.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5229999999999995e-06 2024-04-20T01:47:38.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.832129+0000", 2024-04-20T01:47:38.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051871479999999999 2024-04-20T01:47:38.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.832146+0000", 2024-04-20T01:47:38.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7081000000000001e-05 2024-04-20T01:47:38.966 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.966 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.966 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.967 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/30 ECSubWrite(tid=30, reqid=client.4643.0:133, at_version=33'11, trim_to=0'0, roll_forward_to=33'10))", 2024-04-20T01:47:38.967 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.861919+0000", 2024-04-20T01:47:38.967 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.0638831099999999, 2024-04-20T01:47:38.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034323180000000002, 2024-04-20T01:47:38.967 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.967 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.967 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.861919+0000", 2024-04-20T01:47:38.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.861919+0000", 2024-04-20T01:47:38.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.861918+0000", 2024-04-20T01:47:38.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:38.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.862078+0000", 2024-04-20T01:47:38.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015966799999999999 2024-04-20T01:47:38.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.862098+0000", 2024-04-20T01:47:38.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0794000000000001e-05 2024-04-20T01:47:38.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.862110+0000", 2024-04-20T01:47:38.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1772e-05 2024-04-20T01:47:38.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.862175+0000", 2024-04-20T01:47:38.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5198999999999997e-05 2024-04-20T01:47:38.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.862187+0000", 2024-04-20T01:47:38.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1681999999999999e-05 2024-04-20T01:47:38.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.865326+0000", 2024-04-20T01:47:38.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0031394890000000001 2024-04-20T01:47:38.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.865352+0000", 2024-04-20T01:47:38.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5038e-05 2024-04-20T01:47:38.970 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.970 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.971 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.971 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/30 ECSubWrite(tid=32, reqid=client.4643.0:134, at_version=33'12, trim_to=0'0, roll_forward_to=33'10))", 2024-04-20T01:47:38.972 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.862933+0000", 2024-04-20T01:47:38.972 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.0628690570000003, 2024-04-20T01:47:38.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0049452619999999997, 2024-04-20T01:47:38.972 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.972 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.972 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.862933+0000", 2024-04-20T01:47:38.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.862933+0000", 2024-04-20T01:47:38.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.862932+0000", 2024-04-20T01:47:38.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:38.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.862957+0000", 2024-04-20T01:47:38.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4629000000000001e-05 2024-04-20T01:47:38.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.862975+0000", 2024-04-20T01:47:38.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7618999999999999e-05 2024-04-20T01:47:38.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.862990+0000", 2024-04-20T01:47:38.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5551999999999999e-05 2024-04-20T01:47:38.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.863224+0000", 2024-04-20T01:47:38.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023420400000000001 2024-04-20T01:47:38.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.863234+0000", 2024-04-20T01:47:38.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6700000000000006e-06 2024-04-20T01:47:38.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.867852+0000", 2024-04-20T01:47:38.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046182419999999998 2024-04-20T01:47:38.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.867879+0000", 2024-04-20T01:47:38.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6241000000000002e-05 2024-04-20T01:47:38.975 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.975 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.976 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.976 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs5 33/30 ECSubWrite(tid=9, reqid=client.4643.0:151, at_version=33'6, trim_to=0'0, roll_forward_to=33'4))", 2024-04-20T01:47:38.976 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.885588+0000", 2024-04-20T01:47:38.976 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.0402140019999999, 2024-04-20T01:47:38.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0037389630000000001, 2024-04-20T01:47:38.976 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.976 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.976 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.885588+0000", 2024-04-20T01:47:38.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.885588+0000", 2024-04-20T01:47:38.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.885588+0000", 2024-04-20T01:47:38.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:38.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.885738+0000", 2024-04-20T01:47:38.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000150168 2024-04-20T01:47:38.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.885749+0000", 2024-04-20T01:47:38.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1463e-05 2024-04-20T01:47:38.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.885759+0000", 2024-04-20T01:47:38.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3379999999999994e-06 2024-04-20T01:47:38.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.885836+0000", 2024-04-20T01:47:38.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6959999999999995e-05 2024-04-20T01:47:38.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.885845+0000", 2024-04-20T01:47:38.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8779999999999993e-06 2024-04-20T01:47:38.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.889298+0000", 2024-04-20T01:47:38.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0034528390000000001 2024-04-20T01:47:38.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.889327+0000", 2024-04-20T01:47:38.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9013000000000002e-05 2024-04-20T01:47:38.980 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.980 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.980 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.980 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.es6 33/30 ECSubWrite(tid=2, reqid=client.4643.0:103, at_version=33'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:38.980 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.965527+0000", 2024-04-20T01:47:38.980 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.960275024, 2024-04-20T01:47:38.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048030770000000002, 2024-04-20T01:47:38.981 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.981 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.981 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.965527+0000", 2024-04-20T01:47:38.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.965527+0000", 2024-04-20T01:47:38.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.965527+0000", 2024-04-20T01:47:38.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:38.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.965977+0000", 2024-04-20T01:47:38.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00044985499999999998 2024-04-20T01:47:38.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.965991+0000", 2024-04-20T01:47:38.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4572999999999999e-05 2024-04-20T01:47:38.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966001+0000", 2024-04-20T01:47:38.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9110000000000005e-06 2024-04-20T01:47:38.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966059+0000", 2024-04-20T01:47:38.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8248999999999997e-05 2024-04-20T01:47:38.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966075+0000", 2024-04-20T01:47:38.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5347999999999999e-05 2024-04-20T01:47:38.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.970302+0000", 2024-04-20T01:47:38.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042272509999999996 2024-04-20T01:47:38.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.970330+0000", 2024-04-20T01:47:38.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8396000000000001e-05 2024-04-20T01:47:38.984 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.984 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.984 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.985 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.es6 33/30 ECSubWrite(tid=3, reqid=client.4643.0:104, at_version=33'4, trim_to=0'0, roll_forward_to=33'1))", 2024-04-20T01:47:38.985 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.966859+0000", 2024-04-20T01:47:38.985 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.958942832, 2024-04-20T01:47:38.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0063969730000000002, 2024-04-20T01:47:38.985 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.985 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.985 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966859+0000", 2024-04-20T01:47:38.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966859+0000", 2024-04-20T01:47:38.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966859+0000", 2024-04-20T01:47:38.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:38.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966876+0000", 2024-04-20T01:47:38.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7116999999999999e-05 2024-04-20T01:47:38.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966887+0000", 2024-04-20T01:47:38.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1508999999999999e-05 2024-04-20T01:47:38.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966894+0000", 2024-04-20T01:47:38.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4219999999999997e-06 2024-04-20T01:47:38.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.967342+0000", 2024-04-20T01:47:38.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00044805799999999998 2024-04-20T01:47:38.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.967352+0000", 2024-04-20T01:47:38.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0497e-05 2024-04-20T01:47:38.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.973223+0000", 2024-04-20T01:47:38.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0058706749999999997 2024-04-20T01:47:38.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.973256+0000", 2024-04-20T01:47:38.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3544999999999998e-05 2024-04-20T01:47:38.988 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.988 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.989 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.989 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.es6 33/30 ECSubWrite(tid=4, reqid=client.4643.0:109, at_version=33'5, trim_to=0'0, roll_forward_to=33'1))", 2024-04-20T01:47:38.989 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.969319+0000", 2024-04-20T01:47:38.989 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.9564837690000001, 2024-04-20T01:47:38.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0039491470000000001, 2024-04-20T01:47:38.989 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.989 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.989 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.969319+0000", 2024-04-20T01:47:38.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.969319+0000", 2024-04-20T01:47:38.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.969318+0000", 2024-04-20T01:47:38.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:38.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.969598+0000", 2024-04-20T01:47:38.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00028054399999999998 2024-04-20T01:47:38.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.969609+0000", 2024-04-20T01:47:38.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0319999999999999e-05 2024-04-20T01:47:38.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.969616+0000", 2024-04-20T01:47:38.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0700000000000001e-06 2024-04-20T01:47:38.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.969664+0000", 2024-04-20T01:47:38.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8439000000000002e-05 2024-04-20T01:47:38.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.969675+0000", 2024-04-20T01:47:38.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0559e-05 2024-04-20T01:47:38.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.973260+0000", 2024-04-20T01:47:38.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0035855930000000002 2024-04-20T01:47:38.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.973268+0000", 2024-04-20T01:47:38.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2969999999999998e-06 2024-04-20T01:47:38.993 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.993 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.993 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.993 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.es6 33/30 ECSubWrite(tid=5, reqid=client.4643.0:110, at_version=33'6, trim_to=0'0, roll_forward_to=33'1))", 2024-04-20T01:47:38.993 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.969623+0000", 2024-04-20T01:47:38.993 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.956179015, 2024-04-20T01:47:38.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003650623, 2024-04-20T01:47:38.993 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.993 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.994 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.969623+0000", 2024-04-20T01:47:38.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.969623+0000", 2024-04-20T01:47:38.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.969623+0000", 2024-04-20T01:47:38.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:38.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.969631+0000", 2024-04-20T01:47:38.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4370000000000007e-06 2024-04-20T01:47:38.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.969636+0000", 2024-04-20T01:47:38.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2220000000000004e-06 2024-04-20T01:47:38.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:38.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.969640+0000", 2024-04-20T01:47:38.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3250000000000001e-06 2024-04-20T01:47:38.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:38.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.970119+0000", 2024-04-20T01:47:38.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00047854000000000001 2024-04-20T01:47:38.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:38.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.970129+0000", 2024-04-20T01:47:38.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0356999999999999e-05 2024-04-20T01:47:38.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:38.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.973269+0000", 2024-04-20T01:47:38.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0031398149999999998 2024-04-20T01:47:38.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:38.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.973274+0000", 2024-04-20T01:47:38.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.237e-06 2024-04-20T01:47:38.997 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.997 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:38.997 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:38.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.997 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.es6 33/30 ECSubWrite(tid=7, reqid=client.4643.0:142, at_version=33'7, trim_to=0'0, roll_forward_to=33'3))", 2024-04-20T01:47:38.997 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.972401+0000", 2024-04-20T01:47:38.997 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 3.9534014109999998, 2024-04-20T01:47:38.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0036387580000000002, 2024-04-20T01:47:38.998 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:38.998 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:38.998 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:38.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:38.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.972401+0000", 2024-04-20T01:47:38.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:38.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.972401+0000", 2024-04-20T01:47:38.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:38.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:38.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.972400+0000", 2024-04-20T01:47:38.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:38.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:38.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.972742+0000", 2024-04-20T01:47:38.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00034158799999999998 2024-04-20T01:47:38.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:38.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.972754+0000", 2024-04-20T01:47:38.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2444e-05 2024-04-20T01:47:38.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:38.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:38.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.972764+0000", 2024-04-20T01:47:39.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7059999999999999e-06 2024-04-20T01:47:39.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:39.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:39.000 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:39.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.972812+0000", 2024-04-20T01:47:39.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8282000000000003e-05 2024-04-20T01:47:39.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:39.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:39.000 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:39.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.972821+0000", 2024-04-20T01:47:39.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9260000000000006e-06 2024-04-20T01:47:39.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:39.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:39.000 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:39.001 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.976013+0000", 2024-04-20T01:47:39.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0031914339999999999 2024-04-20T01:47:39.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:39.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:39.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:39.001 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.976040+0000", 2024-04-20T01:47:39.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6979999999999999e-05 2024-04-20T01:47:39.001 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:39.001 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:39.001 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:39.001 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:39.002 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:39.002 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:39.003 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-20T01:47:39.004 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:39.017+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:39.104 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:47:39.118+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:47:39.145 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:39.145 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:47:39.145 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 0 2024-04-20T01:47:39.145 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:39.155 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-20T01:47:39.205 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:47:39.219+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:47:39.284 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:39.284 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:47:39.284 INFO:teuthology.orchestra.run.smithi121.stdout: "complaint_time": 30, 2024-04-20T01:47:39.284 INFO:teuthology.orchestra.run.smithi121.stdout: "num_blocked_ops": 0 2024-04-20T01:47:39.284 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:39.293 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-20T01:47:39.337 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:39.351+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:47:39.407 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:39.421+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:47:39.422 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:39.422 INFO:teuthology.orchestra.run.smithi121.stdout: "size": 20, 2024-04-20T01:47:39.422 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 600, 2024-04-20T01:47:39.422 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-04-20T01:47:39.422 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.422 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 36/30 ECSubWrite(tid=59, reqid=client.4643.0:214, at_version=36'14, trim_to=0'0, roll_forward_to=33'12))", 2024-04-20T01:47:39.422 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:37.785654+0000", 2024-04-20T01:47:39.422 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 1.6498610089999999, 2024-04-20T01:47:39.423 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0014526999999999999, 2024-04-20T01:47:39.423 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:39.423 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:39.423 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:39.423 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.423 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:39.423 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785654+0000", 2024-04-20T01:47:39.423 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.423 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.423 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.423 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:39.423 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785654+0000", 2024-04-20T01:47:39.424 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.424 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.424 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.424 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:39.424 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785654+0000", 2024-04-20T01:47:39.424 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:47:39.424 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.424 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.424 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:39.424 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785682+0000", 2024-04-20T01:47:39.424 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8036999999999999e-05 2024-04-20T01:47:39.424 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.424 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.424 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:39.425 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785691+0000", 2024-04-20T01:47:39.425 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3840000000000003e-06 2024-04-20T01:47:39.425 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.425 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.425 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.425 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785701+0000", 2024-04-20T01:47:39.425 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.5889999999999992e-06 2024-04-20T01:47:39.425 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.425 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.425 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:39.425 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785778+0000", 2024-04-20T01:47:39.425 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.7577999999999995e-05 2024-04-20T01:47:39.425 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.426 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.426 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:39.426 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785796+0000", 2024-04-20T01:47:39.426 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7431e-05 2024-04-20T01:47:39.426 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.426 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.426 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:39.426 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.787077+0000", 2024-04-20T01:47:39.426 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0012813340000000001 2024-04-20T01:47:39.426 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.426 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.427 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:39.427 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.787106+0000", 2024-04-20T01:47:39.427 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9507000000000001e-05 2024-04-20T01:47:39.427 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.427 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.427 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.427 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.427 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.427 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 36/30 ECSubWrite(tid=32, reqid=osd.8.0:32, at_version=36'22, trim_to=0'0, roll_forward_to=35'20))", 2024-04-20T01:47:39.427 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:37.789795+0000", 2024-04-20T01:47:39.427 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 1.6457201860000001, 2024-04-20T01:47:39.427 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00080479399999999995, 2024-04-20T01:47:39.428 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:39.428 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:39.428 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:39.428 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.428 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:39.428 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.789795+0000", 2024-04-20T01:47:39.428 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.428 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.428 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.428 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:39.428 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.789795+0000", 2024-04-20T01:47:39.428 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.428 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.428 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.428 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:39.429 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.789794+0000", 2024-04-20T01:47:39.429 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:47:39.429 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.429 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.429 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:39.429 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.789812+0000", 2024-04-20T01:47:39.429 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7183000000000001e-05 2024-04-20T01:47:39.429 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.429 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.429 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:39.429 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.789820+0000", 2024-04-20T01:47:39.429 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.2810000000000008e-06 2024-04-20T01:47:39.429 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.429 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.430 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.430 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.789826+0000", 2024-04-20T01:47:39.430 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.4980000000000003e-06 2024-04-20T01:47:39.430 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.430 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.430 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:39.430 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.789878+0000", 2024-04-20T01:47:39.430 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.1388999999999999e-05 2024-04-20T01:47:39.430 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.430 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.430 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:39.430 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.789892+0000", 2024-04-20T01:47:39.430 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4712000000000001e-05 2024-04-20T01:47:39.431 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.431 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.431 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:39.431 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.790569+0000", 2024-04-20T01:47:39.431 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00067674800000000002 2024-04-20T01:47:39.431 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.431 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.431 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:39.431 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.790599+0000", 2024-04-20T01:47:39.431 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0134999999999999e-05 2024-04-20T01:47:39.431 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.431 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.431 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.431 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.432 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.432 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 36/30 ECSubWrite(tid=33, reqid=osd.8.0:33, at_version=36'24, trim_to=0'0, roll_forward_to=35'20))", 2024-04-20T01:47:39.432 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:37.790170+0000", 2024-04-20T01:47:39.432 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 1.6453452079999999, 2024-04-20T01:47:39.432 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000824787, 2024-04-20T01:47:39.432 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:39.432 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:39.432 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:39.432 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.432 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:39.432 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.790170+0000", 2024-04-20T01:47:39.432 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.432 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.432 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.432 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:39.433 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.790170+0000", 2024-04-20T01:47:39.433 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.433 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.433 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.433 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:39.433 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.790169+0000", 2024-04-20T01:47:39.433 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:39.433 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.433 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.433 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:39.433 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.790188+0000", 2024-04-20T01:47:39.433 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9315999999999999e-05 2024-04-20T01:47:39.433 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.433 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.434 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:39.434 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.790197+0000", 2024-04-20T01:47:39.434 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4369999999999996e-06 2024-04-20T01:47:39.434 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.434 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.434 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.434 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.790204+0000", 2024-04-20T01:47:39.434 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.7279999999999998e-06 2024-04-20T01:47:39.434 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.434 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.434 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:39.434 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.790251+0000", 2024-04-20T01:47:39.434 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.7212000000000001e-05 2024-04-20T01:47:39.434 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.435 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.435 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:39.435 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.790263+0000", 2024-04-20T01:47:39.435 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1929e-05 2024-04-20T01:47:39.435 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.435 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.435 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:39.435 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.790974+0000", 2024-04-20T01:47:39.435 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00071038399999999999 2024-04-20T01:47:39.435 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.435 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.435 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:39.435 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.790994+0000", 2024-04-20T01:47:39.436 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0755999999999999e-05 2024-04-20T01:47:39.436 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.436 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.436 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.436 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.436 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.436 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 36/30 ECSubWrite(tid=34, reqid=osd.8.0:33, at_version=0'0, trim_to=0'0, roll_forward_to=36'24))", 2024-04-20T01:47:39.436 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:37.795633+0000", 2024-04-20T01:47:39.436 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 1.6398815689999999, 2024-04-20T01:47:39.436 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0010909139999999999, 2024-04-20T01:47:39.436 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:39.436 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:39.436 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:39.436 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.437 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:39.437 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.795633+0000", 2024-04-20T01:47:39.437 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.437 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.437 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.437 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:39.437 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.795633+0000", 2024-04-20T01:47:39.437 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.437 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.437 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.437 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:39.437 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.795633+0000", 2024-04-20T01:47:39.437 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:39.438 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.438 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.438 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:39.438 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.795643+0000", 2024-04-20T01:47:39.438 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.8500000000000006e-06 2024-04-20T01:47:39.438 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.438 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.438 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:39.438 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.795651+0000", 2024-04-20T01:47:39.438 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.9610000000000005e-06 2024-04-20T01:47:39.438 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.438 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.438 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.438 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.795659+0000", 2024-04-20T01:47:39.439 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.901e-06 2024-04-20T01:47:39.439 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.439 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.439 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:39.439 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.795755+0000", 2024-04-20T01:47:39.439 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.5261999999999997e-05 2024-04-20T01:47:39.439 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.439 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.439 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:39.439 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.795773+0000", 2024-04-20T01:47:39.439 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7946000000000001e-05 2024-04-20T01:47:39.439 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.439 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.440 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:39.440 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.796702+0000", 2024-04-20T01:47:39.440 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00092939800000000001 2024-04-20T01:47:39.440 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.440 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.440 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:39.440 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.796724+0000", 2024-04-20T01:47:39.440 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1967000000000002e-05 2024-04-20T01:47:39.440 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.440 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.440 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.440 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.440 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.440 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 36/30 ECSubWrite(tid=35, reqid=client.4643.0:217, at_version=36'26, trim_to=0'0, roll_forward_to=36'24))", 2024-04-20T01:47:39.441 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:37.800452+0000", 2024-04-20T01:47:39.441 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 1.635062445, 2024-04-20T01:47:39.441 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0017959180000000001, 2024-04-20T01:47:39.441 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:39.441 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:39.441 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:39.441 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.441 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:39.441 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.800452+0000", 2024-04-20T01:47:39.441 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.441 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.441 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.441 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:39.443 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.800452+0000", 2024-04-20T01:47:39.443 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.443 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.443 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.443 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:39.443 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.800451+0000", 2024-04-20T01:47:39.443 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:39.443 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.443 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.443 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:39.443 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.800557+0000", 2024-04-20T01:47:39.443 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000105303 2024-04-20T01:47:39.443 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.443 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.444 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:39.444 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.800569+0000", 2024-04-20T01:47:39.444 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1871e-05 2024-04-20T01:47:39.444 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.444 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.444 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.444 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.800577+0000", 2024-04-20T01:47:39.444 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.2300000000000008e-06 2024-04-20T01:47:39.444 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.444 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.444 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:39.444 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.800626+0000", 2024-04-20T01:47:39.444 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.9478999999999998e-05 2024-04-20T01:47:39.445 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.445 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.445 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:39.445 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.800634+0000", 2024-04-20T01:47:39.445 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.6720000000000004e-06 2024-04-20T01:47:39.445 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.445 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.445 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:39.445 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.802227+0000", 2024-04-20T01:47:39.445 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001592977 2024-04-20T01:47:39.445 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.445 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.445 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:39.445 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.802248+0000", 2024-04-20T01:47:39.446 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1299000000000001e-05 2024-04-20T01:47:39.446 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.446 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.446 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.446 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.446 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.446 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 36/30 ECSubWrite(tid=40, reqid=client.4643.0:215, at_version=36'28, trim_to=0'0, roll_forward_to=36'26))", 2024-04-20T01:47:39.446 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:37.808702+0000", 2024-04-20T01:47:39.446 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 1.6268131910000001, 2024-04-20T01:47:39.446 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00097697600000000006, 2024-04-20T01:47:39.446 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:39.446 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:39.446 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:39.447 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.447 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:39.447 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.808702+0000", 2024-04-20T01:47:39.447 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.447 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.447 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.447 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:39.447 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.808702+0000", 2024-04-20T01:47:39.447 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.447 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.447 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.447 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:39.447 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.808701+0000", 2024-04-20T01:47:39.447 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:39.448 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.448 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.448 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:39.448 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.808728+0000", 2024-04-20T01:47:39.448 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7341000000000001e-05 2024-04-20T01:47:39.448 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.448 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.448 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:39.448 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.808739+0000", 2024-04-20T01:47:39.448 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0138e-05 2024-04-20T01:47:39.448 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.448 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.448 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.448 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.808746+0000", 2024-04-20T01:47:39.449 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.824e-06 2024-04-20T01:47:39.449 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.449 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.449 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:39.449 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.808779+0000", 2024-04-20T01:47:39.449 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.2159000000000002e-05 2024-04-20T01:47:39.449 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.449 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.449 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:39.449 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.808784+0000", 2024-04-20T01:47:39.449 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.378e-06 2024-04-20T01:47:39.450 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.450 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.450 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:39.450 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.809652+0000", 2024-04-20T01:47:39.450 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00086780799999999997 2024-04-20T01:47:39.450 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.450 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.450 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:39.450 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.809679+0000", 2024-04-20T01:47:39.450 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6784999999999999e-05 2024-04-20T01:47:39.450 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.450 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.451 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.451 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.451 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.451 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 36/30 ECSubWrite(tid=41, reqid=client.4643.0:215, at_version=0'0, trim_to=0'0, roll_forward_to=36'28))", 2024-04-20T01:47:39.451 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:37.817494+0000", 2024-04-20T01:47:39.451 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 1.6180205590000001, 2024-04-20T01:47:39.451 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00076162199999999999, 2024-04-20T01:47:39.451 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:39.451 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:39.451 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:39.451 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.451 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:39.452 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.817494+0000", 2024-04-20T01:47:39.452 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.452 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.452 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.452 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:39.452 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.817494+0000", 2024-04-20T01:47:39.452 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.452 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.452 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.452 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:39.452 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.817492+0000", 2024-04-20T01:47:39.452 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999981 2024-04-20T01:47:39.452 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.453 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.453 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:39.453 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.817520+0000", 2024-04-20T01:47:39.453 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8129000000000001e-05 2024-04-20T01:47:39.453 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.453 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.453 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:39.453 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.817558+0000", 2024-04-20T01:47:39.453 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7907000000000001e-05 2024-04-20T01:47:39.453 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.453 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.453 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.453 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.817581+0000", 2024-04-20T01:47:39.454 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2289e-05 2024-04-20T01:47:39.454 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.454 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.454 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:39.454 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.817708+0000", 2024-04-20T01:47:39.454 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00012778900000000001 2024-04-20T01:47:39.454 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.454 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.454 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:39.454 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.817724+0000", 2024-04-20T01:47:39.454 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5713e-05 2024-04-20T01:47:39.454 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.454 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.455 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:39.455 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.818230+0000", 2024-04-20T01:47:39.455 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00050620400000000001 2024-04-20T01:47:39.455 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.455 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.455 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:39.455 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.818256+0000", 2024-04-20T01:47:39.455 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5443999999999999e-05 2024-04-20T01:47:39.455 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.455 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.455 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.455 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.455 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.456 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 37/30 ECSubWrite(tid=42, reqid=client.4643.0:240, at_version=37'30, trim_to=0'0, roll_forward_to=36'28))", 2024-04-20T01:47:39.456 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.792271+0000", 2024-04-20T01:47:39.456 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.64324383399999996, 2024-04-20T01:47:39.456 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.003848785, 2024-04-20T01:47:39.456 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:39.456 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:39.456 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:39.456 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.456 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:39.456 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792271+0000", 2024-04-20T01:47:39.456 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.456 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.456 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.457 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:39.457 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792271+0000", 2024-04-20T01:47:39.457 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.457 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.457 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.457 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:39.457 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792271+0000", 2024-04-20T01:47:39.457 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:39.457 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.457 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.457 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:39.457 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792301+0000", 2024-04-20T01:47:39.457 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0295999999999999e-05 2024-04-20T01:47:39.457 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.458 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.458 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:39.458 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792317+0000", 2024-04-20T01:47:39.458 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5928999999999999e-05 2024-04-20T01:47:39.458 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.458 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.458 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.458 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792329+0000", 2024-04-20T01:47:39.458 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1764e-05 2024-04-20T01:47:39.458 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.458 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.458 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:39.459 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792396+0000", 2024-04-20T01:47:39.459 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.7780999999999994e-05 2024-04-20T01:47:39.459 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.459 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.459 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:39.459 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792409+0000", 2024-04-20T01:47:39.459 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2947e-05 2024-04-20T01:47:39.459 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.459 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.459 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:39.459 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796084+0000", 2024-04-20T01:47:39.459 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0036750730000000001 2024-04-20T01:47:39.460 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.460 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.460 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:39.460 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796120+0000", 2024-04-20T01:47:39.460 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.5367999999999997e-05 2024-04-20T01:47:39.460 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.460 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.460 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.460 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.460 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.460 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s3 37/30 ECSubWrite(tid=18, reqid=client.4643.0:234, at_version=37'8, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:39.460 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.792596+0000", 2024-04-20T01:47:39.460 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.64291906200000004, 2024-04-20T01:47:39.461 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0034997589999999999, 2024-04-20T01:47:39.461 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:39.461 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:39.461 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:39.461 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.461 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:39.461 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792596+0000", 2024-04-20T01:47:39.461 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.461 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.461 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.461 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:39.461 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792596+0000", 2024-04-20T01:47:39.461 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.461 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.462 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.462 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:39.462 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792595+0000", 2024-04-20T01:47:39.462 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:47:39.462 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.462 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.462 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:39.462 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792625+0000", 2024-04-20T01:47:39.462 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9833999999999999e-05 2024-04-20T01:47:39.462 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.462 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.462 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:39.462 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792637+0000", 2024-04-20T01:47:39.463 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1914e-05 2024-04-20T01:47:39.463 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.463 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.463 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.463 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792646+0000", 2024-04-20T01:47:39.463 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.0289999999999994e-06 2024-04-20T01:47:39.463 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.463 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.463 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:39.463 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792743+0000", 2024-04-20T01:47:39.463 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7161e-05 2024-04-20T01:47:39.463 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.463 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.463 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:39.464 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792754+0000", 2024-04-20T01:47:39.464 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0462000000000001e-05 2024-04-20T01:47:39.464 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.464 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.464 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:39.464 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796052+0000", 2024-04-20T01:47:39.464 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0032983040000000002 2024-04-20T01:47:39.464 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.464 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.464 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:39.464 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796095+0000", 2024-04-20T01:47:39.464 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.3226000000000003e-05 2024-04-20T01:47:39.464 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.464 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.464 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.465 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.465 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.465 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 37/30 ECSubWrite(tid=64, reqid=client.4643.0:225, at_version=37'16, trim_to=0'0, roll_forward_to=36'14))", 2024-04-20T01:47:39.465 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.792788+0000", 2024-04-20T01:47:39.465 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.642726307, 2024-04-20T01:47:39.465 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0037545550000000001, 2024-04-20T01:47:39.465 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:39.465 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:39.465 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:39.465 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.465 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:39.465 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792788+0000", 2024-04-20T01:47:39.465 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.465 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.466 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.466 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:39.466 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792788+0000", 2024-04-20T01:47:39.466 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.466 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.466 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.466 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:39.466 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792788+0000", 2024-04-20T01:47:39.466 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:47:39.466 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.466 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.466 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:39.466 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793103+0000", 2024-04-20T01:47:39.466 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00031501700000000001 2024-04-20T01:47:39.467 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.467 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.467 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:39.467 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793117+0000", 2024-04-20T01:47:39.467 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3372e-05 2024-04-20T01:47:39.467 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.467 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.467 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.467 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793130+0000", 2024-04-20T01:47:39.467 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.385e-05 2024-04-20T01:47:39.467 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.467 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.467 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:39.468 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793192+0000", 2024-04-20T01:47:39.468 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.1029999999999997e-05 2024-04-20T01:47:39.468 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.468 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.468 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:39.468 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793202+0000", 2024-04-20T01:47:39.468 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0856000000000001e-05 2024-04-20T01:47:39.468 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.468 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.468 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:39.468 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796516+0000", 2024-04-20T01:47:39.468 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.003313215 2024-04-20T01:47:39.468 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.469 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.469 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:39.469 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796543+0000", 2024-04-20T01:47:39.469 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7384000000000001e-05 2024-04-20T01:47:39.469 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.469 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.469 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.469 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.469 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.469 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 37/30 ECSubWrite(tid=65, reqid=client.4643.0:226, at_version=37'17, trim_to=0'0, roll_forward_to=36'14))", 2024-04-20T01:47:39.469 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.793149+0000", 2024-04-20T01:47:39.469 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.64236541999999996, 2024-04-20T01:47:39.469 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.003421262, 2024-04-20T01:47:39.469 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:39.470 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:39.470 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:39.470 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.470 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:39.470 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793149+0000", 2024-04-20T01:47:39.470 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.470 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.470 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.470 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:39.470 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793149+0000", 2024-04-20T01:47:39.470 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.470 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.470 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.470 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:39.470 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793149+0000", 2024-04-20T01:47:39.471 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:39.471 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.471 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.471 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:39.471 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793256+0000", 2024-04-20T01:47:39.471 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000107203 2024-04-20T01:47:39.471 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.471 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.471 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:39.471 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793271+0000", 2024-04-20T01:47:39.471 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.558e-05 2024-04-20T01:47:39.471 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.471 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.472 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.472 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793281+0000", 2024-04-20T01:47:39.472 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3850000000000008e-06 2024-04-20T01:47:39.472 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.472 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.472 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:39.472 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.794184+0000", 2024-04-20T01:47:39.472 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00090338699999999996 2024-04-20T01:47:39.472 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.472 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.472 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:39.472 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.794195+0000", 2024-04-20T01:47:39.472 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0319999999999999e-05 2024-04-20T01:47:39.473 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.473 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.473 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:39.473 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796556+0000", 2024-04-20T01:47:39.473 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0023616969999999998 2024-04-20T01:47:39.473 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.473 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.473 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:39.473 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796571+0000", 2024-04-20T01:47:39.473 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4345000000000001e-05 2024-04-20T01:47:39.473 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.473 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.473 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.473 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.473 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.474 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 37/30 ECSubWrite(tid=66, reqid=client.4643.0:227, at_version=37'18, trim_to=0'0, roll_forward_to=36'14))", 2024-04-20T01:47:39.474 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.793289+0000", 2024-04-20T01:47:39.474 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.64222608299999995, 2024-04-20T01:47:39.474 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0033325590000000001, 2024-04-20T01:47:39.474 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:39.474 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:39.474 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:39.474 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.474 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:39.474 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793289+0000", 2024-04-20T01:47:39.474 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.474 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.474 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.475 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:39.475 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793289+0000", 2024-04-20T01:47:39.475 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.475 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.475 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.475 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:39.475 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793288+0000", 2024-04-20T01:47:39.475 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:39.475 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.475 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.475 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:39.475 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793309+0000", 2024-04-20T01:47:39.475 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0826e-05 2024-04-20T01:47:39.475 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.475 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.476 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:39.476 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793317+0000", 2024-04-20T01:47:39.476 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.8900000000000007e-06 2024-04-20T01:47:39.476 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.476 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.476 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.476 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793323+0000", 2024-04-20T01:47:39.476 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.3459999999999999e-06 2024-04-20T01:47:39.476 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.476 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.476 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:39.476 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.794811+0000", 2024-04-20T01:47:39.476 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001488163 2024-04-20T01:47:39.476 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.477 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.477 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:39.477 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.794824+0000", 2024-04-20T01:47:39.477 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2686e-05 2024-04-20T01:47:39.477 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.477 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.477 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:39.477 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796583+0000", 2024-04-20T01:47:39.477 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0017587709999999999 2024-04-20T01:47:39.477 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.477 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.477 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:39.477 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796621+0000", 2024-04-20T01:47:39.477 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.8291999999999999e-05 2024-04-20T01:47:39.477 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.478 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.478 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.478 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.478 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.478 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpRead(3.bs2 37/30 ECSubRead(tid=67, to_read={3:da81107c:::smithi16579500-34:head=0,139264,0}, subchunks={3:da81107c:::smithi16579500-34:head=[0,1]}, attrs_to_read=))", 2024-04-20T01:47:39.478 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.793329+0000", 2024-04-20T01:47:39.478 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.64218523999999999, 2024-04-20T01:47:39.478 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0020603140000000002, 2024-04-20T01:47:39.478 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:39.478 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:39.478 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:39.478 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.478 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:39.478 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793329+0000", 2024-04-20T01:47:39.479 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.479 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.479 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.479 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:39.479 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793329+0000", 2024-04-20T01:47:39.479 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.479 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.479 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.479 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:39.479 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793329+0000", 2024-04-20T01:47:39.479 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:47:39.479 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.479 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.479 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:39.479 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793346+0000", 2024-04-20T01:47:39.480 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.696e-05 2024-04-20T01:47:39.480 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.480 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.480 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:39.480 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793356+0000", 2024-04-20T01:47:39.480 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3139999999999993e-06 2024-04-20T01:47:39.480 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.480 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.480 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.480 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793366+0000", 2024-04-20T01:47:39.480 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0456e-05 2024-04-20T01:47:39.480 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.480 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.480 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:39.480 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795257+0000", 2024-04-20T01:47:39.480 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00189124 2024-04-20T01:47:39.481 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.481 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.481 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:39.481 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795390+0000", 2024-04-20T01:47:39.481 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000132576 2024-04-20T01:47:39.481 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.481 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.481 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.481 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.481 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.481 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 37/30 ECSubWrite(tid=43, reqid=client.4643.0:241, at_version=37'31, trim_to=0'0, roll_forward_to=36'28))", 2024-04-20T01:47:39.481 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.793442+0000", 2024-04-20T01:47:39.481 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.64207280799999999, 2024-04-20T01:47:39.481 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0030637860000000002, 2024-04-20T01:47:39.482 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:39.482 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:39.482 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:39.482 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.482 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:39.482 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793442+0000", 2024-04-20T01:47:39.482 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.482 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.482 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.482 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:39.482 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793442+0000", 2024-04-20T01:47:39.482 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.483 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.483 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.483 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:39.483 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793441+0000", 2024-04-20T01:47:39.483 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:39.483 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.485 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.485 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:39.485 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793466+0000", 2024-04-20T01:47:39.486 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4494999999999998e-05 2024-04-20T01:47:39.486 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.486 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.486 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:39.486 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793479+0000", 2024-04-20T01:47:39.486 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3831e-05 2024-04-20T01:47:39.486 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.486 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.486 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.486 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793489+0000", 2024-04-20T01:47:39.486 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4800000000000007e-06 2024-04-20T01:47:39.486 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.486 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.487 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:39.487 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.794256+0000", 2024-04-20T01:47:39.487 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00076756100000000005 2024-04-20T01:47:39.487 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.487 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.487 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:39.487 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.794267+0000", 2024-04-20T01:47:39.487 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0869999999999999e-05 2024-04-20T01:47:39.487 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.487 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.487 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:39.487 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796466+0000", 2024-04-20T01:47:39.487 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0021985249999999998 2024-04-20T01:47:39.487 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.488 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.488 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:39.488 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796506+0000", 2024-04-20T01:47:39.488 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.9884e-05 2024-04-20T01:47:39.488 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.488 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.488 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.488 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.488 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.488 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s3 37/30 ECSubWrite(tid=19, reqid=client.4643.0:235, at_version=37'9, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:39.488 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.793901+0000", 2024-04-20T01:47:39.488 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.641613345, 2024-04-20T01:47:39.488 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0026976180000000002, 2024-04-20T01:47:39.489 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:39.489 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:39.489 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:39.489 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.489 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:39.489 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793901+0000", 2024-04-20T01:47:39.489 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.489 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.489 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.489 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:39.489 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793901+0000", 2024-04-20T01:47:39.489 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.489 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.489 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.490 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:39.490 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793901+0000", 2024-04-20T01:47:39.490 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:39.490 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.490 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.490 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:39.490 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793923+0000", 2024-04-20T01:47:39.490 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2591000000000001e-05 2024-04-20T01:47:39.490 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.490 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.490 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:39.490 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793937+0000", 2024-04-20T01:47:39.490 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3634e-05 2024-04-20T01:47:39.490 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.491 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.491 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.491 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793962+0000", 2024-04-20T01:47:39.491 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4766e-05 2024-04-20T01:47:39.491 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.491 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.491 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:39.491 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.794040+0000", 2024-04-20T01:47:39.491 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.8638000000000002e-05 2024-04-20T01:47:39.491 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.491 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.491 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:39.491 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.794051+0000", 2024-04-20T01:47:39.492 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0356999999999999e-05 2024-04-20T01:47:39.492 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.492 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.492 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:39.492 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796582+0000", 2024-04-20T01:47:39.492 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.002531045 2024-04-20T01:47:39.492 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.492 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.492 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:39.492 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796599+0000", 2024-04-20T01:47:39.492 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7436e-05 2024-04-20T01:47:39.492 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.492 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.492 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.493 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.493 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.493 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s3 37/30 ECSubWrite(tid=20, reqid=client.4643.0:236, at_version=37'10, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:39.493 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.795585+0000", 2024-04-20T01:47:39.493 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.63992991399999999, 2024-04-20T01:47:39.493 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.003760378, 2024-04-20T01:47:39.493 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:39.493 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:39.493 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:39.493 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.493 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:39.493 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795585+0000", 2024-04-20T01:47:39.493 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.493 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.493 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.494 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:39.494 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795585+0000", 2024-04-20T01:47:39.494 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.494 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.494 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.494 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:39.494 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795584+0000", 2024-04-20T01:47:39.494 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:39.494 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.494 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.494 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:39.494 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795605+0000", 2024-04-20T01:47:39.494 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1240999999999999e-05 2024-04-20T01:47:39.494 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.495 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.495 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:39.495 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795616+0000", 2024-04-20T01:47:39.495 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1060000000000001e-05 2024-04-20T01:47:39.495 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.495 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.495 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.495 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795625+0000", 2024-04-20T01:47:39.495 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.9360000000000005e-06 2024-04-20T01:47:39.495 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.495 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.495 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:39.495 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795688+0000", 2024-04-20T01:47:39.495 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.2565999999999997e-05 2024-04-20T01:47:39.495 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.496 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.496 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:39.496 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795696+0000", 2024-04-20T01:47:39.496 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.551e-06 2024-04-20T01:47:39.496 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.496 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.496 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:39.496 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.799321+0000", 2024-04-20T01:47:39.496 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.003624632 2024-04-20T01:47:39.496 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.496 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.496 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:39.496 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.799345+0000", 2024-04-20T01:47:39.496 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4250999999999999e-05 2024-04-20T01:47:39.496 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.497 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.497 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.497 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.497 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.497 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 37/30 ECSubWrite(tid=44, reqid=client.4643.0:242, at_version=37'32, trim_to=0'0, roll_forward_to=36'28))", 2024-04-20T01:47:39.497 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.797307+0000", 2024-04-20T01:47:39.497 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.63820808600000001, 2024-04-20T01:47:39.497 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0023789919999999999, 2024-04-20T01:47:39.497 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:39.497 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:39.497 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:39.497 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.498 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:39.498 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.797307+0000", 2024-04-20T01:47:39.498 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.498 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.498 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.498 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:39.498 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.797307+0000", 2024-04-20T01:47:39.498 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.498 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.498 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.498 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:39.498 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.797306+0000", 2024-04-20T01:47:39.498 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:39.499 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.499 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.499 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:39.499 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.797326+0000", 2024-04-20T01:47:39.499 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9758e-05 2024-04-20T01:47:39.499 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.499 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.499 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:39.499 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.797334+0000", 2024-04-20T01:47:39.499 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8449999999999992e-06 2024-04-20T01:47:39.499 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.499 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.499 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.499 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.797342+0000", 2024-04-20T01:47:39.500 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.6629999999999993e-06 2024-04-20T01:47:39.500 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.500 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.500 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:39.500 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.797381+0000", 2024-04-20T01:47:39.500 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.8785999999999998e-05 2024-04-20T01:47:39.500 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.500 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.500 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:39.500 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.797387+0000", 2024-04-20T01:47:39.500 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.5250000000000002e-06 2024-04-20T01:47:39.500 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.500 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.500 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:39.501 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.799659+0000", 2024-04-20T01:47:39.501 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0022715069999999999 2024-04-20T01:47:39.501 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.501 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.501 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:39.501 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.799686+0000", 2024-04-20T01:47:39.501 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6696e-05 2024-04-20T01:47:39.501 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.501 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.501 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.501 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.501 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.501 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 37/30 ECSubWrite(tid=45, reqid=client.4643.0:242, at_version=0'0, trim_to=0'0, roll_forward_to=37'32))", 2024-04-20T01:47:39.501 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.801620+0000", 2024-04-20T01:47:39.501 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.63389442200000001, 2024-04-20T01:47:39.502 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00066492000000000005, 2024-04-20T01:47:39.502 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:39.502 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:39.502 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:39.502 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.502 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:39.502 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.801620+0000", 2024-04-20T01:47:39.502 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.502 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.502 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.502 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:39.502 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.801620+0000", 2024-04-20T01:47:39.502 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.503 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.503 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.503 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:39.503 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.801620+0000", 2024-04-20T01:47:39.503 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:39.503 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.503 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.503 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:39.503 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.801632+0000", 2024-04-20T01:47:39.503 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2605e-05 2024-04-20T01:47:39.503 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.503 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.503 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:39.503 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.801644+0000", 2024-04-20T01:47:39.504 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.226e-05 2024-04-20T01:47:39.504 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.504 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.504 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.504 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.801652+0000", 2024-04-20T01:47:39.504 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.1210000000000001e-06 2024-04-20T01:47:39.504 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.504 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.504 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:39.504 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.801701+0000", 2024-04-20T01:47:39.504 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.9168000000000001e-05 2024-04-20T01:47:39.504 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.504 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.504 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:39.505 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.801713+0000", 2024-04-20T01:47:39.505 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1895999999999999e-05 2024-04-20T01:47:39.505 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.505 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.505 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:39.505 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.802257+0000", 2024-04-20T01:47:39.505 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00054419400000000001 2024-04-20T01:47:39.505 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.505 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.505 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:39.505 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.802285+0000", 2024-04-20T01:47:39.505 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8379999999999999e-05 2024-04-20T01:47:39.505 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.505 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.506 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.506 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.506 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.506 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 37/30 ECSubWrite(tid=70, reqid=client.4643.0:227, at_version=0'0, trim_to=0'0, roll_forward_to=37'18))", 2024-04-20T01:47:39.506 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.802642+0000", 2024-04-20T01:47:39.506 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.63287290200000001, 2024-04-20T01:47:39.506 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00080944000000000003, 2024-04-20T01:47:39.506 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:39.506 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:39.506 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:39.506 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.506 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:39.506 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.802642+0000", 2024-04-20T01:47:39.506 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.507 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.507 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.507 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:39.507 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.802642+0000", 2024-04-20T01:47:39.507 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.507 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.507 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.507 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:39.507 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.802641+0000", 2024-04-20T01:47:39.507 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:39.507 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.507 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.507 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:39.507 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.802652+0000", 2024-04-20T01:47:39.507 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1555e-05 2024-04-20T01:47:39.508 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.508 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.508 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:39.508 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.802664+0000", 2024-04-20T01:47:39.508 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1538e-05 2024-04-20T01:47:39.508 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.508 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.508 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.508 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.802671+0000", 2024-04-20T01:47:39.508 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0119999999999999e-06 2024-04-20T01:47:39.508 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.508 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.508 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:39.508 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.802723+0000", 2024-04-20T01:47:39.508 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.1999999999999997e-05 2024-04-20T01:47:39.509 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.509 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.509 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:39.509 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.802733+0000", 2024-04-20T01:47:39.509 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0288999999999999e-05 2024-04-20T01:47:39.509 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.509 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.509 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:39.509 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.803429+0000", 2024-04-20T01:47:39.509 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00069544799999999999 2024-04-20T01:47:39.509 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.509 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.509 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:39.509 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.803451+0000", 2024-04-20T01:47:39.510 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2493e-05 2024-04-20T01:47:39.510 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.510 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.510 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.510 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.510 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.510 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s3 37/30 ECSubWrite(tid=21, reqid=client.4643.0:236, at_version=0'0, trim_to=0'0, roll_forward_to=37'10))", 2024-04-20T01:47:39.510 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.807021+0000", 2024-04-20T01:47:39.510 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.62849369600000005, 2024-04-20T01:47:39.510 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00088964100000000004, 2024-04-20T01:47:39.510 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:39.510 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:39.510 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:39.511 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.511 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:39.511 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807021+0000", 2024-04-20T01:47:39.511 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.511 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.511 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.511 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:39.511 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807021+0000", 2024-04-20T01:47:39.511 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:39.511 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.511 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.511 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:39.511 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807019+0000", 2024-04-20T01:47:39.511 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999976 2024-04-20T01:47:39.512 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.512 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.512 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:39.512 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807058+0000", 2024-04-20T01:47:39.512 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.9619999999999997e-05 2024-04-20T01:47:39.512 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.512 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.512 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:39.512 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807096+0000", 2024-04-20T01:47:39.512 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7635999999999997e-05 2024-04-20T01:47:39.512 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.512 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.512 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:39.512 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807121+0000", 2024-04-20T01:47:39.512 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4562999999999998e-05 2024-04-20T01:47:39.513 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.513 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.513 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:39.513 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807213+0000", 2024-04-20T01:47:39.513 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2621000000000006e-05 2024-04-20T01:47:39.513 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.513 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.513 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:39.513 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807222+0000", 2024-04-20T01:47:39.513 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.541e-06 2024-04-20T01:47:39.513 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.513 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.513 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:39.513 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807876+0000", 2024-04-20T01:47:39.514 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00065372200000000003 2024-04-20T01:47:39.514 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:39.514 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:39.514 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:39.514 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807911+0000", 2024-04-20T01:47:39.514 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.5085e-05 2024-04-20T01:47:39.514 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.514 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.514 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.514 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:39.514 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:39.514 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:39.516 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-20T01:47:39.516 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:39.530+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:39.616 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:39.631+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:39.685 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:39.685 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:47:39.685 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 0 2024-04-20T01:47:39.686 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:39.696 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-20T01:47:39.718 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:39.732+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:39.770 INFO:tasks.rados.rados.0.smithi165.stdout:91: done (0 left) 2024-04-20T01:47:39.771 INFO:tasks.rados.rados.0.smithi165.stdout:92: copy_from oid 35 from oid 33 current snap is 3 2024-04-20T01:47:39.771 INFO:tasks.rados.rados.0.smithi165.stdout:93: read oid 12 snap -1 2024-04-20T01:47:39.771 INFO:tasks.rados.rados.0.smithi165.stdout:93: expect (ObjNum 11 snap 0 seq_num 11) 2024-04-20T01:47:39.771 INFO:tasks.rados.rados.0.smithi165.stdout:94: delete oid 20 current snap is 3 2024-04-20T01:47:39.779 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.5 to 1 (10000) 2024-04-20T01:47:39.781 INFO:tasks.rados.rados.0.smithi165.stdout:94: done (2 left) 2024-04-20T01:47:39.781 INFO:tasks.rados.rados.0.smithi165.stdout:95: read oid 38 snap 0 2024-04-20T01:47:39.781 INFO:tasks.rados.rados.0.smithi165.stdout:95: expect (ObjNum 37 snap 0 seq_num 37) 2024-04-20T01:47:39.781 INFO:tasks.rados.rados.0.smithi165.stdout:96: setattr oid 44 current snap is 3 2024-04-20T01:47:39.782 INFO:tasks.rados.rados.0.smithi165.stdout:93: expect (ObjNum 11 snap 0 seq_num 11) 2024-04-20T01:47:39.788 INFO:tasks.rados.rados.0.smithi165.stdout:93: done (3 left) 2024-04-20T01:47:39.788 INFO:tasks.rados.rados.0.smithi165.stdout:97: snap_create 2024-04-20T01:47:39.791 INFO:tasks.rados.rados.0.smithi165.stdout:92: finishing copy_from racing read to smithi16579500-35 2024-04-20T01:47:39.793 INFO:tasks.rados.rados.0.smithi165.stdout:95: expect (ObjNum 37 snap 0 seq_num 37) 2024-04-20T01:47:39.795 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-20T01:47:39.809 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 44 v 10 (ObjNum 57 snap 2 seq_num 57) dirty exists 2024-04-20T01:47:39.810 INFO:tasks.rados.rados.0.smithi165.stdout:92: finishing copy_from to smithi16579500-35 2024-04-20T01:47:39.810 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 35 v 10 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-04-20T01:47:39.819 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:39.833+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:39.832 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:39.832 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:47:39.832 INFO:teuthology.orchestra.run.smithi121.stdout: "complaint_time": 30, 2024-04-20T01:47:39.832 INFO:teuthology.orchestra.run.smithi121.stdout: "num_blocked_ops": 0 2024-04-20T01:47:39.832 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:39.842 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-20T01:47:39.927 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:39.942+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:47:40.013 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:40.013 INFO:teuthology.orchestra.run.smithi121.stdout: "size": 20, 2024-04-20T01:47:40.013 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 600, 2024-04-20T01:47:40.013 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-04-20T01:47:40.013 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.013 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.1s3 36/30 ECSubWrite(tid=55, reqid=osd.4.0:55, at_version=36'8, trim_to=0'0, roll_forward_to=35'6))", 2024-04-20T01:47:40.013 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:37.767370+0000", 2024-04-20T01:47:40.013 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 2.2592432489999998, 2024-04-20T01:47:40.013 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0022326350000000002, 2024-04-20T01:47:40.014 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:40.014 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:40.014 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:40.014 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.014 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:40.014 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.767370+0000", 2024-04-20T01:47:40.014 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.014 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.014 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.014 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:40.015 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.767370+0000", 2024-04-20T01:47:40.015 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.015 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.015 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.015 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:40.015 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.767370+0000", 2024-04-20T01:47:40.015 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:47:40.015 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.015 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.015 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:40.015 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.767397+0000", 2024-04-20T01:47:40.015 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7124999999999999e-05 2024-04-20T01:47:40.015 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.015 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.016 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:40.016 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.767410+0000", 2024-04-20T01:47:40.016 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.328e-05 2024-04-20T01:47:40.016 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.016 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.016 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:40.016 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.767420+0000", 2024-04-20T01:47:40.016 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7869999999999996e-06 2024-04-20T01:47:40.016 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.016 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.016 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:40.016 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.767489+0000", 2024-04-20T01:47:40.016 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.8652000000000006e-05 2024-04-20T01:47:40.016 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.016 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.017 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:40.017 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.767501+0000", 2024-04-20T01:47:40.017 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2294e-05 2024-04-20T01:47:40.017 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.017 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.017 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:40.017 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.769565+0000", 2024-04-20T01:47:40.017 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.002064396 2024-04-20T01:47:40.017 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.017 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.017 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:40.017 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.769603+0000", 2024-04-20T01:47:40.017 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7311999999999998e-05 2024-04-20T01:47:40.017 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.018 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.018 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.018 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.018 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.018 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.1s3 36/30 ECSubWrite(tid=56, reqid=osd.4.0:55, at_version=0'0, trim_to=0'0, roll_forward_to=36'8))", 2024-04-20T01:47:40.018 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:37.770498+0000", 2024-04-20T01:47:40.018 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 2.256115195, 2024-04-20T01:47:40.018 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00089571300000000002, 2024-04-20T01:47:40.018 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:40.018 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:40.018 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:40.018 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.018 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:40.019 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.770498+0000", 2024-04-20T01:47:40.019 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.019 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.019 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.019 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:40.019 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.770498+0000", 2024-04-20T01:47:40.019 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.019 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.019 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.019 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:40.019 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.770497+0000", 2024-04-20T01:47:40.019 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:40.019 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.019 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.019 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:40.020 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.770510+0000", 2024-04-20T01:47:40.020 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2999e-05 2024-04-20T01:47:40.020 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.020 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.020 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:40.020 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.770524+0000", 2024-04-20T01:47:40.020 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4002000000000001e-05 2024-04-20T01:47:40.020 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.020 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.020 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:40.020 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.770535+0000", 2024-04-20T01:47:40.020 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0732e-05 2024-04-20T01:47:40.020 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.021 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.021 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:40.021 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.770606+0000", 2024-04-20T01:47:40.021 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.1048000000000001e-05 2024-04-20T01:47:40.021 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.021 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.021 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:40.021 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.770616+0000", 2024-04-20T01:47:40.021 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6269999999999995e-06 2024-04-20T01:47:40.021 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.021 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.021 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:40.021 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.771373+0000", 2024-04-20T01:47:40.022 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00075714399999999996 2024-04-20T01:47:40.022 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.022 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.022 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:40.022 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.771394+0000", 2024-04-20T01:47:40.022 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0911e-05 2024-04-20T01:47:40.022 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.022 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.022 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.022 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.022 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.022 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s5 36/30 ECSubWrite(tid=57, reqid=client.4643.0:210, at_version=36'7, trim_to=0'0, roll_forward_to=33'5))", 2024-04-20T01:47:40.022 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:37.778503+0000", 2024-04-20T01:47:40.023 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 2.2481100719999998, 2024-04-20T01:47:40.023 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00203418, 2024-04-20T01:47:40.023 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:40.023 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:40.023 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:40.023 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.023 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:40.023 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.778503+0000", 2024-04-20T01:47:40.023 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.023 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.023 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.023 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:40.024 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.778503+0000", 2024-04-20T01:47:40.024 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.024 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.024 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.024 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:40.024 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.778503+0000", 2024-04-20T01:47:40.024 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:40.024 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.024 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.024 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:40.024 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.778724+0000", 2024-04-20T01:47:40.024 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00022111599999999999 2024-04-20T01:47:40.024 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.024 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.025 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:40.025 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.778737+0000", 2024-04-20T01:47:40.025 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3373000000000001e-05 2024-04-20T01:47:40.025 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.025 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.025 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:40.025 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.778746+0000", 2024-04-20T01:47:40.025 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.6000000000000007e-06 2024-04-20T01:47:40.025 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.025 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.025 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:40.025 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.778804+0000", 2024-04-20T01:47:40.025 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.8786999999999998e-05 2024-04-20T01:47:40.025 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.026 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.026 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:40.026 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.778815+0000", 2024-04-20T01:47:40.026 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0985000000000001e-05 2024-04-20T01:47:40.026 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.026 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.026 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:40.026 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.780513+0000", 2024-04-20T01:47:40.026 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0016973660000000001 2024-04-20T01:47:40.026 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.026 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.026 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:40.026 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.780538+0000", 2024-04-20T01:47:40.027 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4752e-05 2024-04-20T01:47:40.027 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.027 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.027 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.027 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.027 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.027 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s5 36/30 ECSubWrite(tid=58, reqid=client.4643.0:211, at_version=36'8, trim_to=0'0, roll_forward_to=33'5))", 2024-04-20T01:47:40.027 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:37.785593+0000", 2024-04-20T01:47:40.027 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 2.2410207400000002, 2024-04-20T01:47:40.027 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001615411, 2024-04-20T01:47:40.027 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:40.027 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:40.027 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:40.027 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.028 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:40.028 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785593+0000", 2024-04-20T01:47:40.028 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.028 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.028 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.028 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:40.028 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785593+0000", 2024-04-20T01:47:40.028 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.028 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.028 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.028 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:40.028 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785592+0000", 2024-04-20T01:47:40.029 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:40.029 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.029 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.029 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:40.029 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785697+0000", 2024-04-20T01:47:40.029 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000104647 2024-04-20T01:47:40.029 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.029 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.029 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:40.029 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785709+0000", 2024-04-20T01:47:40.029 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2918999999999999e-05 2024-04-20T01:47:40.029 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.029 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.029 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:40.030 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785719+0000", 2024-04-20T01:47:40.030 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2669999999999995e-06 2024-04-20T01:47:40.030 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.030 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.030 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:40.030 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785766+0000", 2024-04-20T01:47:40.030 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.7398000000000003e-05 2024-04-20T01:47:40.030 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.030 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.030 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:40.030 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785776+0000", 2024-04-20T01:47:40.030 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6539999999999994e-06 2024-04-20T01:47:40.030 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.031 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.031 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:40.031 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.787186+0000", 2024-04-20T01:47:40.031 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00141015 2024-04-20T01:47:40.031 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.031 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.031 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:40.031 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.787208+0000", 2024-04-20T01:47:40.031 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2149000000000001e-05 2024-04-20T01:47:40.031 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.031 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.031 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.031 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.031 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.032 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s5 36/30 ECSubWrite(tid=60, reqid=client.4643.0:212, at_version=36'9, trim_to=0'0, roll_forward_to=33'5))", 2024-04-20T01:47:40.032 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:37.785731+0000", 2024-04-20T01:47:40.032 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 2.2408826670000002, 2024-04-20T01:47:40.032 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001774657, 2024-04-20T01:47:40.032 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:40.032 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:40.032 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:40.032 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.032 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:40.032 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785731+0000", 2024-04-20T01:47:40.032 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.032 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.032 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.032 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:40.033 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785731+0000", 2024-04-20T01:47:40.033 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.033 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.033 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.033 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:40.033 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785730+0000", 2024-04-20T01:47:40.033 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:40.033 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.033 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.033 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:40.033 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785746+0000", 2024-04-20T01:47:40.033 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5508000000000001e-05 2024-04-20T01:47:40.033 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.033 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.034 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:40.034 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785754+0000", 2024-04-20T01:47:40.034 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.8990000000000001e-06 2024-04-20T01:47:40.034 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.034 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.034 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:40.034 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.785761+0000", 2024-04-20T01:47:40.034 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.8510000000000001e-06 2024-04-20T01:47:40.034 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.034 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.034 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:40.034 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.786208+0000", 2024-04-20T01:47:40.034 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00044790699999999999 2024-04-20T01:47:40.035 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.035 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.035 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:40.035 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.786220+0000", 2024-04-20T01:47:40.035 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1737e-05 2024-04-20T01:47:40.035 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.035 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.035 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:40.035 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.787480+0000", 2024-04-20T01:47:40.035 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001259706 2024-04-20T01:47:40.035 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.035 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.035 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:40.035 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.787505+0000", 2024-04-20T01:47:40.036 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5525999999999999e-05 2024-04-20T01:47:40.036 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.036 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.036 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.036 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.036 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.036 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s5 36/30 ECSubWrite(tid=61, reqid=client.4643.0:212, at_version=0'0, trim_to=0'0, roll_forward_to=36'9))", 2024-04-20T01:47:40.036 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:37.792899+0000", 2024-04-20T01:47:40.036 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 2.233714687, 2024-04-20T01:47:40.036 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00059159000000000002, 2024-04-20T01:47:40.036 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:40.036 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:40.036 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:40.036 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.037 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:40.037 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.792899+0000", 2024-04-20T01:47:40.037 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.037 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.037 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.037 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:40.037 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.792899+0000", 2024-04-20T01:47:40.037 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.037 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.037 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.037 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:40.037 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.792898+0000", 2024-04-20T01:47:40.037 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:40.037 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.038 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.038 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:40.038 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.792909+0000", 2024-04-20T01:47:40.038 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0745e-05 2024-04-20T01:47:40.038 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.038 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.038 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:40.038 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.792921+0000", 2024-04-20T01:47:40.038 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.243e-05 2024-04-20T01:47:40.038 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.038 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.038 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:40.038 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.792928+0000", 2024-04-20T01:47:40.038 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.9619999999999996e-06 2024-04-20T01:47:40.039 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.039 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.039 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:40.039 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.792974+0000", 2024-04-20T01:47:40.039 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.6045999999999999e-05 2024-04-20T01:47:40.039 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.039 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.039 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:40.039 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.792981+0000", 2024-04-20T01:47:40.039 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.1579999999999999e-06 2024-04-20T01:47:40.039 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.039 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.039 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:40.039 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.793465+0000", 2024-04-20T01:47:40.040 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00048349800000000001 2024-04-20T01:47:40.040 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.040 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.040 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:40.040 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.793490+0000", 2024-04-20T01:47:40.040 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5542e-05 2024-04-20T01:47:40.040 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.040 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.040 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.040 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.040 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.040 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.cs6 36/30 ECSubWrite(tid=19, reqid=client.4643.0:218, at_version=36'13, trim_to=0'0, roll_forward_to=34'11))", 2024-04-20T01:47:40.040 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:37.798903+0000", 2024-04-20T01:47:40.040 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 2.2277107919999999, 2024-04-20T01:47:40.041 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00084568200000000005, 2024-04-20T01:47:40.041 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:40.041 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:40.041 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:40.041 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.041 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:40.041 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.798903+0000", 2024-04-20T01:47:40.041 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.041 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.041 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.041 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:40.041 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.798903+0000", 2024-04-20T01:47:40.041 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.041 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.041 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.042 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:40.042 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.798902+0000", 2024-04-20T01:47:40.042 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:40.042 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.042 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.042 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:40.042 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.798921+0000", 2024-04-20T01:47:40.042 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8357999999999999e-05 2024-04-20T01:47:40.042 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.042 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.042 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:40.042 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.798930+0000", 2024-04-20T01:47:40.042 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.5410000000000006e-06 2024-04-20T01:47:40.043 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.043 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.043 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:40.043 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.798944+0000", 2024-04-20T01:47:40.043 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3772e-05 2024-04-20T01:47:40.043 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.043 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.043 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:40.043 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.798988+0000", 2024-04-20T01:47:40.043 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.3501000000000002e-05 2024-04-20T01:47:40.043 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.043 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.043 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:40.044 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.798994+0000", 2024-04-20T01:47:40.044 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.5679999999999996e-06 2024-04-20T01:47:40.044 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.044 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.044 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:40.044 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.799725+0000", 2024-04-20T01:47:40.044 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00073067499999999997 2024-04-20T01:47:40.044 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.044 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.044 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:40.044 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.799748+0000", 2024-04-20T01:47:40.044 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3527e-05 2024-04-20T01:47:40.044 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.044 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.045 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.045 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.045 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.045 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.cs6 36/30 ECSubWrite(tid=21, reqid=client.4643.0:220, at_version=36'15, trim_to=0'0, roll_forward_to=34'11))", 2024-04-20T01:47:40.045 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:37.800863+0000", 2024-04-20T01:47:40.045 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 2.2257500220000002, 2024-04-20T01:47:40.045 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0013179960000000001, 2024-04-20T01:47:40.045 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:40.045 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:40.045 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:40.045 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.046 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:40.046 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.800863+0000", 2024-04-20T01:47:40.046 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.046 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.046 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.046 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:40.046 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.800863+0000", 2024-04-20T01:47:40.046 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.046 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.046 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.046 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:40.046 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.800863+0000", 2024-04-20T01:47:40.046 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:40.047 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.047 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.047 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:40.047 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.800878+0000", 2024-04-20T01:47:40.047 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5356e-05 2024-04-20T01:47:40.047 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.047 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.047 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:40.047 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.800897+0000", 2024-04-20T01:47:40.047 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8720999999999999e-05 2024-04-20T01:47:40.047 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.047 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.047 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:40.047 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.800904+0000", 2024-04-20T01:47:40.048 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.2579999999999998e-06 2024-04-20T01:47:40.048 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.048 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.048 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:40.048 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.800956+0000", 2024-04-20T01:47:40.048 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.2037999999999999e-05 2024-04-20T01:47:40.048 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.048 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.048 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:40.048 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.800965+0000", 2024-04-20T01:47:40.048 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2820000000000002e-06 2024-04-20T01:47:40.048 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.048 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.048 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:40.049 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.802163+0000", 2024-04-20T01:47:40.049 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0011972910000000001 2024-04-20T01:47:40.049 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.049 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.049 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:40.049 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.802181+0000", 2024-04-20T01:47:40.049 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8705000000000001e-05 2024-04-20T01:47:40.049 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.049 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.049 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.049 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.049 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.049 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.cs6 36/30 ECSubWrite(tid=22, reqid=client.4643.0:220, at_version=0'0, trim_to=0'0, roll_forward_to=36'15))", 2024-04-20T01:47:40.049 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:37.807056+0000", 2024-04-20T01:47:40.049 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 2.2195570739999999, 2024-04-20T01:47:40.050 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00063461099999999999, 2024-04-20T01:47:40.050 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:40.050 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:40.050 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:40.050 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.050 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:40.050 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.807056+0000", 2024-04-20T01:47:40.050 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.050 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.050 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.050 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:40.050 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.807056+0000", 2024-04-20T01:47:40.051 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.051 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.051 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.051 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:40.051 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.807056+0000", 2024-04-20T01:47:40.051 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:40.051 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.051 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.051 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:40.051 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.807073+0000", 2024-04-20T01:47:40.051 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6840000000000001e-05 2024-04-20T01:47:40.051 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.051 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.051 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:40.052 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.807082+0000", 2024-04-20T01:47:40.052 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.5910000000000002e-06 2024-04-20T01:47:40.052 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.052 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.052 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:40.052 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.807089+0000", 2024-04-20T01:47:40.052 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.4749999999999998e-06 2024-04-20T01:47:40.052 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.052 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.052 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:40.052 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.807126+0000", 2024-04-20T01:47:40.052 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7264000000000001e-05 2024-04-20T01:47:40.052 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.052 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.052 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:40.053 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.807132+0000", 2024-04-20T01:47:40.053 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.9689999999999999e-06 2024-04-20T01:47:40.053 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.053 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.053 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:40.053 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.807648+0000", 2024-04-20T01:47:40.053 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00051609500000000005 2024-04-20T01:47:40.053 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.053 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.053 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:40.053 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.807691+0000", 2024-04-20T01:47:40.053 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2914999999999998e-05 2024-04-20T01:47:40.053 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.053 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.053 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.054 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.054 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.054 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s4 37/30 ECSubWrite(tid=18, reqid=client.4643.0:234, at_version=37'8, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:40.054 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.792501+0000", 2024-04-20T01:47:40.054 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 1.2341123730000001, 2024-04-20T01:47:40.054 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0026505190000000001, 2024-04-20T01:47:40.054 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:40.054 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:40.054 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:40.054 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.054 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:40.055 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792501+0000", 2024-04-20T01:47:40.055 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.055 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.055 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.055 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:40.055 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792501+0000", 2024-04-20T01:47:40.055 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.055 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.055 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.055 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:40.056 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792501+0000", 2024-04-20T01:47:40.056 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:40.056 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.056 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.056 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:40.056 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792533+0000", 2024-04-20T01:47:40.056 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.1998000000000001e-05 2024-04-20T01:47:40.056 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.056 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.056 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:40.056 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792547+0000", 2024-04-20T01:47:40.056 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4681999999999999e-05 2024-04-20T01:47:40.056 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.056 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.057 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:40.057 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792559+0000", 2024-04-20T01:47:40.057 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1413000000000001e-05 2024-04-20T01:47:40.057 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.057 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.057 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:40.057 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792664+0000", 2024-04-20T01:47:40.057 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000104929 2024-04-20T01:47:40.057 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.057 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.057 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:40.057 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.792675+0000", 2024-04-20T01:47:40.057 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1097000000000001e-05 2024-04-20T01:47:40.057 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.058 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.058 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:40.058 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795116+0000", 2024-04-20T01:47:40.058 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0024408400000000001 2024-04-20T01:47:40.058 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.058 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.058 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:40.058 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795152+0000", 2024-04-20T01:47:40.058 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.5862000000000003e-05 2024-04-20T01:47:40.058 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.058 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.058 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.058 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.058 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.059 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s4 37/30 ECSubWrite(tid=19, reqid=client.4643.0:235, at_version=37'9, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:40.059 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.794584+0000", 2024-04-20T01:47:40.059 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 1.2320297650000001, 2024-04-20T01:47:40.059 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0023436580000000002, 2024-04-20T01:47:40.059 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:40.059 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:40.059 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:40.059 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.059 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:40.059 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.794584+0000", 2024-04-20T01:47:40.059 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.059 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.059 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.059 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:40.060 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.794584+0000", 2024-04-20T01:47:40.060 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.060 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.060 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.060 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:40.060 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.794583+0000", 2024-04-20T01:47:40.060 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:40.060 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.060 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.060 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:40.060 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.794610+0000", 2024-04-20T01:47:40.060 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7949e-05 2024-04-20T01:47:40.060 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.060 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.061 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:40.061 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.794628+0000", 2024-04-20T01:47:40.061 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7994000000000001e-05 2024-04-20T01:47:40.061 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.061 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.061 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:40.061 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.794641+0000", 2024-04-20T01:47:40.061 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.234e-05 2024-04-20T01:47:40.061 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.061 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.061 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:40.061 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.794726+0000", 2024-04-20T01:47:40.061 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4979000000000001e-05 2024-04-20T01:47:40.061 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.062 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.062 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:40.062 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.794737+0000", 2024-04-20T01:47:40.062 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1408e-05 2024-04-20T01:47:40.062 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.062 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.062 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:40.062 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796899+0000", 2024-04-20T01:47:40.062 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0021617939999999999 2024-04-20T01:47:40.062 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.062 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.062 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:40.062 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796927+0000", 2024-04-20T01:47:40.062 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8319000000000001e-05 2024-04-20T01:47:40.063 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.063 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.063 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.063 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.063 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.063 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s4 37/30 ECSubWrite(tid=20, reqid=client.4643.0:236, at_version=37'10, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:40.063 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.795440+0000", 2024-04-20T01:47:40.063 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 1.2311735269999999, 2024-04-20T01:47:40.063 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001500384, 2024-04-20T01:47:40.063 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:40.063 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:40.063 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:40.063 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.063 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:40.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795440+0000", 2024-04-20T01:47:40.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:40.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795440+0000", 2024-04-20T01:47:40.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:40.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795439+0000", 2024-04-20T01:47:40.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:40.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:40.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795460+0000", 2024-04-20T01:47:40.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1209999999999999e-05 2024-04-20T01:47:40.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:40.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795473+0000", 2024-04-20T01:47:40.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2272999999999999e-05 2024-04-20T01:47:40.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:40.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795484+0000", 2024-04-20T01:47:40.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1333e-05 2024-04-20T01:47:40.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.066 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.066 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:40.066 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795651+0000", 2024-04-20T01:47:40.066 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00016733899999999999 2024-04-20T01:47:40.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.066 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.066 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:40.066 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795661+0000", 2024-04-20T01:47:40.066 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.8050000000000001e-06 2024-04-20T01:47:40.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.066 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.066 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:40.067 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796932+0000", 2024-04-20T01:47:40.067 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0012712260000000001 2024-04-20T01:47:40.067 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.067 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.067 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:40.067 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796940+0000", 2024-04-20T01:47:40.067 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.9680000000000006e-06 2024-04-20T01:47:40.067 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.067 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.067 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.067 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.067 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.067 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.3s5 37/30 ECSubWrite(tid=202, reqid=client.4643.0:232, at_version=37'10, trim_to=0'0, roll_forward_to=33'8))", 2024-04-20T01:47:40.067 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.797727+0000", 2024-04-20T01:47:40.067 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 1.22888657, 2024-04-20T01:47:40.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0019542029999999998, 2024-04-20T01:47:40.068 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:40.068 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:40.068 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:40.068 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.068 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:40.068 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.797727+0000", 2024-04-20T01:47:40.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.068 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.068 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:40.068 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.797727+0000", 2024-04-20T01:47:40.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.068 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:40.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.797726+0000", 2024-04-20T01:47:40.069 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:40.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.069 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:40.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.797904+0000", 2024-04-20T01:47:40.069 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00017812200000000001 2024-04-20T01:47:40.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.069 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:40.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.797923+0000", 2024-04-20T01:47:40.069 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8703999999999999e-05 2024-04-20T01:47:40.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:40.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.797940+0000", 2024-04-20T01:47:40.070 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7007000000000002e-05 2024-04-20T01:47:40.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:40.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.798003+0000", 2024-04-20T01:47:40.070 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.3204e-05 2024-04-20T01:47:40.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:40.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.798015+0000", 2024-04-20T01:47:40.071 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2543e-05 2024-04-20T01:47:40.071 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.071 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.071 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:40.071 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.799653+0000", 2024-04-20T01:47:40.071 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0016375319999999999 2024-04-20T01:47:40.071 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.071 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.071 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:40.071 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.799681+0000", 2024-04-20T01:47:40.071 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8235999999999999e-05 2024-04-20T01:47:40.071 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.071 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.072 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.072 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.072 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.072 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s4 37/30 ECSubWrite(tid=21, reqid=client.4643.0:236, at_version=0'0, trim_to=0'0, roll_forward_to=37'10))", 2024-04-20T01:47:40.072 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.807077+0000", 2024-04-20T01:47:40.072 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 1.219536194, 2024-04-20T01:47:40.072 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00071157000000000002, 2024-04-20T01:47:40.072 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:40.072 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:40.072 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:40.072 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.072 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:40.072 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807077+0000", 2024-04-20T01:47:40.072 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.073 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.073 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:40.073 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807077+0000", 2024-04-20T01:47:40.073 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.073 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.073 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:40.073 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807075+0000", 2024-04-20T01:47:40.073 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999981 2024-04-20T01:47:40.073 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.073 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:40.073 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807102+0000", 2024-04-20T01:47:40.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6906999999999998e-05 2024-04-20T01:47:40.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:40.074 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807130+0000", 2024-04-20T01:47:40.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.758e-05 2024-04-20T01:47:40.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:40.074 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807150+0000", 2024-04-20T01:47:40.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0373e-05 2024-04-20T01:47:40.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:40.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807224+0000", 2024-04-20T01:47:40.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.4065000000000003e-05 2024-04-20T01:47:40.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.075 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:40.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807231+0000", 2024-04-20T01:47:40.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0249999999999997e-06 2024-04-20T01:47:40.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.075 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:40.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807762+0000", 2024-04-20T01:47:40.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00053059099999999996 2024-04-20T01:47:40.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.076 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.076 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:40.076 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807789+0000", 2024-04-20T01:47:40.076 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6863e-05 2024-04-20T01:47:40.076 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.076 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.076 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.076 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.076 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.076 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.3s5 37/30 ECSubWrite(tid=203, reqid=client.4643.0:232, at_version=0'0, trim_to=0'0, roll_forward_to=37'10))", 2024-04-20T01:47:40.076 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.807520+0000", 2024-04-20T01:47:40.076 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 1.2190935300000001, 2024-04-20T01:47:40.076 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00061929800000000001, 2024-04-20T01:47:40.076 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:40.077 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:40.077 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:40.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.077 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:40.077 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807520+0000", 2024-04-20T01:47:40.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.077 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:40.077 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807520+0000", 2024-04-20T01:47:40.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.077 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:40.078 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807519+0000", 2024-04-20T01:47:40.078 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:40.078 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.078 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.078 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:40.078 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807537+0000", 2024-04-20T01:47:40.078 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8096000000000001e-05 2024-04-20T01:47:40.078 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.078 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.078 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:40.078 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807549+0000", 2024-04-20T01:47:40.078 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2417e-05 2024-04-20T01:47:40.078 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.079 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:40.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807558+0000", 2024-04-20T01:47:40.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.789e-06 2024-04-20T01:47:40.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.079 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:40.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807604+0000", 2024-04-20T01:47:40.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5980999999999999e-05 2024-04-20T01:47:40.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.079 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:40.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807615+0000", 2024-04-20T01:47:40.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0769e-05 2024-04-20T01:47:40.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.080 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:40.080 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.808113+0000", 2024-04-20T01:47:40.080 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000498085 2024-04-20T01:47:40.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.080 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:40.080 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.808139+0000", 2024-04-20T01:47:40.080 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.614e-05 2024-04-20T01:47:40.080 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.080 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.080 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.081 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s4 38/30 ECSubWrite(tid=22, reqid=osd.3.0:22, at_version=38'12, trim_to=0'0, roll_forward_to=37'10))", 2024-04-20T01:47:40.081 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:39.783947+0000", 2024-04-20T01:47:40.081 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.24266663199999999, 2024-04-20T01:47:40.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00091613199999999995, 2024-04-20T01:47:40.081 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:40.081 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:40.081 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:40.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.081 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:40.081 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.783947+0000", 2024-04-20T01:47:40.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.082 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:40.082 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.783947+0000", 2024-04-20T01:47:40.082 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.082 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.082 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:40.082 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.783947+0000", 2024-04-20T01:47:40.082 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:47:40.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.082 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.082 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:40.082 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.783967+0000", 2024-04-20T01:47:40.082 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0491000000000001e-05 2024-04-20T01:47:40.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.083 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:40.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.783978+0000", 2024-04-20T01:47:40.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1318000000000001e-05 2024-04-20T01:47:40.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.083 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:40.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.783989+0000", 2024-04-20T01:47:40.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0441999999999999e-05 2024-04-20T01:47:40.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.083 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:40.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.784047+0000", 2024-04-20T01:47:40.084 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.7958000000000003e-05 2024-04-20T01:47:40.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:40.084 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.784058+0000", 2024-04-20T01:47:40.084 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1226999999999999e-05 2024-04-20T01:47:40.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:40.084 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.784835+0000", 2024-04-20T01:47:40.084 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00077726200000000003 2024-04-20T01:47:40.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:40.085 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.784863+0000", 2024-04-20T01:47:40.085 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7563e-05 2024-04-20T01:47:40.085 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.085 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.085 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.085 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.085 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.3s5 38/30 ECSubWrite(tid=206, reqid=osd.13.0:206, at_version=38'12, trim_to=0'0, roll_forward_to=37'10))", 2024-04-20T01:47:40.085 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:39.800260+0000", 2024-04-20T01:47:40.085 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.22635374699999999, 2024-04-20T01:47:40.085 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001801001, 2024-04-20T01:47:40.085 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:40.085 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:40.086 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:40.086 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.086 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:40.086 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.800260+0000", 2024-04-20T01:47:40.086 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.086 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.086 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.086 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:40.086 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.800260+0000", 2024-04-20T01:47:40.086 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.086 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.086 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.086 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:40.087 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.800260+0000", 2024-04-20T01:47:40.087 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:47:40.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.087 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.087 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:40.087 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.800279+0000", 2024-04-20T01:47:40.087 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9663000000000001e-05 2024-04-20T01:47:40.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.087 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.087 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:40.087 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.800289+0000", 2024-04-20T01:47:40.087 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0309000000000001e-05 2024-04-20T01:47:40.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.087 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.088 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:40.088 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.800297+0000", 2024-04-20T01:47:40.088 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.3100000000000003e-06 2024-04-20T01:47:40.088 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.088 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.088 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:40.088 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.800360+0000", 2024-04-20T01:47:40.088 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.3638000000000003e-05 2024-04-20T01:47:40.088 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.088 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.088 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:40.088 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.800373+0000", 2024-04-20T01:47:40.088 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2294e-05 2024-04-20T01:47:40.088 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.089 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.089 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:40.089 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.802036+0000", 2024-04-20T01:47:40.089 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0016629570000000001 2024-04-20T01:47:40.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.089 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.089 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:40.089 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.802061+0000", 2024-04-20T01:47:40.089 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4981999999999999e-05 2024-04-20T01:47:40.089 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.089 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.089 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.090 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.090 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s5 38/30 ECSubWrite(tid=75, reqid=client.4643.0:254, at_version=38'10, trim_to=0'0, roll_forward_to=36'9))", 2024-04-20T01:47:40.090 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:39.805326+0000", 2024-04-20T01:47:40.090 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.22128703999999999, 2024-04-20T01:47:40.090 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0022504690000000002, 2024-04-20T01:47:40.090 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:40.090 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:40.090 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:40.090 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.090 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:40.090 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805326+0000", 2024-04-20T01:47:40.090 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.090 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.090 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.091 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:40.091 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805326+0000", 2024-04-20T01:47:40.091 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.091 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.091 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.091 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:40.091 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805326+0000", 2024-04-20T01:47:40.091 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:47:40.091 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.091 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.091 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:40.091 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805520+0000", 2024-04-20T01:47:40.091 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00019346599999999999 2024-04-20T01:47:40.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.092 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.092 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:40.092 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805531+0000", 2024-04-20T01:47:40.092 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1792e-05 2024-04-20T01:47:40.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.092 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.092 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:40.092 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805540+0000", 2024-04-20T01:47:40.092 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.7269999999999996e-06 2024-04-20T01:47:40.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.092 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.092 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:40.092 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805611+0000", 2024-04-20T01:47:40.092 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0531999999999995e-05 2024-04-20T01:47:40.093 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.093 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.093 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:40.093 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805627+0000", 2024-04-20T01:47:40.093 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6554000000000001e-05 2024-04-20T01:47:40.093 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.093 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.093 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:40.093 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.807554+0000", 2024-04-20T01:47:40.093 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0019263850000000001 2024-04-20T01:47:40.093 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.093 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.093 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:40.093 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.807577+0000", 2024-04-20T01:47:40.093 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3232999999999999e-05 2024-04-20T01:47:40.094 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.094 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.094 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.094 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.094 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.094 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.3s5 38/30 ECSubWrite(tid=209, reqid=osd.13.0:206, at_version=0'0, trim_to=0'0, roll_forward_to=38'12))", 2024-04-20T01:47:40.094 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:39.805486+0000", 2024-04-20T01:47:40.094 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.221127711, 2024-04-20T01:47:40.094 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00070395699999999996, 2024-04-20T01:47:40.094 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:40.094 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:40.094 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:40.094 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.094 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:40.095 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805486+0000", 2024-04-20T01:47:40.095 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.095 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.095 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.095 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:40.095 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805486+0000", 2024-04-20T01:47:40.095 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.095 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.095 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.095 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:40.095 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805485+0000", 2024-04-20T01:47:40.095 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:40.095 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.095 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.096 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:40.096 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805503+0000", 2024-04-20T01:47:40.096 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8128999999999999e-05 2024-04-20T01:47:40.096 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.096 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.096 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:40.096 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805519+0000", 2024-04-20T01:47:40.096 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5412000000000001e-05 2024-04-20T01:47:40.096 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.096 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.096 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:40.096 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805534+0000", 2024-04-20T01:47:40.096 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5557999999999999e-05 2024-04-20T01:47:40.096 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.097 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.097 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:40.097 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805606+0000", 2024-04-20T01:47:40.097 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.1833000000000002e-05 2024-04-20T01:47:40.097 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.097 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.097 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:40.097 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805616+0000", 2024-04-20T01:47:40.097 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0563e-05 2024-04-20T01:47:40.097 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.097 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.097 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:40.097 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.806166+0000", 2024-04-20T01:47:40.097 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00054968599999999997 2024-04-20T01:47:40.097 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.098 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.098 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:40.098 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.806190+0000", 2024-04-20T01:47:40.098 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3489000000000002e-05 2024-04-20T01:47:40.098 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.098 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.098 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.098 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.098 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.098 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s5 38/30 ECSubWrite(tid=76, reqid=client.4643.0:254, at_version=0'0, trim_to=0'0, roll_forward_to=38'10))", 2024-04-20T01:47:40.098 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:39.810956+0000", 2024-04-20T01:47:40.098 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.21565720499999999, 2024-04-20T01:47:40.098 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00062415500000000004, 2024-04-20T01:47:40.098 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:40.099 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:40.099 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:40.099 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.099 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:40.099 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.810956+0000", 2024-04-20T01:47:40.099 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.099 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.099 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.099 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:40.099 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.810956+0000", 2024-04-20T01:47:40.099 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:40.099 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.099 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.099 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:40.099 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.810955+0000", 2024-04-20T01:47:40.100 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:40.100 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.100 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.100 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:40.100 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.810967+0000", 2024-04-20T01:47:40.100 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1352000000000001e-05 2024-04-20T01:47:40.100 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.100 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.100 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:40.100 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.810976+0000", 2024-04-20T01:47:40.100 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7659999999999993e-06 2024-04-20T01:47:40.100 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.100 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.100 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:40.100 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.810995+0000", 2024-04-20T01:47:40.101 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8873e-05 2024-04-20T01:47:40.101 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.101 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.101 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:40.101 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.811046+0000", 2024-04-20T01:47:40.101 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.0268000000000001e-05 2024-04-20T01:47:40.101 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.101 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.101 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:40.101 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.811065+0000", 2024-04-20T01:47:40.101 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9831e-05 2024-04-20T01:47:40.101 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.101 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.101 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:40.102 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.811562+0000", 2024-04-20T01:47:40.102 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00049610699999999997 2024-04-20T01:47:40.102 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:40.102 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:40.102 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:40.102 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.811580+0000", 2024-04-20T01:47:40.102 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8788999999999999e-05 2024-04-20T01:47:40.102 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.102 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.102 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.102 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:40.102 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:40.102 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:40.106 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-20T01:47:40.107 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:47:40.120+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:47:40.108 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-20T01:47:40.207 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:40.222+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:40.268 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:47:40.277 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:40.277 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:47:40.277 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 0 2024-04-20T01:47:40.277 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:40.287 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-20T01:47:40.289 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:47:40.308 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:40.322+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:47:40.310 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:47:40.332 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:47:40.351 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:47:40.371 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:47:40.391 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:47:40.409 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:40.423+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:40.412 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:47:40.415 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:40.415 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:47:40.415 INFO:teuthology.orchestra.run.smithi121.stdout: "complaint_time": 30, 2024-04-20T01:47:40.415 INFO:teuthology.orchestra.run.smithi121.stdout: "num_blocked_ops": 0 2024-04-20T01:47:40.415 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:40.426 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-20T01:47:40.433 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:47:40.453 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:47:40.474 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:47:40.496 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:47:40.509 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:40.523+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:47:40.516 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:47:40.551 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:47:40.572 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:47:40.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:47:40.587+0000 7f1779c6f700 0 -- 172.21.15.44:0/2244031012 >> [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] conn(0x7f1774221430 msgr2=0x7f1774223820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-20T01:47:40.594 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:40.594 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:40.604 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-20T01:47:40.610 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:40.625+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:47:40.711 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:47:40.725+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:47:40.733 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:40.733 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:40.744 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-20T01:47:40.780 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.6 to 1 (10000) 2024-04-20T01:47:40.781 INFO:tasks.rados.rados.0.smithi165.stdout:92: done (3 left) 2024-04-20T01:47:40.781 INFO:tasks.rados.rados.0.smithi165.stdout:95: done (2 left) 2024-04-20T01:47:40.781 INFO:tasks.rados.rados.0.smithi165.stdout:96: done (1 left) 2024-04-20T01:47:40.781 INFO:tasks.rados.rados.0.smithi165.stdout:97: done (0 left) 2024-04-20T01:47:40.781 INFO:tasks.rados.rados.0.smithi165.stdout:98: read oid 8 snap -1 2024-04-20T01:47:40.781 INFO:tasks.rados.rados.0.smithi165.stdout:98: expect (ObjNum 7 snap 0 seq_num 7) 2024-04-20T01:47:40.781 INFO:tasks.rados.rados.0.smithi165.stdout:99: delete oid 35 current snap is 4 2024-04-20T01:47:40.786 INFO:tasks.rados.rados.0.smithi165.stdout:99: done (1 left) 2024-04-20T01:47:40.786 INFO:tasks.rados.rados.0.smithi165.stdout:100: rmattr oid 36 current snap is 4 2024-04-20T01:47:40.786 INFO:tasks.rados.rados.0.smithi165.stdout:100: done (1 left) 2024-04-20T01:47:40.786 INFO:tasks.rados.rados.0.smithi165.stdout:101: copy_from oid 2 from oid 28 current snap is 4 2024-04-20T01:47:40.786 INFO:tasks.rados.rados.0.smithi165.stdout:102: snap_remove snap 3 2024-04-20T01:47:40.790 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-20T01:47:40.798 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-20T01:47:40.803 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:47:40.805 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:47:40.805 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:47:40.805 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:47:40.805 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:47:40.805 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:47:40.805 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:47:40.805 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:47:40.805 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:47:40.805 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:47:40.805 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:47:40.806 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:47:40.806 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:47:40.806 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:47:40.806 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:47:40.806 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:47:40.806 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:47:40.812 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:40.827+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:47:40.877 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:40.877 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:40.887 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-20T01:47:40.914 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:40.927+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:41.014 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:47:41.028+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:47:41.058 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:41.058 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:41.068 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-20T01:47:41.116 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:41.130+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:41.217 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:41.217 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:41.217 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:41.231+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:47:41.228 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-20T01:47:41.318 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:41.332+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:41.356 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:41.356 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:41.366 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-20T01:47:41.419 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:41.433+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:41.493 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:41.493 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:41.503 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-20T01:47:41.519 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:41.534+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:41.620 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:41.634+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:41.631 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:41.632 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:41.642 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-20T01:47:41.721 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:47:41.735+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:47:41.785 INFO:tasks.rados.rados.0.smithi165.stdout:102: done (2 left) 2024-04-20T01:47:41.785 INFO:tasks.rados.rados.0.smithi165.stdout:103: read oid 12 snap 0 2024-04-20T01:47:41.785 INFO:tasks.rados.rados.0.smithi165.stdout:103: expect (ObjNum 11 snap 0 seq_num 11) 2024-04-20T01:47:41.786 INFO:tasks.rados.rados.0.smithi165.stdout:104: copy_from oid 26 from oid 7 current snap is 4 2024-04-20T01:47:41.786 INFO:tasks.rados.rados.0.smithi165.stdout:98: expect (ObjNum 7 snap 0 seq_num 7) 2024-04-20T01:47:41.786 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is set 2024-04-20T01:47:41.789 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.7 to 1 (10000) 2024-04-20T01:47:41.790 INFO:tasks.rados.rados.0.smithi165.stdout:101: finishing copy_from racing read to smithi16579500-2 2024-04-20T01:47:41.790 INFO:tasks.rados.rados.0.smithi165.stdout:101: finishing copy_from to smithi16579500-2 2024-04-20T01:47:41.790 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 2 v 11 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-04-20T01:47:41.790 INFO:tasks.rados.rados.0.smithi165.stdout:104: finishing copy_from to smithi16579500-26 2024-04-20T01:47:41.790 INFO:tasks.rados.rados.0.smithi165.stdout:104: got expected ENOENT (src dne) 2024-04-20T01:47:41.790 INFO:tasks.rados.rados.0.smithi165.stdout:104: finishing copy_from racing read to smithi16579500-26 2024-04-20T01:47:41.790 INFO:tasks.rados.rados.0.smithi165.stdout:98: done (3 left) 2024-04-20T01:47:41.790 INFO:tasks.rados.rados.0.smithi165.stdout:101: done (2 left) 2024-04-20T01:47:41.790 INFO:tasks.rados.rados.0.smithi165.stdout:104: done (1 left) 2024-04-20T01:47:41.790 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 40 current snap is 4 2024-04-20T01:47:41.790 INFO:tasks.rados.rados.0.smithi165.stdout:105: seq_num 63 ranges {671744=688128,1359872=376832} 2024-04-20T01:47:41.797 INFO:tasks.rados.rados.0.smithi165.stdout:105: writing smithi16579500-40 from 671744 to 1359872 tid 1 2024-04-20T01:47:41.800 INFO:tasks.rados.rados.0.smithi165.stdout:105: writing smithi16579500-40 from 1359872 to 1736704 tid 2 2024-04-20T01:47:41.800 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 45 current snap is 4 2024-04-20T01:47:41.800 INFO:tasks.rados.rados.0.smithi165.stdout:103: expect (ObjNum 11 snap 0 seq_num 11) 2024-04-20T01:47:41.806 INFO:tasks.rados.rados.0.smithi165.stdout:106: seq_num 64 ranges {1753088=704512,2457600=376832} 2024-04-20T01:47:41.809 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-04-20T01:47:41.816 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:41.816 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:41.819 INFO:tasks.rados.rados.0.smithi165.stdout:106: writing smithi16579500-45 from 1753088 to 2457600 tid 1 2024-04-20T01:47:41.821 INFO:tasks.rados.rados.0.smithi165.stdout:106: writing smithi16579500-45 from 2457600 to 2834432 tid 2 2024-04-20T01:47:41.822 INFO:tasks.rados.rados.0.smithi165.stdout:103: done (2 left) 2024-04-20T01:47:41.822 INFO:tasks.rados.rados.0.smithi165.stdout:107: rmattr oid 25 current snap is 4 2024-04-20T01:47:41.822 INFO:tasks.rados.rados.0.smithi165.stdout:107: done (2 left) 2024-04-20T01:47:41.822 INFO:tasks.rados.rados.0.smithi165.stdout:108: snap_remove snap 0 2024-04-20T01:47:41.822 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:41.836+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:41.827 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-20T01:47:41.923 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:41.936+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:41.955 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:41.955 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:41.965 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-20T01:47:42.023 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:47:42.038+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:47:42.124 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:42.138+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:42.134 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:42.134 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:42.144 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-20T01:47:42.225 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:42.239+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:47:42.313 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:42.313 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:42.323 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-20T01:47:42.326 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:42.340+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:42.426 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:42.441+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:47:42.450 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:42.450 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:42.460 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-20T01:47:42.527 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:42.541+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:42.588 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:42.588 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:42.599 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-20T01:47:42.628 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:42.643+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:42.729 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:47:42.743+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:47:42.766 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:42.767 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:42.777 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-20T01:47:42.786 INFO:tasks.rados.rados.0.smithi165.stdout:108: done (2 left) 2024-04-20T01:47:42.786 INFO:tasks.rados.rados.0.smithi165.stdout:109: read oid 25 snap -1 2024-04-20T01:47:42.786 INFO:tasks.rados.rados.0.smithi165.stdout:109: expect (ObjNum 24 snap 0 seq_num 24) 2024-04-20T01:47:42.787 INFO:tasks.rados.rados.0.smithi165.stdout:110: setattr oid 18 current snap is 4 2024-04-20T01:47:42.787 INFO:tasks.rados.rados.0.smithi165.stdout:105: finishing write tid 1 to smithi16579500-40 2024-04-20T01:47:42.787 INFO:tasks.rados.rados.0.smithi165.stdout:105: finishing write tid 2 to smithi16579500-40 2024-04-20T01:47:42.787 INFO:tasks.rados.rados.0.smithi165.stdout:105: finishing write tid 3 to smithi16579500-40 2024-04-20T01:47:42.787 INFO:tasks.rados.rados.0.smithi165.stdout:105: finishing write tid 4 to smithi16579500-40 2024-04-20T01:47:42.787 INFO:tasks.rados.rados.0.smithi165.stdout:105: oid 40 updating version 0 to 4 2024-04-20T01:47:42.787 INFO:tasks.rados.rados.0.smithi165.stdout:105: oid 40 updating version 4 to 5 2024-04-20T01:47:42.787 INFO:tasks.rados.rados.0.smithi165.stdout:105: oid 40 updating version 5 to 6 2024-04-20T01:47:42.787 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 40 v 6 (ObjNum 63 snap 4 seq_num 63) dirty exists 2024-04-20T01:47:42.787 INFO:tasks.rados.rados.0.smithi165.stdout:105: left oid 40 (ObjNum 63 snap 4 seq_num 63) 2024-04-20T01:47:42.787 INFO:tasks.rados.rados.0.smithi165.stdout:106: finishing write tid 1 to smithi16579500-45 2024-04-20T01:47:42.787 INFO:tasks.rados.rados.0.smithi165.stdout:106: finishing write tid 2 to smithi16579500-45 2024-04-20T01:47:42.787 INFO:tasks.rados.rados.0.smithi165.stdout:106: finishing write tid 3 to smithi16579500-45 2024-04-20T01:47:42.788 INFO:tasks.rados.rados.0.smithi165.stdout:106: finishing write tid 4 to smithi16579500-45 2024-04-20T01:47:42.788 INFO:tasks.rados.rados.0.smithi165.stdout:106: oid 45 updating version 0 to 24 2024-04-20T01:47:42.788 INFO:tasks.rados.rados.0.smithi165.stdout:106: oid 45 version 24 is already newer than 23 2024-04-20T01:47:42.788 INFO:tasks.rados.rados.0.smithi165.stdout:106: oid 45 updating version 24 to 25 2024-04-20T01:47:42.788 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 45 v 25 (ObjNum 64 snap 4 seq_num 64) dirty exists 2024-04-20T01:47:42.788 INFO:tasks.rados.rados.0.smithi165.stdout:106: left oid 45 (ObjNum 64 snap 4 seq_num 64) 2024-04-20T01:47:42.788 INFO:tasks.rados.rados.0.smithi165.stdout:105: done (3 left) 2024-04-20T01:47:42.788 INFO:tasks.rados.rados.0.smithi165.stdout:106: done (2 left) 2024-04-20T01:47:42.788 INFO:tasks.rados.rados.0.smithi165.stdout:111: delete oid 4 current snap is 4 2024-04-20T01:47:42.792 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.8 to 1 (10000) 2024-04-20T01:47:42.798 INFO:tasks.rados.rados.0.smithi165.stdout:109: expect (ObjNum 24 snap 0 seq_num 24) 2024-04-20T01:47:42.811 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 18 v 11 (ObjNum 289 snap 0 seq_num 1) dirty exists 2024-04-20T01:47:42.811 INFO:tasks.rados.rados.0.smithi165.stdout:109: done (2 left) 2024-04-20T01:47:42.812 INFO:tasks.rados.rados.0.smithi165.stdout:110: done (1 left) 2024-04-20T01:47:42.812 INFO:tasks.rados.rados.0.smithi165.stdout:111: done (0 left) 2024-04-20T01:47:42.812 INFO:tasks.rados.rados.0.smithi165.stdout:112: setattr oid 42 current snap is 4 2024-04-20T01:47:42.812 INFO:tasks.rados.rados.0.smithi165.stdout:113: rollback oid 18 current snap is 4 2024-04-20T01:47:42.812 INFO:tasks.rados.rados.0.smithi165.stdout:113: done (1 left) 2024-04-20T01:47:42.813 INFO:tasks.rados.rados.0.smithi165.stdout:114: setattr oid 38 current snap is 4 2024-04-20T01:47:42.814 INFO:tasks.rados.rados.0.smithi165.stdout:115: snap_create 2024-04-20T01:47:42.815 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-04-20T01:47:42.818 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 38 v 23 (ObjNum 50 snap 1 seq_num 50) dirty exists 2024-04-20T01:47:42.818 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 42 v 12 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-04-20T01:47:42.821 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-20T01:47:42.856 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:47:42.870+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:47:42.913 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:42.913 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:42.923 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-20T01:47:42.931 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:42.945+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:47:43.031 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:43.045+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:43.041 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:47:43.061 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:43.061 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:43.062 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:47:43.072 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-20T01:47:43.083 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:47:43.104 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:47:43.124 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:47:43.132 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:43.147+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:47:43.146 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:47:43.166 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:47:43.187 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:47:43.201 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:47:43.201 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:47:43.201 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:47:43.201 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:47:43.201 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:47:43.207 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:47:43.211 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-20T01:47:43.227 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:47:43.233 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:43.247+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:43.248 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:47:43.269 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:47:43.289 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:47:43.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:47:43.304+0000 7f863659c700 0 -- 172.21.15.44:0/4293465129 >> [v2:172.21.15.121:6808/74355,v1:172.21.15.121:6809/74355] conn(0x7f8638219590 msgr2=0x7f863821b980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-20T01:47:43.310 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:47:43.331 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:47:43.334 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:43.349+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:47:43.351 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:47:43.353 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:47:43.353 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:47:43.353 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:47:43.353 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:47:43.353 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:47:43.353 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:47:43.354 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:47:43.354 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:47:43.354 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:47:43.354 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:47:43.354 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:47:43.354 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:47:43.354 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:47:43.354 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:47:43.354 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:47:43.354 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:47:43.383 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:47:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:47:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:47:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:47:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.1s5 36/30 ECSubWrite(tid=55, reqid=osd.4.0:55, at_version=36'8, trim_to=0'0, roll_forward_to=35'6))", 2024-04-20T01:47:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:37.767104+0000", 2024-04-20T01:47:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.6292476540000003, 2024-04-20T01:47:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0025099810000000001, 2024-04-20T01:47:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.767104+0000", 2024-04-20T01:47:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.767104+0000", 2024-04-20T01:47:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.767104+0000", 2024-04-20T01:47:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.767137+0000", 2024-04-20T01:47:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2561999999999998e-05 2024-04-20T01:47:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.767147+0000", 2024-04-20T01:47:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0231000000000001e-05 2024-04-20T01:47:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.767159+0000", 2024-04-20T01:47:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2473000000000001e-05 2024-04-20T01:47:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.767245+0000", 2024-04-20T01:47:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6212999999999996e-05 2024-04-20T01:47:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.767263+0000", 2024-04-20T01:47:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8026e-05 2024-04-20T01:47:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.769596+0000", 2024-04-20T01:47:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023321840000000002 2024-04-20T01:47:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.769614+0000", 2024-04-20T01:47:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8612000000000001e-05 2024-04-20T01:47:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.1s5 36/30 ECSubWrite(tid=56, reqid=osd.4.0:55, at_version=0'0, trim_to=0'0, roll_forward_to=36'8))", 2024-04-20T01:47:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:37.770237+0000", 2024-04-20T01:47:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.6261146630000001, 2024-04-20T01:47:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.004358214, 2024-04-20T01:47:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.770237+0000", 2024-04-20T01:47:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.770237+0000", 2024-04-20T01:47:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.770236+0000", 2024-04-20T01:47:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.770250+0000", 2024-04-20T01:47:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3879e-05 2024-04-20T01:47:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.770265+0000", 2024-04-20T01:47:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.468e-05 2024-04-20T01:47:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.770276+0000", 2024-04-20T01:47:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.095e-05 2024-04-20T01:47:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.770382+0000", 2024-04-20T01:47:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106421 2024-04-20T01:47:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.770429+0000", 2024-04-20T01:47:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6313999999999997e-05 2024-04-20T01:47:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.774568+0000", 2024-04-20T01:47:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0041392460000000001 2024-04-20T01:47:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.774596+0000", 2024-04-20T01:47:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7654999999999999e-05 2024-04-20T01:47:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs3 36/30 ECSubWrite(tid=19, reqid=client.4643.0:218, at_version=36'13, trim_to=0'0, roll_forward_to=34'11))", 2024-04-20T01:47:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:37.798799+0000", 2024-04-20T01:47:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5975531409999997, 2024-04-20T01:47:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00089701699999999995, 2024-04-20T01:47:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.798799+0000", 2024-04-20T01:47:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.798799+0000", 2024-04-20T01:47:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.798798+0000", 2024-04-20T01:47:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.798820+0000", 2024-04-20T01:47:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1226999999999999e-05 2024-04-20T01:47:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.798828+0000", 2024-04-20T01:47:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8699999999999998e-06 2024-04-20T01:47:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.798836+0000", 2024-04-20T01:47:43.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6939999999999995e-06 2024-04-20T01:47:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.798892+0000", 2024-04-20T01:47:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5368999999999998e-05 2024-04-20T01:47:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.798908+0000", 2024-04-20T01:47:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6024000000000001e-05 2024-04-20T01:47:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799668+0000", 2024-04-20T01:47:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00076025500000000005 2024-04-20T01:47:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799696+0000", 2024-04-20T01:47:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8028e-05 2024-04-20T01:47:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs3 36/30 ECSubWrite(tid=21, reqid=client.4643.0:220, at_version=36'15, trim_to=0'0, roll_forward_to=34'11))", 2024-04-20T01:47:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:37.801103+0000", 2024-04-20T01:47:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5952485909999998, 2024-04-20T01:47:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021535040000000001, 2024-04-20T01:47:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.801103+0000", 2024-04-20T01:47:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.801103+0000", 2024-04-20T01:47:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.801103+0000", 2024-04-20T01:47:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.801123+0000", 2024-04-20T01:47:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0534e-05 2024-04-20T01:47:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.801137+0000", 2024-04-20T01:47:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4269e-05 2024-04-20T01:47:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.801147+0000", 2024-04-20T01:47:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1889999999999996e-06 2024-04-20T01:47:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.801194+0000", 2024-04-20T01:47:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6986e-05 2024-04-20T01:47:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.801204+0000", 2024-04-20T01:47:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0866000000000001e-05 2024-04-20T01:47:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.803229+0000", 2024-04-20T01:47:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0020244870000000002 2024-04-20T01:47:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.803257+0000", 2024-04-20T01:47:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7926999999999999e-05 2024-04-20T01:47:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s5 36/30 ECSubWrite(tid=35, reqid=client.4643.0:217, at_version=36'26, trim_to=0'0, roll_forward_to=36'24))", 2024-04-20T01:47:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:37.802281+0000", 2024-04-20T01:47:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5940705560000001, 2024-04-20T01:47:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0041255930000000003, 2024-04-20T01:47:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.802281+0000", 2024-04-20T01:47:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.802281+0000", 2024-04-20T01:47:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.802281+0000", 2024-04-20T01:47:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.802411+0000", 2024-04-20T01:47:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000129818 2024-04-20T01:47:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.802427+0000", 2024-04-20T01:47:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6524999999999998e-05 2024-04-20T01:47:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.802437+0000", 2024-04-20T01:47:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0210999999999999e-05 2024-04-20T01:47:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.802480+0000", 2024-04-20T01:47:43.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2367999999999998e-05 2024-04-20T01:47:43.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.802503+0000", 2024-04-20T01:47:43.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3519999999999998e-05 2024-04-20T01:47:43.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806368+0000", 2024-04-20T01:47:43.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0038649969999999998 2024-04-20T01:47:43.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806407+0000", 2024-04-20T01:47:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8863999999999999e-05 2024-04-20T01:47:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs3 36/30 ECSubWrite(tid=22, reqid=client.4643.0:220, at_version=0'0, trim_to=0'0, roll_forward_to=36'15))", 2024-04-20T01:47:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:37.807042+0000", 2024-04-20T01:47:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5893102089999998, 2024-04-20T01:47:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0048291180000000003, 2024-04-20T01:47:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807042+0000", 2024-04-20T01:47:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807042+0000", 2024-04-20T01:47:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807041+0000", 2024-04-20T01:47:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807054+0000", 2024-04-20T01:47:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3039e-05 2024-04-20T01:47:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807068+0000", 2024-04-20T01:47:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4059e-05 2024-04-20T01:47:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807078+0000", 2024-04-20T01:47:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7070000000000004e-06 2024-04-20T01:47:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807121+0000", 2024-04-20T01:47:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2981000000000001e-05 2024-04-20T01:47:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807130+0000", 2024-04-20T01:47:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.55e-06 2024-04-20T01:47:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.811845+0000", 2024-04-20T01:47:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047150400000000002 2024-04-20T01:47:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.811871+0000", 2024-04-20T01:47:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.544e-05 2024-04-20T01:47:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s5 36/30 ECSubWrite(tid=39, reqid=client.4643.0:217, at_version=0'0, trim_to=0'0, roll_forward_to=36'26))", 2024-04-20T01:47:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:37.807183+0000", 2024-04-20T01:47:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5891687929999998, 2024-04-20T01:47:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047013510000000003, 2024-04-20T01:47:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807183+0000", 2024-04-20T01:47:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807183+0000", 2024-04-20T01:47:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807182+0000", 2024-04-20T01:47:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807194+0000", 2024-04-20T01:47:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1083e-05 2024-04-20T01:47:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807204+0000", 2024-04-20T01:47:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0314e-05 2024-04-20T01:47:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807223+0000", 2024-04-20T01:47:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9043e-05 2024-04-20T01:47:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807271+0000", 2024-04-20T01:47:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8040000000000001e-05 2024-04-20T01:47:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807281+0000", 2024-04-20T01:47:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7729999999999994e-06 2024-04-20T01:47:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.811874+0000", 2024-04-20T01:47:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0045932569999999999 2024-04-20T01:47:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.811885+0000", 2024-04-20T01:47:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0553e-05 2024-04-20T01:47:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s5 36/30 ECSubWrite(tid=40, reqid=client.4643.0:215, at_version=36'28, trim_to=0'0, roll_forward_to=36'26))", 2024-04-20T01:47:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:37.808713+0000", 2024-04-20T01:47:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5876391119999997, 2024-04-20T01:47:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059407280000000002, 2024-04-20T01:47:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.808713+0000", 2024-04-20T01:47:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.808713+0000", 2024-04-20T01:47:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.808712+0000", 2024-04-20T01:47:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.808741+0000", 2024-04-20T01:47:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9128999999999998e-05 2024-04-20T01:47:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.808753+0000", 2024-04-20T01:47:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1314999999999999e-05 2024-04-20T01:47:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.808759+0000", 2024-04-20T01:47:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7560000000000002e-06 2024-04-20T01:47:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.808810+0000", 2024-04-20T01:47:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0291999999999999e-05 2024-04-20T01:47:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.808820+0000", 2024-04-20T01:47:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7739999999999999e-06 2024-04-20T01:47:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.814625+0000", 2024-04-20T01:47:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0058058550000000004 2024-04-20T01:47:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.814654+0000", 2024-04-20T01:47:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8212999999999999e-05 2024-04-20T01:47:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:234 3.0s0 3:0c1ed9f1:::smithi16579500-46:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:47:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.787453+0000", 2024-04-20T01:47:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.6088992710000003, 2024-04-20T01:47:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016795529, 2024-04-20T01:47:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:47:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:47:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 234 2024-04-20T01:47:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787453+0000", 2024-04-20T01:47:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787453+0000", 2024-04-20T01:47:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787451+0000", 2024-04-20T01:47:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:47:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787462+0000", 2024-04-20T01:47:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1083e-05 2024-04-20T01:47:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787483+0000", 2024-04-20T01:47:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1656999999999999e-05 2024-04-20T01:47:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787503+0000", 2024-04-20T01:47:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9623000000000002e-05 2024-04-20T01:47:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787595+0000", 2024-04-20T01:47:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2417000000000006e-05 2024-04-20T01:47:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:47:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787604+0000", 2024-04-20T01:47:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.952e-06 2024-04-20T01:47:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.791584+0000", 2024-04-20T01:47:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039792250000000003 2024-04-20T01:47:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.791699+0000", 2024-04-20T01:47:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115692 2024-04-20T01:47:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.792530+0000", 2024-04-20T01:47:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083077900000000002 2024-04-20T01:47:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.798777+0000", 2024-04-20T01:47:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062472379999999996 2024-04-20T01:47:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:47:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.804175+0000", 2024-04-20T01:47:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005397158 2024-04-20T01:47:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.804248+0000", 2024-04-20T01:47:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3682000000000004e-05 2024-04-20T01:47:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:235 3.0s0 3:0c1ed9f1:::smithi16579500-46:head [rollback 5] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:47:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.787517+0000", 2024-04-20T01:47:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.6088354420000002, 2024-04-20T01:47:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019202061999999999, 2024-04-20T01:47:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:47:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:47:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 235 2024-04-20T01:47:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787517+0000", 2024-04-20T01:47:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787517+0000", 2024-04-20T01:47:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787516+0000", 2024-04-20T01:47:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787521+0000", 2024-04-20T01:47:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2599999999999996e-06 2024-04-20T01:47:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787534+0000", 2024-04-20T01:47:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2717e-05 2024-04-20T01:47:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787538+0000", 2024-04-20T01:47:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3540000000000002e-06 2024-04-20T01:47:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787673+0000", 2024-04-20T01:47:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013480799999999999 2024-04-20T01:47:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:47:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787678+0000", 2024-04-20T01:47:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5229999999999997e-06 2024-04-20T01:47:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793075+0000", 2024-04-20T01:47:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053977549999999997 2024-04-20T01:47:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793190+0000", 2024-04-20T01:47:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114775 2024-04-20T01:47:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793760+0000", 2024-04-20T01:47:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00056945599999999998 2024-04-20T01:47:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799198+0000", 2024-04-20T01:47:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0054385589999999999 2024-04-20T01:47:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:47:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806653+0000", 2024-04-20T01:47:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0074547739999999999 2024-04-20T01:47:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806719+0000", 2024-04-20T01:47:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5732e-05 2024-04-20T01:47:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:236 3.0s0 3:0c1ed9f1:::smithi16579500-46:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:47:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:47:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.6087758279999997, 2024-04-20T01:47:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019273109, 2024-04-20T01:47:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:47:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:47:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 236 2024-04-20T01:47:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:47:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:47:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:47:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787579+0000", 2024-04-20T01:47:43.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6380000000000002e-06 2024-04-20T01:47:43.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787586+0000", 2024-04-20T01:47:43.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.81e-06 2024-04-20T01:47:43.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787592+0000", 2024-04-20T01:47:43.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5180000000000004e-06 2024-04-20T01:47:43.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787705+0000", 2024-04-20T01:47:43.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011387000000000001 2024-04-20T01:47:43.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:47:43.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787712+0000", 2024-04-20T01:47:43.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6229999999999999e-06 2024-04-20T01:47:43.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794765+0000", 2024-04-20T01:47:43.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0070530519999999998 2024-04-20T01:47:43.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:43.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794873+0000", 2024-04-20T01:47:43.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107861 2024-04-20T01:47:43.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795365+0000", 2024-04-20T01:47:43.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049192300000000001 2024-04-20T01:47:43.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799201+0000", 2024-04-20T01:47:43.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003836454 2024-04-20T01:47:43.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:47:43.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806808+0000", 2024-04-20T01:47:43.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076070119999999998 2024-04-20T01:47:43.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806849+0000", 2024-04-20T01:47:43.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0852999999999997e-05 2024-04-20T01:47:43.438 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.438 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.438 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.439 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s5 37/30 ECSubWrite(tid=42, reqid=client.4643.0:240, at_version=37'30, trim_to=0'0, roll_forward_to=36'28))", 2024-04-20T01:47:43.439 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.795079+0000", 2024-04-20T01:47:43.439 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.601272754, 2024-04-20T01:47:43.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0041411709999999999, 2024-04-20T01:47:43.439 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.439 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.439 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795079+0000", 2024-04-20T01:47:43.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795079+0000", 2024-04-20T01:47:43.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795079+0000", 2024-04-20T01:47:43.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:47:43.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795105+0000", 2024-04-20T01:47:43.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6319999999999999e-05 2024-04-20T01:47:43.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795115+0000", 2024-04-20T01:47:43.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9049999999999992e-06 2024-04-20T01:47:43.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795122+0000", 2024-04-20T01:47:43.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4330000000000001e-06 2024-04-20T01:47:43.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795162+0000", 2024-04-20T01:47:43.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9929999999999999e-05 2024-04-20T01:47:43.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795169+0000", 2024-04-20T01:47:43.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999999e-06 2024-04-20T01:47:43.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799191+0000", 2024-04-20T01:47:43.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040215249999999998 2024-04-20T01:47:43.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799220+0000", 2024-04-20T01:47:43.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9774e-05 2024-04-20T01:47:43.443 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.443 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.443 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.443 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s5 37/30 ECSubWrite(tid=43, reqid=client.4643.0:241, at_version=37'31, trim_to=0'0, roll_forward_to=36'28))", 2024-04-20T01:47:43.443 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.795130+0000", 2024-04-20T01:47:43.443 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.6012223060000004, 2024-04-20T01:47:43.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.004118989, 2024-04-20T01:47:43.443 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.443 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.443 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795130+0000", 2024-04-20T01:47:43.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795130+0000", 2024-04-20T01:47:43.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795129+0000", 2024-04-20T01:47:43.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:43.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795144+0000", 2024-04-20T01:47:43.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5068e-05 2024-04-20T01:47:43.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795149+0000", 2024-04-20T01:47:43.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9799999999999998e-06 2024-04-20T01:47:43.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795161+0000", 2024-04-20T01:47:43.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1148000000000001e-05 2024-04-20T01:47:43.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795507+0000", 2024-04-20T01:47:43.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034630099999999998 2024-04-20T01:47:43.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795517+0000", 2024-04-20T01:47:43.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0271e-05 2024-04-20T01:47:43.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799224+0000", 2024-04-20T01:47:43.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037071780000000002 2024-04-20T01:47:43.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799249+0000", 2024-04-20T01:47:43.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.429e-05 2024-04-20T01:47:43.447 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.447 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.447 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.447 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s5 37/30 ECSubWrite(tid=44, reqid=client.4643.0:242, at_version=37'32, trim_to=0'0, roll_forward_to=36'28))", 2024-04-20T01:47:43.447 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.795168+0000", 2024-04-20T01:47:43.447 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.6011841020000004, 2024-04-20T01:47:43.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040968649999999999, 2024-04-20T01:47:43.447 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.448 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.448 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795168+0000", 2024-04-20T01:47:43.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795168+0000", 2024-04-20T01:47:43.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795168+0000", 2024-04-20T01:47:43.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:47:43.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795175+0000", 2024-04-20T01:47:43.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3329999999999999e-06 2024-04-20T01:47:43.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795179+0000", 2024-04-20T01:47:43.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1330000000000001e-06 2024-04-20T01:47:43.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795183+0000", 2024-04-20T01:47:43.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5279999999999999e-06 2024-04-20T01:47:43.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795778+0000", 2024-04-20T01:47:43.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00059490400000000005 2024-04-20T01:47:43.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795788+0000", 2024-04-20T01:47:43.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0342e-05 2024-04-20T01:47:43.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799251+0000", 2024-04-20T01:47:43.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003463172 2024-04-20T01:47:43.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799265+0000", 2024-04-20T01:47:43.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3611999999999999e-05 2024-04-20T01:47:43.451 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.451 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.451 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.451 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 37/30 ECSubWriteReply(tid=19, last_complete=37'9, committed=1, applied=1))", 2024-04-20T01:47:43.451 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.796598+0000", 2024-04-20T01:47:43.452 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.5997538919999998, 2024-04-20T01:47:43.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0020534619999999998, 2024-04-20T01:47:43.452 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.452 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.452 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796598+0000", 2024-04-20T01:47:43.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796598+0000", 2024-04-20T01:47:43.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796597+0000", 2024-04-20T01:47:43.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:43.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796616+0000", 2024-04-20T01:47:43.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8564e-05 2024-04-20T01:47:43.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796629+0000", 2024-04-20T01:47:43.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3539e-05 2024-04-20T01:47:43.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796638+0000", 2024-04-20T01:47:43.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6899999999999998e-06 2024-04-20T01:47:43.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.798637+0000", 2024-04-20T01:47:43.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001998976 2024-04-20T01:47:43.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.798652+0000", 2024-04-20T01:47:43.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4391e-05 2024-04-20T01:47:43.455 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.455 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.455 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.455 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 37/30 ECSubWriteReply(tid=19, last_complete=37'9, committed=1, applied=1))", 2024-04-20T01:47:43.455 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.796653+0000", 2024-04-20T01:47:43.455 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.5996993149999996, 2024-04-20T01:47:43.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0020628769999999999, 2024-04-20T01:47:43.455 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.455 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.456 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796653+0000", 2024-04-20T01:47:43.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796653+0000", 2024-04-20T01:47:43.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796652+0000", 2024-04-20T01:47:43.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:43.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796656+0000", 2024-04-20T01:47:43.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9720000000000003e-06 2024-04-20T01:47:43.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796663+0000", 2024-04-20T01:47:43.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0090000000000001e-06 2024-04-20T01:47:43.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796677+0000", 2024-04-20T01:47:43.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3893000000000001e-05 2024-04-20T01:47:43.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.798706+0000", 2024-04-20T01:47:43.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0020292299999999999 2024-04-20T01:47:43.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.798716+0000", 2024-04-20T01:47:43.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2429999999999994e-06 2024-04-20T01:47:43.458 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.458 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.458 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.459 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 37/30 ECSubWriteReply(tid=18, last_complete=37'8, committed=1, applied=1))", 2024-04-20T01:47:43.459 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.796682+0000", 2024-04-20T01:47:43.459 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.5996701419999999, 2024-04-20T01:47:43.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0020018549999999999, 2024-04-20T01:47:43.459 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.459 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.459 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796682+0000", 2024-04-20T01:47:43.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796682+0000", 2024-04-20T01:47:43.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796681+0000", 2024-04-20T01:47:43.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:43.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796686+0000", 2024-04-20T01:47:43.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3539999999999999e-06 2024-04-20T01:47:43.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796696+0000", 2024-04-20T01:47:43.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0002e-05 2024-04-20T01:47:43.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796703+0000", 2024-04-20T01:47:43.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4019999999999999e-06 2024-04-20T01:47:43.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.798673+0000", 2024-04-20T01:47:43.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019707750000000001 2024-04-20T01:47:43.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.798684+0000", 2024-04-20T01:47:43.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0210000000000001e-05 2024-04-20T01:47:43.462 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.462 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.462 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.462 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s5 37/30 ECSubWrite(tid=45, reqid=client.4643.0:242, at_version=0'0, trim_to=0'0, roll_forward_to=37'32))", 2024-04-20T01:47:43.462 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.801645+0000", 2024-04-20T01:47:43.462 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.59470665, 2024-04-20T01:47:43.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0010556350000000001, 2024-04-20T01:47:43.462 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.463 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.463 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.801645+0000", 2024-04-20T01:47:43.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.801645+0000", 2024-04-20T01:47:43.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.801644+0000", 2024-04-20T01:47:43.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:43.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.801659+0000", 2024-04-20T01:47:43.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4609e-05 2024-04-20T01:47:43.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.801676+0000", 2024-04-20T01:47:43.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7183000000000001e-05 2024-04-20T01:47:43.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.801687+0000", 2024-04-20T01:47:43.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0791000000000001e-05 2024-04-20T01:47:43.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.801741+0000", 2024-04-20T01:47:43.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4036999999999998e-05 2024-04-20T01:47:43.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.801752+0000", 2024-04-20T01:47:43.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0924000000000001e-05 2024-04-20T01:47:43.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.802668+0000", 2024-04-20T01:47:43.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00091601699999999998 2024-04-20T01:47:43.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.802701+0000", 2024-04-20T01:47:43.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2974e-05 2024-04-20T01:47:43.466 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.466 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.466 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.467 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s5 38/30 ECSubWrite(tid=46, reqid=osd.8.0:46, at_version=38'34, trim_to=0'0, roll_forward_to=37'32))", 2024-04-20T01:47:43.467 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.791931+0000", 2024-04-20T01:47:43.467 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.6044210329999999, 2024-04-20T01:47:43.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027768530000000001, 2024-04-20T01:47:43.467 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.467 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.467 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791931+0000", 2024-04-20T01:47:43.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791931+0000", 2024-04-20T01:47:43.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791931+0000", 2024-04-20T01:47:43.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:47:43.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791954+0000", 2024-04-20T01:47:43.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319000000000002e-05 2024-04-20T01:47:43.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791962+0000", 2024-04-20T01:47:43.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3350000000000007e-06 2024-04-20T01:47:43.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791973+0000", 2024-04-20T01:47:43.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1107999999999999e-05 2024-04-20T01:47:43.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792021+0000", 2024-04-20T01:47:43.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7315999999999999e-05 2024-04-20T01:47:43.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792031+0000", 2024-04-20T01:47:43.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0496000000000001e-05 2024-04-20T01:47:43.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794681+0000", 2024-04-20T01:47:43.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0026496129999999999 2024-04-20T01:47:43.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794708+0000", 2024-04-20T01:47:43.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6888000000000001e-05 2024-04-20T01:47:43.471 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.471 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.471 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.471 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.es5 38/30 ECSubWrite(tid=19, reqid=client.4643.0:245, at_version=38'12, trim_to=0'0, roll_forward_to=34'11))", 2024-04-20T01:47:43.471 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.805596+0000", 2024-04-20T01:47:43.471 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.5907557419999998, 2024-04-20T01:47:43.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019413379999999999, 2024-04-20T01:47:43.471 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.471 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.471 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.805596+0000", 2024-04-20T01:47:43.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.805596+0000", 2024-04-20T01:47:43.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.805596+0000", 2024-04-20T01:47:43.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:43.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.805672+0000", 2024-04-20T01:47:43.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6056000000000005e-05 2024-04-20T01:47:43.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.805686+0000", 2024-04-20T01:47:43.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3624e-05 2024-04-20T01:47:43.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.805696+0000", 2024-04-20T01:47:43.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0176999999999999e-05 2024-04-20T01:47:43.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.805753+0000", 2024-04-20T01:47:43.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7679000000000001e-05 2024-04-20T01:47:43.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.805765+0000", 2024-04-20T01:47:43.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.171e-05 2024-04-20T01:47:43.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.807504+0000", 2024-04-20T01:47:43.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0017388810000000001 2024-04-20T01:47:43.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.807538+0000", 2024-04-20T01:47:43.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3559000000000002e-05 2024-04-20T01:47:43.475 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.475 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.475 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.475 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.475 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.475 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:47:43.477 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-20T01:47:43.477 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:43.491+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:47:43.578 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:47:43.592+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:47:43.647 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:47:43.648 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:47:43.648 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-04-20T01:47:43.648 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:47:43.658 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-20T01:47:43.678 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:47:43.692+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:47:43.779 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:47:43.793+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:47:43.785 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:47:43.786 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:47:43.786 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:47:43.786 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:47:43.786 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:47:43.793 INFO:tasks.rados.rados.0.smithi165.stdout:112: done (2 left) 2024-04-20T01:47:43.793 INFO:tasks.rados.rados.0.smithi165.stdout:114: done (1 left) 2024-04-20T01:47:43.794 INFO:tasks.rados.rados.0.smithi165.stdout:115: done (0 left) 2024-04-20T01:47:43.794 INFO:tasks.rados.rados.0.smithi165.stdout:116: read oid 11 snap -1 2024-04-20T01:47:43.794 INFO:tasks.rados.rados.0.smithi165.stdout:116: expect (ObjNum 10 snap 0 seq_num 10) 2024-04-20T01:47:43.794 INFO:tasks.rados.rados.0.smithi165.stdout:117: setattr oid 13 current snap is 5 2024-04-20T01:47:43.795 INFO:tasks.rados.rados.0.smithi165.stdout:118: copy_from oid 3 from oid 26 current snap is 5 2024-04-20T01:47:43.795 INFO:tasks.rados.rados.0.smithi165.stdout:119: read oid 19 snap -1 2024-04-20T01:47:43.795 INFO:tasks.rados.rados.0.smithi165.stdout:119: expect (ObjNum 56 snap 2 seq_num 56) 2024-04-20T01:47:43.795 INFO:tasks.rados.rados.0.smithi165.stdout:120: read oid 38 snap -1 2024-04-20T01:47:43.795 INFO:tasks.rados.rados.0.smithi165.stdout:120: expect (ObjNum 50 snap 1 seq_num 50) 2024-04-20T01:47:43.795 INFO:tasks.rados.rados.0.smithi165.stdout:121: setattr oid 20 current snap is 5 2024-04-20T01:47:43.795 INFO:tasks.rados.rados.0.smithi165.stdout:122: rollback oid 33 current snap is 5 2024-04-20T01:47:43.795 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 33 to 4 2024-04-20T01:47:43.795 INFO:tasks.rados.rados.0.smithi165.stdout:123: delete oid 30 current snap is 5 2024-04-20T01:47:43.798 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-20T01:47:43.798 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.9 to 1 (10000) 2024-04-20T01:47:43.799 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 13 v 32 (ObjNum 65 snap 0 seq_num 8) dirty exists 2024-04-20T01:47:43.801 INFO:tasks.rados.rados.0.smithi165.stdout:116: expect (ObjNum 10 snap 0 seq_num 10) 2024-04-20T01:47:43.807 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-20T01:47:43.808 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 20 v 33 (ObjNum 129 snap 0 seq_num 2584377840) dirty exists 2024-04-20T01:47:43.808 INFO:tasks.rados.rados.0.smithi165.stdout:118: finishing copy_from racing read to smithi16579500-3 2024-04-20T01:47:43.808 INFO:tasks.rados.rados.0.smithi165.stdout:116: done (7 left) 2024-04-20T01:47:43.808 INFO:tasks.rados.rados.0.smithi165.stdout:117: done (6 left) 2024-04-20T01:47:43.808 INFO:tasks.rados.rados.0.smithi165.stdout:121: done (5 left) 2024-04-20T01:47:43.809 INFO:tasks.rados.rados.0.smithi165.stdout:123: done (4 left) 2024-04-20T01:47:43.809 INFO:tasks.rados.rados.0.smithi165.stdout:124: copy_from oid 29 from oid 34 current snap is 5 2024-04-20T01:47:43.809 INFO:tasks.rados.rados.0.smithi165.stdout:125: read oid 4 snap -1 2024-04-20T01:47:43.809 INFO:tasks.rados.rados.0.smithi165.stdout:125: expect deleted 2024-04-20T01:47:43.809 INFO:tasks.rados.rados.0.smithi165.stdout:126: rmattr oid 25 current snap is 5 2024-04-20T01:47:43.809 INFO:tasks.rados.rados.0.smithi165.stdout:126: done (6 left) 2024-04-20T01:47:43.809 INFO:tasks.rados.rados.0.smithi165.stdout:127: copy_from oid 14 from oid 8 current snap is 5 2024-04-20T01:47:43.809 INFO:tasks.rados.rados.0.smithi165.stdout:128: delete oid 48 current snap is 5 2024-04-20T01:47:43.809 INFO:tasks.rados.rados.0.smithi165.stdout:118: finishing copy_from to smithi16579500-3 2024-04-20T01:47:43.809 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 3 v 6 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-04-20T01:47:43.810 INFO:tasks.rados.rados.0.smithi165.stdout:120: expect (ObjNum 50 snap 1 seq_num 50) 2024-04-20T01:47:43.818 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-04-20T01:47:43.862 INFO:tasks.rados.rados.0.smithi165.stdout:122: finishing rollback tid 0 to smithi16579500-33 2024-04-20T01:47:43.862 INFO:tasks.rados.rados.0.smithi165.stdout:122: finishing rollback tid 1 to smithi16579500-33 2024-04-20T01:47:43.862 INFO:tasks.rados.rados.0.smithi165.stdout:118: done (7 left) 2024-04-20T01:47:43.862 INFO:tasks.rados.rados.0.smithi165.stdout:120: done (6 left) 2024-04-20T01:47:43.862 INFO:tasks.rados.rados.0.smithi165.stdout:125: done (5 left) 2024-04-20T01:47:43.862 INFO:tasks.rados.rados.0.smithi165.stdout:128: done (4 left) 2024-04-20T01:47:43.862 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 9 current snap is 5 2024-04-20T01:47:43.863 INFO:tasks.rados.rados.0.smithi165.stdout:122: finishing rollback tid 2 to smithi16579500-33 2024-04-20T01:47:43.863 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 33 v 27 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-04-20T01:47:43.863 INFO:tasks.rados.rados.0.smithi165.stdout:119: expect (ObjNum 56 snap 2 seq_num 56) 2024-04-20T01:47:43.880 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:43.894+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:47:43.898 INFO:tasks.rados.rados.0.smithi165.stdout:124: finishing copy_from racing read to smithi16579500-29 2024-04-20T01:47:43.898 INFO:tasks.rados.rados.0.smithi165.stdout:127: finishing copy_from racing read to smithi16579500-14 2024-04-20T01:47:43.899 INFO:tasks.rados.rados.0.smithi165.stdout:124: finishing copy_from to smithi16579500-29 2024-04-20T01:47:43.899 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 29 v 10 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-04-20T01:47:43.899 INFO:tasks.rados.rados.0.smithi165.stdout:127: finishing copy_from to smithi16579500-14 2024-04-20T01:47:43.899 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 14 v 2 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-20T01:47:43.899 INFO:tasks.rados.rados.0.smithi165.stdout:129: seq_num 68 ranges {147456=770048,917504=376832} 2024-04-20T01:47:43.903 INFO:tasks.rados.rados.0.smithi165.stdout:129: writing smithi16579500-9 from 147456 to 917504 tid 1 2024-04-20T01:47:43.905 INFO:tasks.rados.rados.0.smithi165.stdout:129: writing smithi16579500-9 from 917504 to 1294336 tid 2 2024-04-20T01:47:43.905 INFO:tasks.rados.rados.0.smithi165.stdout:119: done (4 left) 2024-04-20T01:47:43.906 INFO:tasks.rados.rados.0.smithi165.stdout:122: done (3 left) 2024-04-20T01:47:43.906 INFO:tasks.rados.rados.0.smithi165.stdout:124: done (2 left) 2024-04-20T01:47:43.906 INFO:tasks.rados.rados.0.smithi165.stdout:127: done (1 left) 2024-04-20T01:47:43.906 INFO:tasks.rados.rados.0.smithi165.stdout:130: copy_from oid 24 from oid 32 current snap is 5 2024-04-20T01:47:43.906 INFO:tasks.rados.rados.0.smithi165.stdout:131: setattr oid 45 current snap is 5 2024-04-20T01:47:43.907 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 19 current snap is 5 2024-04-20T01:47:43.907 INFO:tasks.rados.rados.0.smithi165.stdout:132: seq_num 69 ranges {1261568=786432,2048000=376832} 2024-04-20T01:47:43.918 INFO:tasks.rados.rados.0.smithi165.stdout:132: writing smithi16579500-19 from 1261568 to 2048000 tid 1 2024-04-20T01:47:43.921 INFO:tasks.rados.rados.0.smithi165.stdout:132: writing smithi16579500-19 from 2048000 to 2424832 tid 2 2024-04-20T01:47:43.921 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 13 current snap is 5 2024-04-20T01:47:43.921 INFO:tasks.rados.rados.0.smithi165.stdout:133: seq_num 70 ranges {0=802816,802816=376832} 2024-04-20T01:47:43.925 INFO:tasks.rados.rados.0.smithi165.stdout:133: writing smithi16579500-13 from 0 to 802816 tid 1 2024-04-20T01:47:43.928 INFO:tasks.rados.rados.0.smithi165.stdout:133: writing smithi16579500-13 from 802816 to 1179648 tid 2 2024-04-20T01:47:43.928 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 36 current snap is 5 2024-04-20T01:47:43.928 INFO:tasks.rados.rados.0.smithi165.stdout:129: finishing write tid 1 to smithi16579500-9 2024-04-20T01:47:43.928 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 45 v 35 (ObjNum 64 snap 4 seq_num 64) dirty exists 2024-04-20T01:47:43.928 INFO:tasks.rados.rados.0.smithi165.stdout:130: finishing copy_from racing read to smithi16579500-24 2024-04-20T01:47:43.929 INFO:tasks.rados.rados.0.smithi165.stdout:129: finishing write tid 2 to smithi16579500-9 2024-04-20T01:47:43.929 INFO:tasks.rados.rados.0.smithi165.stdout:134: seq_num 71 ranges {589824=425984,1015808=573440,1589248=196608} 2024-04-20T01:47:43.933 INFO:tasks.rados.rados.0.smithi165.stdout:134: writing smithi16579500-36 from 589824 to 1015808 tid 1 2024-04-20T01:47:43.937 INFO:tasks.rados.rados.0.smithi165.stdout:134: writing smithi16579500-36 from 1015808 to 1589248 tid 2 2024-04-20T01:47:43.938 INFO:tasks.rados.rados.0.smithi165.stdout:134: writing smithi16579500-36 from 1589248 to 1785856 tid 3 2024-04-20T01:47:43.938 INFO:tasks.rados.rados.0.smithi165.stdout:131: done (5 left) 2024-04-20T01:47:43.938 INFO:tasks.rados.rados.0.smithi165.stdout:135: delete oid 35 current snap is 5 2024-04-20T01:47:43.939 INFO:tasks.rados.rados.0.smithi165.stdout:129: finishing write tid 3 to smithi16579500-9 2024-04-20T01:47:43.939 INFO:tasks.rados.rados.0.smithi165.stdout:129: finishing write tid 4 to smithi16579500-9 2024-04-20T01:47:43.939 INFO:tasks.rados.rados.0.smithi165.stdout:129: oid 9 updating version 0 to 12 2024-04-20T01:47:43.939 INFO:tasks.rados.rados.0.smithi165.stdout:129: oid 9 updating version 12 to 13 2024-04-20T01:47:43.939 INFO:tasks.rados.rados.0.smithi165.stdout:129: oid 9 updating version 13 to 14 2024-04-20T01:47:43.939 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 9 v 14 (ObjNum 68 snap 5 seq_num 68) dirty exists 2024-04-20T01:47:43.939 INFO:tasks.rados.rados.0.smithi165.stdout:129: left oid 9 (ObjNum 68 snap 5 seq_num 68) 2024-04-20T01:47:43.939 INFO:tasks.rados.rados.0.smithi165.stdout:130: finishing copy_from to smithi16579500-24 2024-04-20T01:47:43.939 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 24 v 4 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-04-20T01:47:43.939 INFO:tasks.rados.rados.0.smithi165.stdout:132: finishing write tid 1 to smithi16579500-19 2024-04-20T01:47:43.939 INFO:tasks.rados.rados.0.smithi165.stdout:132: finishing write tid 2 to smithi16579500-19 2024-04-20T01:47:43.939 INFO:tasks.rados.rados.0.smithi165.stdout:132: finishing write tid 3 to smithi16579500-19 2024-04-20T01:47:43.939 INFO:tasks.rados.rados.0.smithi165.stdout:132: finishing write tid 4 to smithi16579500-19 2024-04-20T01:47:43.939 INFO:tasks.rados.rados.0.smithi165.stdout:132: oid 19 updating version 0 to 51 2024-04-20T01:47:43.940 INFO:tasks.rados.rados.0.smithi165.stdout:132: oid 19 version 51 is already newer than 50 2024-04-20T01:47:43.940 INFO:tasks.rados.rados.0.smithi165.stdout:132: oid 19 updating version 51 to 52 2024-04-20T01:47:43.940 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 19 v 52 (ObjNum 69 snap 5 seq_num 69) dirty exists 2024-04-20T01:47:43.940 INFO:tasks.rados.rados.0.smithi165.stdout:132: left oid 19 (ObjNum 69 snap 5 seq_num 69) 2024-04-20T01:47:43.940 INFO:tasks.rados.rados.0.smithi165.stdout:133: finishing write tid 1 to smithi16579500-13 2024-04-20T01:47:43.940 INFO:tasks.rados.rados.0.smithi165.stdout:133: finishing write tid 2 to smithi16579500-13 2024-04-20T01:47:43.940 INFO:tasks.rados.rados.0.smithi165.stdout:133: finishing write tid 3 to smithi16579500-13 2024-04-20T01:47:43.941 INFO:tasks.rados.rados.0.smithi165.stdout:133: finishing write tid 4 to smithi16579500-13 2024-04-20T01:47:43.941 INFO:tasks.rados.rados.0.smithi165.stdout:133: oid 13 updating version 0 to 37 2024-04-20T01:47:43.941 INFO:tasks.rados.rados.0.smithi165.stdout:133: oid 13 version 37 is already newer than 36 2024-04-20T01:47:43.941 INFO:tasks.rados.rados.0.smithi165.stdout:133: oid 13 updating version 37 to 38 2024-04-20T01:47:43.941 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 13 v 38 (ObjNum 70 snap 5 seq_num 70) dirty exists 2024-04-20T01:47:43.941 INFO:tasks.rados.rados.0.smithi165.stdout:133: left oid 13 (ObjNum 70 snap 5 seq_num 70) 2024-04-20T01:47:43.942 INFO:tasks.rados.rados.0.smithi165.stdout:129: done (5 left) 2024-04-20T01:47:43.942 INFO:tasks.rados.rados.0.smithi165.stdout:130: done (4 left) 2024-04-20T01:47:43.942 INFO:tasks.rados.rados.0.smithi165.stdout:132: done (3 left) 2024-04-20T01:47:43.942 INFO:tasks.rados.rados.0.smithi165.stdout:133: done (2 left) 2024-04-20T01:47:43.942 INFO:tasks.rados.rados.0.smithi165.stdout:135: done (1 left) 2024-04-20T01:47:43.942 INFO:tasks.rados.rados.0.smithi165.stdout:136: delete oid 8 current snap is 5 2024-04-20T01:47:43.943 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:47:43.943 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:47:43.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:47:43.943 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:47:43.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.944 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 36/30 ECSubWrite(tid=19, reqid=client.4643.0:218, at_version=36'13, trim_to=0'0, roll_forward_to=34'11))", 2024-04-20T01:47:43.944 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:37.798901+0000", 2024-04-20T01:47:43.944 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.1575850179999998, 2024-04-20T01:47:43.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001480756, 2024-04-20T01:47:43.944 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.944 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.944 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.798901+0000", 2024-04-20T01:47:43.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.798901+0000", 2024-04-20T01:47:43.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.798901+0000", 2024-04-20T01:47:43.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:43.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.798928+0000", 2024-04-20T01:47:43.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7645999999999999e-05 2024-04-20T01:47:43.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.798944+0000", 2024-04-20T01:47:43.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5197999999999999e-05 2024-04-20T01:47:43.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.798957+0000", 2024-04-20T01:47:43.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3542e-05 2024-04-20T01:47:43.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799025+0000", 2024-04-20T01:47:43.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8143000000000001e-05 2024-04-20T01:47:43.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799038+0000", 2024-04-20T01:47:43.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2986000000000001e-05 2024-04-20T01:47:43.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.800356+0000", 2024-04-20T01:47:43.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013179820000000001 2024-04-20T01:47:43.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.800382+0000", 2024-04-20T01:47:43.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5669000000000001e-05 2024-04-20T01:47:43.948 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.948 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.948 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.948 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(osd.8.21:1 3.9s0 3:94399c02:::smithi16579500-3:head [assert-version v4,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:47:43.948 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:37.799515+0000", 2024-04-20T01:47:43.948 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.1569710229999997, 2024-04-20T01:47:43.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075519259999999996, 2024-04-20T01:47:43.948 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.948 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:47:43.949 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799515+0000", 2024-04-20T01:47:43.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799515+0000", 2024-04-20T01:47:43.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799513+0000", 2024-04-20T01:47:43.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:47:43.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799523+0000", 2024-04-20T01:47:43.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4020000000000008e-06 2024-04-20T01:47:43.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799536+0000", 2024-04-20T01:47:43.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3448999999999999e-05 2024-04-20T01:47:43.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799555+0000", 2024-04-20T01:47:43.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9148e-05 2024-04-20T01:47:43.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799607+0000", 2024-04-20T01:47:43.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1898999999999999e-05 2024-04-20T01:47:43.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:47:43.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799615+0000", 2024-04-20T01:47:43.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1170000000000004e-06 2024-04-20T01:47:43.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.801155+0000", 2024-04-20T01:47:43.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001539852 2024-04-20T01:47:43.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:43.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.801260+0000", 2024-04-20T01:47:43.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000105079 2024-04-20T01:47:43.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807067+0000", 2024-04-20T01:47:43.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0058068490000000002 2024-04-20T01:47:43.952 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.952 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.953 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.953 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 36/30 ECSubWrite(tid=20, reqid=client.4643.0:219, at_version=36'14, trim_to=0'0, roll_forward_to=34'11))", 2024-04-20T01:47:43.953 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:37.799936+0000", 2024-04-20T01:47:43.953 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.15655003, 2024-04-20T01:47:43.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0034773870000000002, 2024-04-20T01:47:43.953 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.953 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.953 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799936+0000", 2024-04-20T01:47:43.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799936+0000", 2024-04-20T01:47:43.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799935+0000", 2024-04-20T01:47:43.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:43.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799958+0000", 2024-04-20T01:47:43.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2509000000000001e-05 2024-04-20T01:47:43.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799969+0000", 2024-04-20T01:47:43.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1579999999999999e-05 2024-04-20T01:47:43.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799978+0000", 2024-04-20T01:47:43.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8929999999999994e-06 2024-04-20T01:47:43.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.800029+0000", 2024-04-20T01:47:43.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1116000000000003e-05 2024-04-20T01:47:43.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.800040+0000", 2024-04-20T01:47:43.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0475999999999999e-05 2024-04-20T01:47:43.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.803394+0000", 2024-04-20T01:47:43.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033545239999999998 2024-04-20T01:47:43.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.803414+0000", 2024-04-20T01:47:43.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9108999999999999e-05 2024-04-20T01:47:43.957 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.957 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.957 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.957 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 36/30 ECSubWrite(tid=21, reqid=client.4643.0:220, at_version=36'15, trim_to=0'0, roll_forward_to=34'11))", 2024-04-20T01:47:43.957 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:37.800873+0000", 2024-04-20T01:47:43.957 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.1556130060000003, 2024-04-20T01:47:43.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057101610000000001, 2024-04-20T01:47:43.957 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.958 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.958 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.800873+0000", 2024-04-20T01:47:43.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.800873+0000", 2024-04-20T01:47:43.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.800873+0000", 2024-04-20T01:47:43.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:43.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.800892+0000", 2024-04-20T01:47:43.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9364e-05 2024-04-20T01:47:43.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.800903+0000", 2024-04-20T01:47:43.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0869e-05 2024-04-20T01:47:43.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.800913+0000", 2024-04-20T01:47:43.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9850000000000001e-06 2024-04-20T01:47:43.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.800991+0000", 2024-04-20T01:47:43.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7890000000000001e-05 2024-04-20T01:47:43.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.800999+0000", 2024-04-20T01:47:43.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4009999999999997e-06 2024-04-20T01:47:43.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806566+0000", 2024-04-20T01:47:43.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005567153 2024-04-20T01:47:43.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806583+0000", 2024-04-20T01:47:43.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7164999999999999e-05 2024-04-20T01:47:43.961 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.961 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.961 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.962 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpReadReply(3.9s0 36/30 ECSubReadReply(tid=7, attrs_read=0))", 2024-04-20T01:47:43.962 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:37.803007+0000", 2024-04-20T01:47:43.962 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.1534795779999998, 2024-04-20T01:47:43.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00045319300000000001, 2024-04-20T01:47:43.962 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.962 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.962 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.803007+0000", 2024-04-20T01:47:43.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.803007+0000", 2024-04-20T01:47:43.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.803006+0000", 2024-04-20T01:47:43.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:43.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.803023+0000", 2024-04-20T01:47:43.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6980999999999999e-05 2024-04-20T01:47:43.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.803034+0000", 2024-04-20T01:47:43.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1014e-05 2024-04-20T01:47:43.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.803040+0000", 2024-04-20T01:47:43.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2559999999999999e-06 2024-04-20T01:47:43.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.803424+0000", 2024-04-20T01:47:43.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00038429499999999999 2024-04-20T01:47:43.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.803460+0000", 2024-04-20T01:47:43.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5729000000000003e-05 2024-04-20T01:47:43.965 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.965 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.965 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.965 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpRead(3.9s0 36/30 ECSubRead(tid=7, to_read={3:94399c02:::smithi16579500-3:head=0,12288,0}, subchunks={3:94399c02:::smithi16579500-3:head=[0,1]}, attrs_to_read=))", 2024-04-20T01:47:43.966 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:37.803310+0000", 2024-04-20T01:47:43.966 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.1531765329999999, 2024-04-20T01:47:43.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0030795739999999999, 2024-04-20T01:47:43.966 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.966 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.966 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.803310+0000", 2024-04-20T01:47:43.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.803310+0000", 2024-04-20T01:47:43.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.803310+0000", 2024-04-20T01:47:43.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.803310+0000", 2024-04-20T01:47:43.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.803340+0000", 2024-04-20T01:47:43.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0623000000000001e-05 2024-04-20T01:47:43.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.803350+0000", 2024-04-20T01:47:43.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9569999999999997e-06 2024-04-20T01:47:43.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.803601+0000", 2024-04-20T01:47:43.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000251191 2024-04-20T01:47:43.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806389+0000", 2024-04-20T01:47:43.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027878030000000002 2024-04-20T01:47:43.969 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.969 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.969 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.969 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpReadReply(3.9s0 36/30 ECSubReadReply(tid=7, attrs_read=0))", 2024-04-20T01:47:43.969 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:37.804078+0000", 2024-04-20T01:47:43.969 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.1524080530000003, 2024-04-20T01:47:43.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002688349, 2024-04-20T01:47:43.969 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.969 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.969 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.804078+0000", 2024-04-20T01:47:43.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.804078+0000", 2024-04-20T01:47:43.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.804077+0000", 2024-04-20T01:47:43.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:43.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.804090+0000", 2024-04-20T01:47:43.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2295e-05 2024-04-20T01:47:43.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.804102+0000", 2024-04-20T01:47:43.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2126e-05 2024-04-20T01:47:43.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.804110+0000", 2024-04-20T01:47:43.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7360000000000001e-06 2024-04-20T01:47:43.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806725+0000", 2024-04-20T01:47:43.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0026152760000000001 2024-04-20T01:47:43.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806767+0000", 2024-04-20T01:47:43.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1613000000000003e-05 2024-04-20T01:47:43.973 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.973 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.973 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.973 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpReadReply(3.9s0 36/30 ECSubReadReply(tid=7, attrs_read=0))", 2024-04-20T01:47:43.973 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:37.806377+0000", 2024-04-20T01:47:43.973 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.1501090899999999, 2024-04-20T01:47:43.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00045939199999999998, 2024-04-20T01:47:43.973 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.973 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.973 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806377+0000", 2024-04-20T01:47:43.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806377+0000", 2024-04-20T01:47:43.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806376+0000", 2024-04-20T01:47:43.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:43.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806394+0000", 2024-04-20T01:47:43.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7595999999999999e-05 2024-04-20T01:47:43.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806406+0000", 2024-04-20T01:47:43.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2186e-05 2024-04-20T01:47:43.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806413+0000", 2024-04-20T01:47:43.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.306e-06 2024-04-20T01:47:43.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806803+0000", 2024-04-20T01:47:43.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00038968599999999998 2024-04-20T01:47:43.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806837+0000", 2024-04-20T01:47:43.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3488999999999997e-05 2024-04-20T01:47:43.976 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.976 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpReadReply(3.9s0 36/30 ECSubReadReply(tid=7, attrs_read=0))", 2024-04-20T01:47:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:37.806377+0000", 2024-04-20T01:47:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.150108951, 2024-04-20T01:47:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00070629799999999995, 2024-04-20T01:47:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806377+0000", 2024-04-20T01:47:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806377+0000", 2024-04-20T01:47:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806377+0000", 2024-04-20T01:47:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806377+0000", 2024-04-20T01:47:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806394+0000", 2024-04-20T01:47:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7119999999999999e-05 2024-04-20T01:47:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806402+0000", 2024-04-20T01:47:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6129999999999998e-06 2024-04-20T01:47:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.806860+0000", 2024-04-20T01:47:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000457527 2024-04-20T01:47:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807084+0000", 2024-04-20T01:47:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000224038 2024-04-20T01:47:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 36/30 ECSubWrite(tid=22, reqid=client.4643.0:220, at_version=0'0, trim_to=0'0, roll_forward_to=36'15))", 2024-04-20T01:47:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:37.807075+0000", 2024-04-20T01:47:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.1494115330000003, 2024-04-20T01:47:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047601099999999997, 2024-04-20T01:47:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807075+0000", 2024-04-20T01:47:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807075+0000", 2024-04-20T01:47:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807074+0000", 2024-04-20T01:47:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807087+0000", 2024-04-20T01:47:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2943e-05 2024-04-20T01:47:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807099+0000", 2024-04-20T01:47:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2201000000000001e-05 2024-04-20T01:47:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807108+0000", 2024-04-20T01:47:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1449999999999997e-06 2024-04-20T01:47:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807158+0000", 2024-04-20T01:47:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9512999999999998e-05 2024-04-20T01:47:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807167+0000", 2024-04-20T01:47:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4509999999999998e-06 2024-04-20T01:47:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.811809+0000", 2024-04-20T01:47:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046420539999999996 2024-04-20T01:47:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.811835+0000", 2024-04-20T01:47:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5567000000000001e-05 2024-04-20T01:47:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.ds4 37/30 ECSubWrite(tid=16, reqid=client.4643.0:243, at_version=37'8, trim_to=0'0, roll_forward_to=34'6))", 2024-04-20T01:47:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.788588+0000", 2024-04-20T01:47:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.1678982619999996, 2024-04-20T01:47:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0015469730000000001, 2024-04-20T01:47:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.788588+0000", 2024-04-20T01:47:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.788588+0000", 2024-04-20T01:47:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.788588+0000", 2024-04-20T01:47:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:47:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.788615+0000", 2024-04-20T01:47:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6865000000000001e-05 2024-04-20T01:47:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.788628+0000", 2024-04-20T01:47:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3213999999999999e-05 2024-04-20T01:47:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.788657+0000", 2024-04-20T01:47:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9127000000000001e-05 2024-04-20T01:47:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.788765+0000", 2024-04-20T01:47:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010789999999999999 2024-04-20T01:47:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.788777+0000", 2024-04-20T01:47:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1788e-05 2024-04-20T01:47:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.790097+0000", 2024-04-20T01:47:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013198870000000001 2024-04-20T01:47:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.790135+0000", 2024-04-20T01:47:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8358000000000001e-05 2024-04-20T01:47:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s1 37/30 ECSubWrite(tid=18, reqid=client.4643.0:234, at_version=37'8, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.792550+0000", 2024-04-20T01:47:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.1639359520000001, 2024-04-20T01:47:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001146024, 2024-04-20T01:47:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.792550+0000", 2024-04-20T01:47:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.792550+0000", 2024-04-20T01:47:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.792550+0000", 2024-04-20T01:47:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.792572+0000", 2024-04-20T01:47:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2322000000000001e-05 2024-04-20T01:47:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.792581+0000", 2024-04-20T01:47:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8389999999999996e-06 2024-04-20T01:47:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.792590+0000", 2024-04-20T01:47:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0780000000000002e-06 2024-04-20T01:47:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.792681+0000", 2024-04-20T01:47:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0985999999999999e-05 2024-04-20T01:47:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.792693+0000", 2024-04-20T01:47:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2173e-05 2024-04-20T01:47:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793663+0000", 2024-04-20T01:47:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00096949600000000003 2024-04-20T01:47:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793696+0000", 2024-04-20T01:47:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3497999999999997e-05 2024-04-20T01:47:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s1 37/30 ECSubWrite(tid=19, reqid=client.4643.0:235, at_version=37'9, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.794307+0000", 2024-04-20T01:47:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.162178881, 2024-04-20T01:47:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0022373760000000001, 2024-04-20T01:47:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794307+0000", 2024-04-20T01:47:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794307+0000", 2024-04-20T01:47:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794306+0000", 2024-04-20T01:47:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794330+0000", 2024-04-20T01:47:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3292e-05 2024-04-20T01:47:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794345+0000", 2024-04-20T01:47:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5480000000000001e-05 2024-04-20T01:47:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794355+0000", 2024-04-20T01:47:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6020000000000006e-06 2024-04-20T01:47:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794404+0000", 2024-04-20T01:47:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9064999999999999e-05 2024-04-20T01:47:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794411+0000", 2024-04-20T01:47:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3799999999999996e-06 2024-04-20T01:47:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796516+0000", 2024-04-20T01:47:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021050489999999999 2024-04-20T01:47:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796545+0000", 2024-04-20T01:47:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8413000000000001e-05 2024-04-20T01:47:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s1 37/30 ECSubWrite(tid=20, reqid=client.4643.0:236, at_version=37'10, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.795383+0000", 2024-04-20T01:47:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.1611034990000002, 2024-04-20T01:47:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039654169999999997, 2024-04-20T01:47:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795383+0000", 2024-04-20T01:47:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795383+0000", 2024-04-20T01:47:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795382+0000", 2024-04-20T01:47:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795398+0000", 2024-04-20T01:47:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5746000000000001e-05 2024-04-20T01:47:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795410+0000", 2024-04-20T01:47:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2627e-05 2024-04-20T01:47:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795417+0000", 2024-04-20T01:47:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.618e-06 2024-04-20T01:47:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795463+0000", 2024-04-20T01:47:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5782000000000003e-05 2024-04-20T01:47:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795472+0000", 2024-04-20T01:47:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0550000000000005e-06 2024-04-20T01:47:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799323+0000", 2024-04-20T01:47:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003850961 2024-04-20T01:47:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799348+0000", 2024-04-20T01:47:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5398999999999999e-05 2024-04-20T01:47:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.ds4 37/30 ECSubWrite(tid=17, reqid=client.4643.0:243, at_version=0'0, trim_to=0'0, roll_forward_to=37'8))", 2024-04-20T01:47:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.801026+0000", 2024-04-20T01:47:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.1554597769999999, 2024-04-20T01:47:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0014129769999999999, 2024-04-20T01:47:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.801026+0000", 2024-04-20T01:47:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.801026+0000", 2024-04-20T01:47:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.801025+0000", 2024-04-20T01:47:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.801042+0000", 2024-04-20T01:47:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6991e-05 2024-04-20T01:47:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.801075+0000", 2024-04-20T01:47:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2861000000000003e-05 2024-04-20T01:47:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.801086+0000", 2024-04-20T01:47:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0657e-05 2024-04-20T01:47:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.801190+0000", 2024-04-20T01:47:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000104459 2024-04-20T01:47:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.801231+0000", 2024-04-20T01:47:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0667000000000002e-05 2024-04-20T01:47:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.802406+0000", 2024-04-20T01:47:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001175597 2024-04-20T01:47:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.802439+0000", 2024-04-20T01:47:44.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2969000000000002e-05 2024-04-20T01:47:44.007 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.007 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.007 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.007 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s1 37/30 ECSubWrite(tid=21, reqid=client.4643.0:236, at_version=0'0, trim_to=0'0, roll_forward_to=37'10))", 2024-04-20T01:47:44.007 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.807000+0000", 2024-04-20T01:47:44.007 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.1494865990000003, 2024-04-20T01:47:44.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083304400000000002, 2024-04-20T01:47:44.007 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.007 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.807000+0000", 2024-04-20T01:47:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.807000+0000", 2024-04-20T01:47:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806999+0000", 2024-04-20T01:47:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.807011+0000", 2024-04-20T01:47:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2028000000000001e-05 2024-04-20T01:47:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.807024+0000", 2024-04-20T01:47:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3708e-05 2024-04-20T01:47:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.807034+0000", 2024-04-20T01:47:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9299999999999998e-06 2024-04-20T01:47:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.807090+0000", 2024-04-20T01:47:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5562000000000001e-05 2024-04-20T01:47:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.807100+0000", 2024-04-20T01:47:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0458999999999999e-05 2024-04-20T01:47:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.807809+0000", 2024-04-20T01:47:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00070845999999999995 2024-04-20T01:47:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.807833+0000", 2024-04-20T01:47:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3855000000000001e-05 2024-04-20T01:47:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s1 38/30 ECSubWrite(tid=22, reqid=osd.3.0:22, at_version=38'12, trim_to=0'0, roll_forward_to=37'10))", 2024-04-20T01:47:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.783934+0000", 2024-04-20T01:47:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.1725517979999998, 2024-04-20T01:47:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0034964699999999998, 2024-04-20T01:47:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.783934+0000", 2024-04-20T01:47:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.783934+0000", 2024-04-20T01:47:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.783934+0000", 2024-04-20T01:47:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:47:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.783956+0000", 2024-04-20T01:47:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.192e-05 2024-04-20T01:47:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.783969+0000", 2024-04-20T01:47:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3156000000000001e-05 2024-04-20T01:47:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.784000+0000", 2024-04-20T01:47:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0938000000000001e-05 2024-04-20T01:47:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.784104+0000", 2024-04-20T01:47:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103427 2024-04-20T01:47:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.784116+0000", 2024-04-20T01:47:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2604e-05 2024-04-20T01:47:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.787400+0000", 2024-04-20T01:47:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0032841760000000002 2024-04-20T01:47:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.787431+0000", 2024-04-20T01:47:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0443999999999999e-05 2024-04-20T01:47:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.ds4 38/30 ECSubWrite(tid=18, reqid=osd.6.0:18, at_version=38'10, trim_to=0'0, roll_forward_to=37'8))", 2024-04-20T01:47:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.786821+0000", 2024-04-20T01:47:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.1696653750000001, 2024-04-20T01:47:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0061385989999999998, 2024-04-20T01:47:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.786821+0000", 2024-04-20T01:47:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.786821+0000", 2024-04-20T01:47:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.786821+0000", 2024-04-20T01:47:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.786843+0000", 2024-04-20T01:47:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2435000000000001e-05 2024-04-20T01:47:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.786855+0000", 2024-04-20T01:47:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2424e-05 2024-04-20T01:47:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.786867+0000", 2024-04-20T01:47:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1134999999999999e-05 2024-04-20T01:47:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.786927+0000", 2024-04-20T01:47:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0532999999999998e-05 2024-04-20T01:47:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.786943+0000", 2024-04-20T01:47:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5659e-05 2024-04-20T01:47:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792928+0000", 2024-04-20T01:47:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059853750000000002 2024-04-20T01:47:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792959+0000", 2024-04-20T01:47:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1316000000000003e-05 2024-04-20T01:47:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s1 38/30 ECSubWrite(tid=23, reqid=osd.3.0:22, at_version=0'0, trim_to=0'0, roll_forward_to=38'12))", 2024-04-20T01:47:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.788257+0000", 2024-04-20T01:47:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.1682296240000003, 2024-04-20T01:47:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0045893369999999998, 2024-04-20T01:47:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788257+0000", 2024-04-20T01:47:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788257+0000", 2024-04-20T01:47:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788256+0000", 2024-04-20T01:47:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788271+0000", 2024-04-20T01:47:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4907e-05 2024-04-20T01:47:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788287+0000", 2024-04-20T01:47:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6832000000000001e-05 2024-04-20T01:47:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788298+0000", 2024-04-20T01:47:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0504e-05 2024-04-20T01:47:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788347+0000", 2024-04-20T01:47:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9397999999999997e-05 2024-04-20T01:47:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788358+0000", 2024-04-20T01:47:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759e-05 2024-04-20T01:47:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792813+0000", 2024-04-20T01:47:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044547650000000003 2024-04-20T01:47:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792846+0000", 2024-04-20T01:47:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3114000000000002e-05 2024-04-20T01:47:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.ds4 38/30 ECSubWrite(tid=19, reqid=osd.6.0:18, at_version=0'0, trim_to=0'0, roll_forward_to=38'10))", 2024-04-20T01:47:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.794393+0000", 2024-04-20T01:47:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.1620935970000001, 2024-04-20T01:47:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00092344300000000003, 2024-04-20T01:47:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794393+0000", 2024-04-20T01:47:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794393+0000", 2024-04-20T01:47:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794392+0000", 2024-04-20T01:47:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794405+0000", 2024-04-20T01:47:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3050999999999999e-05 2024-04-20T01:47:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794417+0000", 2024-04-20T01:47:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2174999999999999e-05 2024-04-20T01:47:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794429+0000", 2024-04-20T01:47:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1747e-05 2024-04-20T01:47:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794511+0000", 2024-04-20T01:47:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2577000000000006e-05 2024-04-20T01:47:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794521+0000", 2024-04-20T01:47:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8609999999999993e-06 2024-04-20T01:47:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.795288+0000", 2024-04-20T01:47:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00076647 2024-04-20T01:47:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.795316+0000", 2024-04-20T01:47:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8518999999999999e-05 2024-04-20T01:47:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.029 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:47:44.030 INFO:tasks.rados.rados.0.smithi165.stdout:134: finishing write tid 1 to smithi16579500-36 2024-04-20T01:47:44.030 INFO:tasks.rados.rados.0.smithi165.stdout:134: finishing write tid 2 to smithi16579500-36 2024-04-20T01:47:44.030 INFO:tasks.rados.rados.0.smithi165.stdout:134: finishing write tid 3 to smithi16579500-36 2024-04-20T01:47:44.031 INFO:tasks.rados.rados.0.smithi165.stdout:134: finishing write tid 4 to smithi16579500-36 2024-04-20T01:47:44.031 INFO:tasks.rados.rados.0.smithi165.stdout:136: done (1 left) 2024-04-20T01:47:44.031 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 24 current snap is 5 2024-04-20T01:47:44.031 INFO:tasks.rados.rados.0.smithi165.stdout:137: seq_num 74 ranges {524288=475136,999424=622592,1622016=147456} 2024-04-20T01:47:44.031 INFO:tasks.rados.rados.0.smithi165.stdout:137: writing smithi16579500-24 from 524288 to 999424 tid 1 2024-04-20T01:47:44.032 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-20T01:47:44.034 INFO:tasks.rados.rados.0.smithi165.stdout:137: writing smithi16579500-24 from 999424 to 1622016 tid 2 2024-04-20T01:47:44.034 INFO:tasks.rados.rados.0.smithi165.stdout:137: writing smithi16579500-24 from 1622016 to 1769472 tid 3 2024-04-20T01:47:44.034 INFO:tasks.rados.rados.0.smithi165.stdout:134: finishing write tid 5 to smithi16579500-36 2024-04-20T01:47:44.034 INFO:tasks.rados.rados.0.smithi165.stdout:134: oid 36 updating version 0 to 21 2024-04-20T01:47:44.034 INFO:tasks.rados.rados.0.smithi165.stdout:134: oid 36 updating version 21 to 22 2024-04-20T01:47:44.034 INFO:tasks.rados.rados.0.smithi165.stdout:134: oid 36 updating version 22 to 24 2024-04-20T01:47:44.034 INFO:tasks.rados.rados.0.smithi165.stdout:134: oid 36 version 24 is already newer than 23 2024-04-20T01:47:44.034 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 36 v 24 (ObjNum 71 snap 5 seq_num 71) dirty exists 2024-04-20T01:47:44.034 INFO:tasks.rados.rados.0.smithi165.stdout:134: left oid 36 (ObjNum 71 snap 5 seq_num 71) 2024-04-20T01:47:44.034 INFO:tasks.rados.rados.0.smithi165.stdout:134: done (1 left) 2024-04-20T01:47:44.034 INFO:tasks.rados.rados.0.smithi165.stdout:138: snap_create 2024-04-20T01:47:44.035 INFO:tasks.rados.rados.0.smithi165.stdout:137: finishing write tid 1 to smithi16579500-24 2024-04-20T01:47:44.035 INFO:tasks.rados.rados.0.smithi165.stdout:137: finishing write tid 2 to smithi16579500-24 2024-04-20T01:47:44.035 INFO:tasks.rados.rados.0.smithi165.stdout:137: finishing write tid 3 to smithi16579500-24 2024-04-20T01:47:44.035 INFO:tasks.rados.rados.0.smithi165.stdout:137: finishing write tid 4 to smithi16579500-24 2024-04-20T01:47:44.035 INFO:tasks.rados.rados.0.smithi165.stdout:137: finishing write tid 5 to smithi16579500-24 2024-04-20T01:47:44.035 INFO:tasks.rados.rados.0.smithi165.stdout:137: oid 24 updating version 0 to 20 2024-04-20T01:47:44.035 INFO:tasks.rados.rados.0.smithi165.stdout:137: oid 24 version 20 is already newer than 17 2024-04-20T01:47:44.035 INFO:tasks.rados.rados.0.smithi165.stdout:137: oid 24 version 20 is already newer than 19 2024-04-20T01:47:44.035 INFO:tasks.rados.rados.0.smithi165.stdout:137: oid 24 version 20 is already newer than 18 2024-04-20T01:47:44.035 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 24 v 20 (ObjNum 74 snap 5 seq_num 74) dirty exists 2024-04-20T01:47:44.035 INFO:tasks.rados.rados.0.smithi165.stdout:137: left oid 24 (ObjNum 74 snap 5 seq_num 74) 2024-04-20T01:47:44.074 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:44.088+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:47:44.135 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:47:44.150+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:47:44.205 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:47:44.205 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:47:44.205 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-04-20T01:47:44.206 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:47:44.213 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-20T01:47:44.236 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:44.250+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:47:44.337 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:44.351+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:44.341 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:47:44.341 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:47:44.341 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:47:44.341 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:47:44.342 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:47:44.351 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-20T01:47:44.438 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:44.453+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:44.482 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:47:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:47:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:47:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:47:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.bs4 33/30 ECSubWrite(tid=20, reqid=client.4643.0:91, at_version=33'7, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.807872+0000", 2024-04-20T01:47:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.687151965, 2024-04-20T01:47:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0072275780000000001, 2024-04-20T01:47:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.807872+0000", 2024-04-20T01:47:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.807872+0000", 2024-04-20T01:47:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.807871+0000", 2024-04-20T01:47:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.808550+0000", 2024-04-20T01:47:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00067848400000000003 2024-04-20T01:47:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.808561+0000", 2024-04-20T01:47:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1129e-05 2024-04-20T01:47:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.808572+0000", 2024-04-20T01:47:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0839000000000001e-05 2024-04-20T01:47:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.808612+0000", 2024-04-20T01:47:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0567e-05 2024-04-20T01:47:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.808621+0000", 2024-04-20T01:47:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3780000000000001e-06 2024-04-20T01:47:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.815073+0000", 2024-04-20T01:47:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0064526510000000002 2024-04-20T01:47:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.815100+0000", 2024-04-20T01:47:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6163e-05 2024-04-20T01:47:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:35 3.es0 3:79bd6b86:::smithi16579500-12:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962142+0000", 2024-04-20T01:47:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.5328819409999994, 2024-04-20T01:47:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0073088650000000003, 2024-04-20T01:47:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:47:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:47:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 35 2024-04-20T01:47:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962142+0000", 2024-04-20T01:47:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962142+0000", 2024-04-20T01:47:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962142+0000", 2024-04-20T01:47:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962145+0000", 2024-04-20T01:47:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4640000000000002e-06 2024-04-20T01:47:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962150+0000", 2024-04-20T01:47:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0309999999999998e-06 2024-04-20T01:47:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962154+0000", 2024-04-20T01:47:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7000000000000002e-06 2024-04-20T01:47:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963611+0000", 2024-04-20T01:47:44.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0014569660000000001 2024-04-20T01:47:44.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:44.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963727+0000", 2024-04-20T01:47:44.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116284 2024-04-20T01:47:44.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:44.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.964080+0000", 2024-04-20T01:47:44.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00035286999999999997 2024-04-20T01:47:44.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:44.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.965987+0000", 2024-04-20T01:47:44.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001906781 2024-04-20T01:47:44.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:47:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.969392+0000", 2024-04-20T01:47:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00340573 2024-04-20T01:47:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.969451+0000", 2024-04-20T01:47:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.842e-05 2024-04-20T01:47:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:36 3.es0 3:79bd6b86:::smithi16579500-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962173+0000", 2024-04-20T01:47:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.532850818, 2024-04-20T01:47:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090032729999999991, 2024-04-20T01:47:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:47:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:47:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:47:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:44.499 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 36 2024-04-20T01:47:44.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.499 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962173+0000", 2024-04-20T01:47:44.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962173+0000", 2024-04-20T01:47:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962173+0000", 2024-04-20T01:47:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962175+0000", 2024-04-20T01:47:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9269999999999999e-06 2024-04-20T01:47:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962179+0000", 2024-04-20T01:47:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0300000000000004e-06 2024-04-20T01:47:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962183+0000", 2024-04-20T01:47:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7340000000000002e-06 2024-04-20T01:47:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.964291+0000", 2024-04-20T01:47:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021081390000000002 2024-04-20T01:47:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:47:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.964348+0000", 2024-04-20T01:47:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7218e-05 2024-04-20T01:47:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.969516+0000", 2024-04-20T01:47:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0051684540000000003 2024-04-20T01:47:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.969602+0000", 2024-04-20T01:47:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5294e-05 2024-04-20T01:47:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.971176+0000", 2024-04-20T01:47:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001574719 2024-04-20T01:47:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:103 3.es0 3:78f9000f:::smithi16579500-35:head [append 0~573440,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962194+0000", 2024-04-20T01:47:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.5328303569999999, 2024-04-20T01:47:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0087917459999999996, 2024-04-20T01:47:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:47:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:47:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 103 2024-04-20T01:47:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962194+0000", 2024-04-20T01:47:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962194+0000", 2024-04-20T01:47:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962193+0000", 2024-04-20T01:47:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962426+0000", 2024-04-20T01:47:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023287700000000001 2024-04-20T01:47:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962434+0000", 2024-04-20T01:47:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4590000000000001e-06 2024-04-20T01:47:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962438+0000", 2024-04-20T01:47:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0929999999999996e-06 2024-04-20T01:47:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.964426+0000", 2024-04-20T01:47:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019882620000000002 2024-04-20T01:47:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.964549+0000", 2024-04-20T01:47:44.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012331399999999999 2024-04-20T01:47:44.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:44.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.965303+0000", 2024-04-20T01:47:44.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00075378299999999995 2024-04-20T01:47:44.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.969193+0000", 2024-04-20T01:47:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0038896400000000002 2024-04-20T01:47:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:47:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970919+0000", 2024-04-20T01:47:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001726521 2024-04-20T01:47:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970985+0000", 2024-04-20T01:47:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6062000000000005e-05 2024-04-20T01:47:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:104 3.es0 3:78f9000f:::smithi16579500-35:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:47:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.5325815120000001, 2024-04-20T01:47:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012120811, 2024-04-20T01:47:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:47:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:47:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 104 2024-04-20T01:47:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:47:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:47:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:47:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962445+0000", 2024-04-20T01:47:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3210000000000001e-06 2024-04-20T01:47:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962451+0000", 2024-04-20T01:47:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.31e-06 2024-04-20T01:47:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962457+0000", 2024-04-20T01:47:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4819999999999999e-06 2024-04-20T01:47:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966434+0000", 2024-04-20T01:47:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039772239999999997 2024-04-20T01:47:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966501+0000", 2024-04-20T01:47:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6947999999999997e-05 2024-04-20T01:47:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966766+0000", 2024-04-20T01:47:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00026468999999999999 2024-04-20T01:47:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970614+0000", 2024-04-20T01:47:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0038481330000000001 2024-04-20T01:47:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:47:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974486+0000", 2024-04-20T01:47:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003871799 2024-04-20T01:47:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974563+0000", 2024-04-20T01:47:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7416999999999994e-05 2024-04-20T01:47:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:105 3.es0 3:78f9000f:::smithi16579500-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962461+0000", 2024-04-20T01:47:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.5325633019999998, 2024-04-20T01:47:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014134643000000001, 2024-04-20T01:47:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:47:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:47:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:47:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 105 2024-04-20T01:47:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962461+0000", 2024-04-20T01:47:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962461+0000", 2024-04-20T01:47:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962460+0000", 2024-04-20T01:47:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962462+0000", 2024-04-20T01:47:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.897e-06 2024-04-20T01:47:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962466+0000", 2024-04-20T01:47:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4520000000000002e-06 2024-04-20T01:47:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962468+0000", 2024-04-20T01:47:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6570000000000001e-06 2024-04-20T01:47:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966956+0000", 2024-04-20T01:47:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044879730000000001 2024-04-20T01:47:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:47:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.967029+0000", 2024-04-20T01:47:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2657999999999995e-05 2024-04-20T01:47:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974602+0000", 2024-04-20T01:47:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075729370000000001 2024-04-20T01:47:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974650+0000", 2024-04-20T01:47:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8000000000000001e-05 2024-04-20T01:47:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976595+0000", 2024-04-20T01:47:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019453789999999999 2024-04-20T01:47:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:109 3.es0 3:7f925225:::smithi16579500-37:head [append 0~606208,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:47:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.532552871, 2024-04-20T01:47:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012453573000000001, 2024-04-20T01:47:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:47:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:47:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 109 2024-04-20T01:47:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:47:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:47:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:47:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:47:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962732+0000", 2024-04-20T01:47:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00026073200000000001 2024-04-20T01:47:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962742+0000", 2024-04-20T01:47:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0735e-05 2024-04-20T01:47:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962749+0000", 2024-04-20T01:47:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.365e-06 2024-04-20T01:47:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.967197+0000", 2024-04-20T01:47:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044487119999999996 2024-04-20T01:47:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.967324+0000", 2024-04-20T01:47:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126575 2024-04-20T01:47:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968044+0000", 2024-04-20T01:47:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00072015399999999998 2024-04-20T01:47:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970795+0000", 2024-04-20T01:47:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027507650000000001 2024-04-20T01:47:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:47:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974884+0000", 2024-04-20T01:47:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040886469999999999 2024-04-20T01:47:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974925+0000", 2024-04-20T01:47:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1094000000000003e-05 2024-04-20T01:47:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:110 3.es0 3:7f925225:::smithi16579500-37:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962755+0000", 2024-04-20T01:47:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.5322690009999995, 2024-04-20T01:47:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012356723999999999, 2024-04-20T01:47:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:47:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:47:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 110 2024-04-20T01:47:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962755+0000", 2024-04-20T01:47:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962755+0000", 2024-04-20T01:47:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962754+0000", 2024-04-20T01:47:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962758+0000", 2024-04-20T01:47:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8750000000000002e-06 2024-04-20T01:47:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962765+0000", 2024-04-20T01:47:44.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2370000000000003e-06 2024-04-20T01:47:44.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962770+0000", 2024-04-20T01:47:44.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9030000000000003e-06 2024-04-20T01:47:44.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968614+0000", 2024-04-20T01:47:44.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0058437619999999997 2024-04-20T01:47:44.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:44.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968676+0000", 2024-04-20T01:47:44.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1592000000000003e-05 2024-04-20T01:47:44.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:44.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968935+0000", 2024-04-20T01:47:44.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025953599999999999 2024-04-20T01:47:44.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:44.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970828+0000", 2024-04-20T01:47:44.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0018928510000000001 2024-04-20T01:47:44.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:47:44.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975036+0000", 2024-04-20T01:47:44.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042079680000000003 2024-04-20T01:47:44.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975112+0000", 2024-04-20T01:47:44.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5558999999999999e-05 2024-04-20T01:47:44.546 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.546 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.546 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.547 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:111 3.es0 3:7f925225:::smithi16579500-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:44.547 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962775+0000", 2024-04-20T01:47:44.547 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.5322492459999992, 2024-04-20T01:47:44.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014337756, 2024-04-20T01:47:44.547 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.547 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:47:44.547 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:47:44.547 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:47:44.547 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:44.547 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 111 2024-04-20T01:47:44.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.548 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962775+0000", 2024-04-20T01:47:44.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962775+0000", 2024-04-20T01:47:44.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962774+0000", 2024-04-20T01:47:44.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:44.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962777+0000", 2024-04-20T01:47:44.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7839999999999999e-06 2024-04-20T01:47:44.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962784+0000", 2024-04-20T01:47:44.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9809999999999997e-06 2024-04-20T01:47:44.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962790+0000", 2024-04-20T01:47:44.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5509999999999999e-06 2024-04-20T01:47:44.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970564+0000", 2024-04-20T01:47:44.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077740680000000003 2024-04-20T01:47:44.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:47:44.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970609+0000", 2024-04-20T01:47:44.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5059999999999999e-05 2024-04-20T01:47:44.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975165+0000", 2024-04-20T01:47:44.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0045564999999999998 2024-04-20T01:47:44.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:44.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975263+0000", 2024-04-20T01:47:44.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7689000000000006e-05 2024-04-20T01:47:44.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977112+0000", 2024-04-20T01:47:44.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001849552 2024-04-20T01:47:44.553 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.553 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.553 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.553 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:142 3.es0 3:797661d0:::smithi16579500-48:head [append 0~425984,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:44.553 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962794+0000", 2024-04-20T01:47:44.554 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.5322300450000004, 2024-04-20T01:47:44.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013969364, 2024-04-20T01:47:44.554 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.554 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:44.554 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:47:44.554 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:47:44.554 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:44.554 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 142 2024-04-20T01:47:44.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.554 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962794+0000", 2024-04-20T01:47:44.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962794+0000", 2024-04-20T01:47:44.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962793+0000", 2024-04-20T01:47:44.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:44.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963014+0000", 2024-04-20T01:47:44.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000221015 2024-04-20T01:47:44.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963025+0000", 2024-04-20T01:47:44.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0739e-05 2024-04-20T01:47:44.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963032+0000", 2024-04-20T01:47:44.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7580000000000003e-06 2024-04-20T01:47:44.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.971338+0000", 2024-04-20T01:47:44.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008305659 2024-04-20T01:47:44.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:44.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.971477+0000", 2024-04-20T01:47:44.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013920100000000001 2024-04-20T01:47:44.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:44.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.972169+0000", 2024-04-20T01:47:44.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00069257800000000001 2024-04-20T01:47:44.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:44.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974577+0000", 2024-04-20T01:47:44.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024072820000000002 2024-04-20T01:47:44.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:47:44.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976694+0000", 2024-04-20T01:47:44.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021172629999999999 2024-04-20T01:47:44.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976763+0000", 2024-04-20T01:47:44.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9259999999999998e-05 2024-04-20T01:47:44.559 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.559 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.559 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.560 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:143 3.es0 3:797661d0:::smithi16579500-48:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:44.560 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.963039+0000", 2024-04-20T01:47:44.560 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.5319852899999997, 2024-04-20T01:47:44.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013923471999999999, 2024-04-20T01:47:44.560 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.560 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:44.560 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:47:44.560 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:47:44.561 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:44.561 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 143 2024-04-20T01:47:44.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.561 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963039+0000", 2024-04-20T01:47:44.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963039+0000", 2024-04-20T01:47:44.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963038+0000", 2024-04-20T01:47:44.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:44.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963210+0000", 2024-04-20T01:47:44.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017155800000000001 2024-04-20T01:47:44.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963217+0000", 2024-04-20T01:47:44.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7309999999999993e-06 2024-04-20T01:47:44.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963230+0000", 2024-04-20T01:47:44.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.295e-05 2024-04-20T01:47:44.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.972654+0000", 2024-04-20T01:47:44.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094237369999999997 2024-04-20T01:47:44.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:44.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.972720+0000", 2024-04-20T01:47:44.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5958999999999996e-05 2024-04-20T01:47:44.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:44.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.973225+0000", 2024-04-20T01:47:44.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00050500399999999998 2024-04-20T01:47:44.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:44.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974578+0000", 2024-04-20T01:47:44.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001353295 2024-04-20T01:47:44.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:47:44.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976919+0000", 2024-04-20T01:47:44.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023405840000000002 2024-04-20T01:47:44.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976962+0000", 2024-04-20T01:47:44.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3226000000000003e-05 2024-04-20T01:47:44.566 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.566 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.566 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.566 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:144 3.es0 3:797661d0:::smithi16579500-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:44.566 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:47:44.567 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.5317866559999995, 2024-04-20T01:47:44.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014456992, 2024-04-20T01:47:44.567 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.567 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:44.567 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:47:44.567 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:47:44.567 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:44.567 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 144 2024-04-20T01:47:44.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.567 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:47:44.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:47:44.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:47:44.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:44.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963240+0000", 2024-04-20T01:47:44.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6459999999999999e-06 2024-04-20T01:47:44.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963247+0000", 2024-04-20T01:47:44.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9650000000000002e-06 2024-04-20T01:47:44.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963253+0000", 2024-04-20T01:47:44.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1900000000000003e-06 2024-04-20T01:47:44.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.973649+0000", 2024-04-20T01:47:44.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010396865999999999 2024-04-20T01:47:44.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:44.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.973745+0000", 2024-04-20T01:47:44.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5345000000000003e-05 2024-04-20T01:47:44.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:44.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974151+0000", 2024-04-20T01:47:44.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000405893 2024-04-20T01:47:44.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:44.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976606+0000", 2024-04-20T01:47:44.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024551730000000002 2024-04-20T01:47:44.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:47:44.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977622+0000", 2024-04-20T01:47:44.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0010159570000000001 2024-04-20T01:47:44.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977694+0000", 2024-04-20T01:47:44.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2469e-05 2024-04-20T01:47:44.572 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.572 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.572 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.572 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:145 3.es0 3:797661d0:::smithi16579500-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:44.572 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:47:44.572 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.5317648350000006, 2024-04-20T01:47:44.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016339408999999999, 2024-04-20T01:47:44.572 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.572 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:47:44.573 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:47:44.573 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:47:44.573 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:44.573 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 145 2024-04-20T01:47:44.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.573 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:47:44.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:47:44.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:47:44.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:44.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963267+0000", 2024-04-20T01:47:44.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2490000000000001e-06 2024-04-20T01:47:44.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963274+0000", 2024-04-20T01:47:44.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7599999999999997e-06 2024-04-20T01:47:44.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963279+0000", 2024-04-20T01:47:44.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5400000000000003e-06 2024-04-20T01:47:44.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977300+0000", 2024-04-20T01:47:44.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014020608 2024-04-20T01:47:44.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:47:44.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977365+0000", 2024-04-20T01:47:44.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5617999999999994e-05 2024-04-20T01:47:44.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.978057+0000", 2024-04-20T01:47:44.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00069179099999999998 2024-04-20T01:47:44.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:44.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.978136+0000", 2024-04-20T01:47:44.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9132000000000001e-05 2024-04-20T01:47:44.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.979598+0000", 2024-04-20T01:47:44.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0014621599999999999 2024-04-20T01:47:44.577 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.577 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.577 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.577 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.bs4 37/30 ECSubWrite(tid=65, reqid=client.4643.0:226, at_version=37'17, trim_to=0'0, roll_forward_to=36'14))", 2024-04-20T01:47:44.577 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.790236+0000", 2024-04-20T01:47:44.577 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.7047882310000002, 2024-04-20T01:47:44.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091261220000000004, 2024-04-20T01:47:44.577 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.577 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:44.577 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.790236+0000", 2024-04-20T01:47:44.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.790236+0000", 2024-04-20T01:47:44.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.790235+0000", 2024-04-20T01:47:44.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:44.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.790520+0000", 2024-04-20T01:47:44.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028522399999999999 2024-04-20T01:47:44.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.790531+0000", 2024-04-20T01:47:44.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0871e-05 2024-04-20T01:47:44.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.790540+0000", 2024-04-20T01:47:44.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1570000000000006e-06 2024-04-20T01:47:44.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.790875+0000", 2024-04-20T01:47:44.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00033489400000000002 2024-04-20T01:47:44.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:44.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.790880+0000", 2024-04-20T01:47:44.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6629999999999998e-06 2024-04-20T01:47:44.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:44.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799335+0000", 2024-04-20T01:47:44.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0084541489999999993 2024-04-20T01:47:44.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799362+0000", 2024-04-20T01:47:44.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7161e-05 2024-04-20T01:47:44.582 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.582 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.582 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.582 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:245 3.es0 3:78f9000f:::smithi16579500-35:head [copy-from ver 10] snapc 5=[2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:47:44.582 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785668+0000", 2024-04-20T01:47:44.582 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.7093556489999999, 2024-04-20T01:47:44.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025155522, 2024-04-20T01:47:44.582 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.582 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:44.582 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:47:44.582 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:47:44.582 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:44.583 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 245 2024-04-20T01:47:44.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.583 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785668+0000", 2024-04-20T01:47:44.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785668+0000", 2024-04-20T01:47:44.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785666+0000", 2024-04-20T01:47:44.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999976 2024-04-20T01:47:44.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785676+0000", 2024-04-20T01:47:44.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0139999999999999e-05 2024-04-20T01:47:44.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785692+0000", 2024-04-20T01:47:44.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5894999999999999e-05 2024-04-20T01:47:44.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785715+0000", 2024-04-20T01:47:44.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2654999999999999e-05 2024-04-20T01:47:44.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785771+0000", 2024-04-20T01:47:44.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6331e-05 2024-04-20T01:47:44.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:47:44.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785779+0000", 2024-04-20T01:47:44.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4789999999999999e-06 2024-04-20T01:47:44.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791776+0000", 2024-04-20T01:47:44.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059978840000000002 2024-04-20T01:47:44.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:44.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791905+0000", 2024-04-20T01:47:44.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000128147 2024-04-20T01:47:44.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:44.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.805603+0000", 2024-04-20T01:47:44.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013698824 2024-04-20T01:47:44.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:44.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.808612+0000", 2024-04-20T01:47:44.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003008541 2024-04-20T01:47:44.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:47:44.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.810754+0000", 2024-04-20T01:47:44.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021421840000000001 2024-04-20T01:47:44.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.810824+0000", 2024-04-20T01:47:44.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9627000000000003e-05 2024-04-20T01:47:44.588 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.588 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.588 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.588 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:246 3.es0 3:78f9000f:::smithi16579500-35:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:47:44.588 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:47:44.588 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.7092954699999998, 2024-04-20T01:47:44.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020980167000000001, 2024-04-20T01:47:44.588 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.588 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:47:44.588 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:47:44.588 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:47:44.588 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:44.588 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 246 2024-04-20T01:47:44.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.589 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:47:44.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:47:44.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:47:44.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:44.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785731+0000", 2024-04-20T01:47:44.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3550000000000001e-06 2024-04-20T01:47:44.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785738+0000", 2024-04-20T01:47:44.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2080000000000003e-06 2024-04-20T01:47:44.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785747+0000", 2024-04-20T01:47:44.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.388e-06 2024-04-20T01:47:44.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785859+0000", 2024-04-20T01:47:44.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112432 2024-04-20T01:47:44.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:47:44.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785866+0000", 2024-04-20T01:47:44.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0890000000000002e-06 2024-04-20T01:47:44.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792104+0000", 2024-04-20T01:47:44.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062376970000000004 2024-04-20T01:47:44.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for blocked object", 2024-04-20T01:47:44.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792157+0000", 2024-04-20T01:47:44.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3269000000000001e-05 2024-04-20T01:47:44.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.806521+0000", 2024-04-20T01:47:44.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014363651999999999 2024-04-20T01:47:44.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:44.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.806639+0000", 2024-04-20T01:47:44.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011755400000000001 2024-04-20T01:47:44.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.806709+0000", 2024-04-20T01:47:44.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0037999999999996e-05 2024-04-20T01:47:44.594 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.594 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.594 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.594 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:247 3.es0 3:79bd6b86:::smithi16579500-12:head [read 0~196608,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:47:44.594 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785760+0000", 2024-04-20T01:47:44.594 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.7092642209999998, 2024-04-20T01:47:44.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011359487999999999, 2024-04-20T01:47:44.594 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.595 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:47:44.595 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:47:44.595 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:47:44.595 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:44.595 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 247 2024-04-20T01:47:44.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.595 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785760+0000", 2024-04-20T01:47:44.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785760+0000", 2024-04-20T01:47:44.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785759+0000", 2024-04-20T01:47:44.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:44.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785764+0000", 2024-04-20T01:47:44.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.194e-06 2024-04-20T01:47:44.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785772+0000", 2024-04-20T01:47:44.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9150000000000002e-06 2024-04-20T01:47:44.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785780+0000", 2024-04-20T01:47:44.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5639999999999999e-06 2024-04-20T01:47:44.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785905+0000", 2024-04-20T01:47:44.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012477000000000001 2024-04-20T01:47:44.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:47:44.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785909+0000", 2024-04-20T01:47:44.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1099999999999996e-06 2024-04-20T01:47:44.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792209+0000", 2024-04-20T01:47:44.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062998009999999998 2024-04-20T01:47:44.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:44.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792316+0000", 2024-04-20T01:47:44.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107161 2024-04-20T01:47:44.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.797119+0000", 2024-04-20T01:47:44.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0048032860000000004 2024-04-20T01:47:44.600 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.600 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.600 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.600 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:248 3.es0 3:79bd6b86:::smithi16579500-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:47:44.600 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:47:44.600 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.7092397100000003, 2024-04-20T01:47:44.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011602988, 2024-04-20T01:47:44.600 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.600 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:47:44.600 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:47:44.600 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:47:44.601 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:44.601 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 248 2024-04-20T01:47:44.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.601 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:47:44.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:47:44.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:47:44.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:44.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785787+0000", 2024-04-20T01:47:44.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4079999999999998e-06 2024-04-20T01:47:44.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785791+0000", 2024-04-20T01:47:44.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9929999999999997e-06 2024-04-20T01:47:44.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785795+0000", 2024-04-20T01:47:44.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1559999999999997e-06 2024-04-20T01:47:44.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785936+0000", 2024-04-20T01:47:44.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014018 2024-04-20T01:47:44.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:47:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785939+0000", 2024-04-20T01:47:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-04-20T01:47:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794183+0000", 2024-04-20T01:47:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082434339999999991 2024-04-20T01:47:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794272+0000", 2024-04-20T01:47:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9246000000000006e-05 2024-04-20T01:47:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.797387+0000", 2024-04-20T01:47:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0031150269999999998 2024-04-20T01:47:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:249 3.es0 3:79bd6b86:::smithi16579500-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:47:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:47:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.7092254130000004, 2024-04-20T01:47:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011999312, 2024-04-20T01:47:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:47:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:47:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:47:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 249 2024-04-20T01:47:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:47:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:47:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:47:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:47:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785800+0000", 2024-04-20T01:47:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.995e-06 2024-04-20T01:47:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785804+0000", 2024-04-20T01:47:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2990000000000001e-06 2024-04-20T01:47:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785807+0000", 2024-04-20T01:47:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.715e-06 2024-04-20T01:47:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785955+0000", 2024-04-20T01:47:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000147923 2024-04-20T01:47:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:47:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785958+0000", 2024-04-20T01:47:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8940000000000002e-06 2024-04-20T01:47:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794609+0000", 2024-04-20T01:47:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086509289999999999 2024-04-20T01:47:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.795125+0000", 2024-04-20T01:47:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00051564999999999998 2024-04-20T01:47:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.797798+0000", 2024-04-20T01:47:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0026731250000000002 2024-04-20T01:47:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.bs4 38/30 ECSubWrite(tid=72, reqid=client.4643.0:250, at_version=38'20, trim_to=0'0, roll_forward_to=37'18))", 2024-04-20T01:47:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.787459+0000", 2024-04-20T01:47:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.707564885, 2024-04-20T01:47:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080595449999999996, 2024-04-20T01:47:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.787459+0000", 2024-04-20T01:47:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:44.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.787459+0000", 2024-04-20T01:47:44.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.787459+0000", 2024-04-20T01:47:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:47:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.787477+0000", 2024-04-20T01:47:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8553999999999999e-05 2024-04-20T01:47:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.787498+0000", 2024-04-20T01:47:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0673999999999999e-05 2024-04-20T01:47:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.787508+0000", 2024-04-20T01:47:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0319999999999999e-05 2024-04-20T01:47:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.787563+0000", 2024-04-20T01:47:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4694999999999997e-05 2024-04-20T01:47:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.787570+0000", 2024-04-20T01:47:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2439999999999996e-06 2024-04-20T01:47:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.795481+0000", 2024-04-20T01:47:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0079110650000000001 2024-04-20T01:47:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.795519+0000", 2024-04-20T01:47:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7141000000000002e-05 2024-04-20T01:47:44.617 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.617 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.617 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.617 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:44.617 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:44.617 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:47:44.619 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-20T01:47:44.619 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:44.633+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:47:44.719 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:47:44.733+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:47:44.789 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:47:44.789 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:47:44.789 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-04-20T01:47:44.789 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:47:44.793 INFO:tasks.rados.rados.0.smithi165.stdout:137: done (1 left) 2024-04-20T01:47:44.793 INFO:tasks.rados.rados.0.smithi165.stdout:138: done (0 left) 2024-04-20T01:47:44.793 INFO:tasks.rados.rados.0.smithi165.stdout:139: rmattr oid 33 current snap is 6 2024-04-20T01:47:44.794 INFO:tasks.rados.rados.0.smithi165.stdout:139: done (0 left) 2024-04-20T01:47:44.794 INFO:tasks.rados.rados.0.smithi165.stdout:140: snap_create 2024-04-20T01:47:44.797 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.10 to 1 (10000) 2024-04-20T01:47:44.798 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is set 2024-04-20T01:47:44.800 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-20T01:47:44.818 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-04-20T01:47:44.820 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:44.835+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:47:44.920 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:44.935+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:44.928 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:47:44.928 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:47:44.928 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:47:44.928 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:47:44.928 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:47:44.938 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-20T01:47:45.023 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:45.037+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:45.070 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:47:45.071 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:47:45.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:47:45.071 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:47:45.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.071 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4227.0:23 1.0 e23/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 23'13, mlcod=23'11)", 2024-04-20T01:47:45.071 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:17.321217+0000", 2024-04-20T01:47:45.071 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.762980228, 2024-04-20T01:47:45.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037759439999999998, 2024-04-20T01:47:45.071 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:45.071 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:45.071 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:45.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:45.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.321217+0000", 2024-04-20T01:47:45.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:45.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.321217+0000", 2024-04-20T01:47:45.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:45.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.321216+0000", 2024-04-20T01:47:45.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:45.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:45.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.321223+0000", 2024-04-20T01:47:45.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7390000000000002e-06 2024-04-20T01:47:45.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:45.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.321233+0000", 2024-04-20T01:47:45.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3319999999999998e-06 2024-04-20T01:47:45.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:45.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.321238+0000", 2024-04-20T01:47:45.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3650000000000003e-06 2024-04-20T01:47:45.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:45.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.321275+0000", 2024-04-20T01:47:45.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7450999999999998e-05 2024-04-20T01:47:45.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:45.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.321292+0000", 2024-04-20T01:47:45.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6373999999999999e-05 2024-04-20T01:47:45.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:47:45.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.324952+0000", 2024-04-20T01:47:45.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0036598759999999998 2024-04-20T01:47:45.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:45.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.324993+0000", 2024-04-20T01:47:45.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1539000000000003e-05 2024-04-20T01:47:45.075 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.075 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.075 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.075 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4227.0:113 1.0 e23/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 23'75, mlcod=23'67)", 2024-04-20T01:47:45.075 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:17.554338+0000", 2024-04-20T01:47:45.075 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.529859007999999, 2024-04-20T01:47:45.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0038295500000000001, 2024-04-20T01:47:45.075 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:45.075 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:45.076 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:45.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:45.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.554338+0000", 2024-04-20T01:47:45.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:45.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.554338+0000", 2024-04-20T01:47:45.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:45.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.554338+0000", 2024-04-20T01:47:45.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:45.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:45.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.554343+0000", 2024-04-20T01:47:45.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3709999999999999e-06 2024-04-20T01:47:45.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:45.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.554351+0000", 2024-04-20T01:47:45.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2330000000000006e-06 2024-04-20T01:47:45.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:45.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.554356+0000", 2024-04-20T01:47:45.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9660000000000004e-06 2024-04-20T01:47:45.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:45.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.554381+0000", 2024-04-20T01:47:45.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4102999999999999e-05 2024-04-20T01:47:45.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:45.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.554394+0000", 2024-04-20T01:47:45.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3005e-05 2024-04-20T01:47:45.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:47:45.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.558145+0000", 2024-04-20T01:47:45.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037511549999999999 2024-04-20T01:47:45.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:45.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.558168+0000", 2024-04-20T01:47:45.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3281999999999998e-05 2024-04-20T01:47:45.079 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.079 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.079 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.079 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4227.0:114 1.0 e23/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 23'76, mlcod=23'69)", 2024-04-20T01:47:45.079 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:17.554929+0000", 2024-04-20T01:47:45.079 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.529267996000002, 2024-04-20T01:47:45.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0032537260000000002, 2024-04-20T01:47:45.079 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:45.080 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:45.080 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:45.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:45.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.554929+0000", 2024-04-20T01:47:45.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:45.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.554929+0000", 2024-04-20T01:47:45.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:45.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.554929+0000", 2024-04-20T01:47:45.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:45.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:45.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.554933+0000", 2024-04-20T01:47:45.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8670000000000004e-06 2024-04-20T01:47:45.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:45.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.554940+0000", 2024-04-20T01:47:45.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6729999999999992e-06 2024-04-20T01:47:45.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:45.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.554945+0000", 2024-04-20T01:47:45.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1969999999999998e-06 2024-04-20T01:47:45.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:45.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.554970+0000", 2024-04-20T01:47:45.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4973e-05 2024-04-20T01:47:45.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:45.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.554981+0000", 2024-04-20T01:47:45.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1459e-05 2024-04-20T01:47:45.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:47:45.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.558171+0000", 2024-04-20T01:47:45.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0031901659999999999 2024-04-20T01:47:45.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:45.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:17.558183+0000", 2024-04-20T01:47:45.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1854e-05 2024-04-20T01:47:45.083 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.083 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.083 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.083 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4227.0:202 1.0 e25/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'148, mlcod=23'145)", 2024-04-20T01:47:45.083 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:19.166551+0000", 2024-04-20T01:47:45.083 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.917646084000001, 2024-04-20T01:47:45.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037956299999999999, 2024-04-20T01:47:45.084 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:45.084 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:45.084 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:45.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:45.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:19.166551+0000", 2024-04-20T01:47:45.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:45.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:19.166551+0000", 2024-04-20T01:47:45.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:45.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:19.166551+0000", 2024-04-20T01:47:45.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:45.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:45.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:19.166557+0000", 2024-04-20T01:47:45.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.925e-06 2024-04-20T01:47:45.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:45.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:19.166562+0000", 2024-04-20T01:47:45.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0699999999999997e-06 2024-04-20T01:47:45.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:45.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:19.166565+0000", 2024-04-20T01:47:45.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.461e-06 2024-04-20T01:47:45.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:45.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:19.166590+0000", 2024-04-20T01:47:45.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4176e-05 2024-04-20T01:47:45.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:45.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:19.166608+0000", 2024-04-20T01:47:45.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8099000000000001e-05 2024-04-20T01:47:45.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:47:45.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:19.170323+0000", 2024-04-20T01:47:45.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003715781 2024-04-20T01:47:45.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:45.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:19.170347+0000", 2024-04-20T01:47:45.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3519e-05 2024-04-20T01:47:45.087 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.087 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.087 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.087 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4227.0:203 1.0 e25/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'149, mlcod=23'145)", 2024-04-20T01:47:45.088 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:19.166930+0000", 2024-04-20T01:47:45.088 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.917267047999999, 2024-04-20T01:47:45.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003429049, 2024-04-20T01:47:45.088 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:45.088 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:45.088 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:45.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:45.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:19.166930+0000", 2024-04-20T01:47:45.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:45.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:19.166930+0000", 2024-04-20T01:47:45.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:45.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:19.166930+0000", 2024-04-20T01:47:45.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:45.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:45.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:19.167064+0000", 2024-04-20T01:47:45.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000134203 2024-04-20T01:47:45.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:45.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:19.167071+0000", 2024-04-20T01:47:45.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.917e-06 2024-04-20T01:47:45.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:45.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:19.167076+0000", 2024-04-20T01:47:45.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3499999999999996e-06 2024-04-20T01:47:45.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:45.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:19.167110+0000", 2024-04-20T01:47:45.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3929e-05 2024-04-20T01:47:45.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:47:45.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:19.167130+0000", 2024-04-20T01:47:45.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9533000000000001e-05 2024-04-20T01:47:45.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:47:45.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:19.170349+0000", 2024-04-20T01:47:45.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0032188690000000001 2024-04-20T01:47:45.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:45.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:19.170359+0000", 2024-04-20T01:47:45.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0608000000000001e-05 2024-04-20T01:47:45.091 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.091 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.091 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.091 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 33/30 ECSubWrite(tid=2, reqid=client.4643.0:28, at_version=33'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:45.092 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.760940+0000", 2024-04-20T01:47:45.092 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.323257638999999, 2024-04-20T01:47:45.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0032349050000000002, 2024-04-20T01:47:45.092 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:45.092 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:45.092 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:45.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:45.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.760940+0000", 2024-04-20T01:47:45.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:45.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.760940+0000", 2024-04-20T01:47:45.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:45.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.760939+0000", 2024-04-20T01:47:45.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:45.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:45.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.761014+0000", 2024-04-20T01:47:45.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5519000000000007e-05 2024-04-20T01:47:45.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:45.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.761030+0000", 2024-04-20T01:47:45.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5784999999999999e-05 2024-04-20T01:47:45.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:45.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.761041+0000", 2024-04-20T01:47:45.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0899e-05 2024-04-20T01:47:45.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:45.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.761169+0000", 2024-04-20T01:47:45.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012798900000000001 2024-04-20T01:47:45.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:45.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.761196+0000", 2024-04-20T01:47:45.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6974e-05 2024-04-20T01:47:45.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:45.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.764146+0000", 2024-04-20T01:47:45.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002949545 2024-04-20T01:47:45.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:45.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.764175+0000", 2024-04-20T01:47:45.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9128999999999998e-05 2024-04-20T01:47:45.095 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.095 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.095 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.095 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs2 33/30 ECSubWrite(tid=4, reqid=client.4643.0:94, at_version=33'3, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:47:45.096 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.810869+0000", 2024-04-20T01:47:45.096 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.273327957999999, 2024-04-20T01:47:45.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040776120000000004, 2024-04-20T01:47:45.096 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:45.096 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:45.096 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:45.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:45.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.810869+0000", 2024-04-20T01:47:45.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:45.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.810869+0000", 2024-04-20T01:47:45.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:45.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.810869+0000", 2024-04-20T01:47:45.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:45.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:45.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.811245+0000", 2024-04-20T01:47:45.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00037595499999999997 2024-04-20T01:47:45.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:45.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.811257+0000", 2024-04-20T01:47:45.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2099000000000001e-05 2024-04-20T01:47:45.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:45.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.811266+0000", 2024-04-20T01:47:45.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8629999999999997e-06 2024-04-20T01:47:45.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:45.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.811319+0000", 2024-04-20T01:47:45.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3338000000000003e-05 2024-04-20T01:47:45.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:45.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.811331+0000", 2024-04-20T01:47:45.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1457999999999999e-05 2024-04-20T01:47:45.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:45.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.814920+0000", 2024-04-20T01:47:45.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035894189999999999 2024-04-20T01:47:45.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:45.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.814947+0000", 2024-04-20T01:47:45.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7129999999999999e-05 2024-04-20T01:47:45.099 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.100 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.100 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.100 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs2 33/30 ECSubWrite(tid=5, reqid=client.4643.0:95, at_version=33'4, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:47:45.100 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.811795+0000", 2024-04-20T01:47:45.100 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.272401926000001, 2024-04-20T01:47:45.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00384112, 2024-04-20T01:47:45.100 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:45.100 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:45.100 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:45.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:45.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.811795+0000", 2024-04-20T01:47:45.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:45.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.811795+0000", 2024-04-20T01:47:45.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:45.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.811795+0000", 2024-04-20T01:47:45.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:45.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:45.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.811813+0000", 2024-04-20T01:47:45.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8471e-05 2024-04-20T01:47:45.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:45.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.811829+0000", 2024-04-20T01:47:45.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5772000000000001e-05 2024-04-20T01:47:45.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:45.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.811849+0000", 2024-04-20T01:47:45.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0006000000000001e-05 2024-04-20T01:47:45.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:45.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.811899+0000", 2024-04-20T01:47:45.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9662e-05 2024-04-20T01:47:45.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:45.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.811908+0000", 2024-04-20T01:47:45.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7280000000000007e-06 2024-04-20T01:47:45.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:45.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.815617+0000", 2024-04-20T01:47:45.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037081599999999998 2024-04-20T01:47:45.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:45.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.815637+0000", 2024-04-20T01:47:45.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0044e-05 2024-04-20T01:47:45.103 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.104 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.104 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.104 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs2 33/30 ECSubWrite(tid=10, reqid=client.4643.0:125, at_version=0'0, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:45.104 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.854400+0000", 2024-04-20T01:47:45.104 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.229797175, 2024-04-20T01:47:45.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046267000000000001, 2024-04-20T01:47:45.104 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:45.104 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:45.104 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:45.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:45.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.854400+0000", 2024-04-20T01:47:45.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:45.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.854400+0000", 2024-04-20T01:47:45.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:45.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.854399+0000", 2024-04-20T01:47:45.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:45.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:45.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.854410+0000", 2024-04-20T01:47:45.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0743e-05 2024-04-20T01:47:45.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:45.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.854424+0000", 2024-04-20T01:47:45.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.399e-05 2024-04-20T01:47:45.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:45.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.854430+0000", 2024-04-20T01:47:45.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2689999999999996e-06 2024-04-20T01:47:45.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:45.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.854469+0000", 2024-04-20T01:47:45.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8732000000000001e-05 2024-04-20T01:47:45.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:45.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.854478+0000", 2024-04-20T01:47:45.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5159999999999994e-06 2024-04-20T01:47:45.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:45.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.859010+0000", 2024-04-20T01:47:45.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0045325740000000002 2024-04-20T01:47:45.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:45.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.859027+0000", 2024-04-20T01:47:45.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6762999999999999e-05 2024-04-20T01:47:45.108 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.108 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.108 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.108 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 37/30 ECSubWrite(tid=18, reqid=client.4643.0:234, at_version=37'8, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:45.108 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.792628+0000", 2024-04-20T01:47:45.108 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.2915697919999998, 2024-04-20T01:47:45.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040034060000000002, 2024-04-20T01:47:45.108 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:45.108 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:45.108 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:45.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:45.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.792628+0000", 2024-04-20T01:47:45.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:45.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.792628+0000", 2024-04-20T01:47:45.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:45.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.792627+0000", 2024-04-20T01:47:45.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:47:45.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:45.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.792649+0000", 2024-04-20T01:47:45.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1478e-05 2024-04-20T01:47:45.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:45.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.792660+0000", 2024-04-20T01:47:45.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0764999999999999e-05 2024-04-20T01:47:45.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:45.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.792669+0000", 2024-04-20T01:47:45.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3239999999999992e-06 2024-04-20T01:47:45.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:45.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793117+0000", 2024-04-20T01:47:45.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044802999999999998 2024-04-20T01:47:45.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:45.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793127+0000", 2024-04-20T01:47:45.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7710000000000001e-06 2024-04-20T01:47:45.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:45.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796606+0000", 2024-04-20T01:47:45.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003478835 2024-04-20T01:47:45.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:45.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.796631+0000", 2024-04-20T01:47:45.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5439000000000002e-05 2024-04-20T01:47:45.112 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.112 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.112 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.112 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 37/30 ECSubWrite(tid=43, reqid=client.4643.0:241, at_version=37'31, trim_to=0'0, roll_forward_to=36'28))", 2024-04-20T01:47:45.112 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.793501+0000", 2024-04-20T01:47:45.112 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.290696412, 2024-04-20T01:47:45.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005847517, 2024-04-20T01:47:45.112 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:45.112 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:45.112 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:45.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:45.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793501+0000", 2024-04-20T01:47:45.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:45.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793501+0000", 2024-04-20T01:47:45.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:45.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793500+0000", 2024-04-20T01:47:45.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:45.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:45.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793525+0000", 2024-04-20T01:47:45.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4641999999999999e-05 2024-04-20T01:47:45.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:45.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793537+0000", 2024-04-20T01:47:45.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2626000000000001e-05 2024-04-20T01:47:45.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:45.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793547+0000", 2024-04-20T01:47:45.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.842e-06 2024-04-20T01:47:45.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:45.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793599+0000", 2024-04-20T01:47:45.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2301e-05 2024-04-20T01:47:45.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:45.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793607+0000", 2024-04-20T01:47:45.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9070000000000007e-06 2024-04-20T01:47:45.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:45.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799321+0000", 2024-04-20T01:47:45.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005713358 2024-04-20T01:47:45.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:45.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799349+0000", 2024-04-20T01:47:45.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7832999999999999e-05 2024-04-20T01:47:45.116 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.116 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.116 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.116 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 37/30 ECSubWrite(tid=19, reqid=client.4643.0:235, at_version=37'9, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:45.116 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.793816+0000", 2024-04-20T01:47:45.116 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.2903809370000001, 2024-04-20T01:47:45.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0055550790000000001, 2024-04-20T01:47:45.116 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:45.116 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:45.116 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:45.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:45.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793816+0000", 2024-04-20T01:47:45.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:45.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793816+0000", 2024-04-20T01:47:45.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:45.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793816+0000", 2024-04-20T01:47:45.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:45.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:45.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793838+0000", 2024-04-20T01:47:45.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2075000000000001e-05 2024-04-20T01:47:45.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:45.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793849+0000", 2024-04-20T01:47:45.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1433999999999999e-05 2024-04-20T01:47:45.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:45.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793859+0000", 2024-04-20T01:47:45.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4849999999999998e-06 2024-04-20T01:47:45.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:45.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793956+0000", 2024-04-20T01:47:45.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7114000000000006e-05 2024-04-20T01:47:45.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:45.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793965+0000", 2024-04-20T01:47:45.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0820000000000005e-06 2024-04-20T01:47:45.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:45.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799344+0000", 2024-04-20T01:47:45.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053794400000000001 2024-04-20T01:47:45.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:45.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799372+0000", 2024-04-20T01:47:45.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7231000000000001e-05 2024-04-20T01:47:45.120 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.120 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.120 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.120 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 37/30 ECSubWrite(tid=44, reqid=client.4643.0:242, at_version=37'32, trim_to=0'0, roll_forward_to=36'28))", 2024-04-20T01:47:45.120 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.794496+0000", 2024-04-20T01:47:45.120 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.2897010450000002, 2024-04-20T01:47:45.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0049252380000000002, 2024-04-20T01:47:45.120 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:45.120 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:45.120 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:45.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:45.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794496+0000", 2024-04-20T01:47:45.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:45.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794496+0000", 2024-04-20T01:47:45.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:45.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794496+0000", 2024-04-20T01:47:45.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:45.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:45.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794516+0000", 2024-04-20T01:47:45.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0475999999999998e-05 2024-04-20T01:47:45.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:45.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794530+0000", 2024-04-20T01:47:45.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3565e-05 2024-04-20T01:47:45.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:45.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794539+0000", 2024-04-20T01:47:45.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9600000000000006e-06 2024-04-20T01:47:45.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:45.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794591+0000", 2024-04-20T01:47:45.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2574999999999997e-05 2024-04-20T01:47:45.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:45.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794602+0000", 2024-04-20T01:47:45.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1338999999999999e-05 2024-04-20T01:47:45.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:45.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799353+0000", 2024-04-20T01:47:45.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047508999999999997 2024-04-20T01:47:45.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:45.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799422+0000", 2024-04-20T01:47:45.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8281999999999994e-05 2024-04-20T01:47:45.124 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.124 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.124 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.124 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 37/30 ECSubWrite(tid=20, reqid=client.4643.0:236, at_version=37'10, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:45.124 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.795402+0000", 2024-04-20T01:47:45.124 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.2887954349999999, 2024-04-20T01:47:45.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039852380000000003, 2024-04-20T01:47:45.125 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:45.125 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:45.125 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:45.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:45.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795402+0000", 2024-04-20T01:47:45.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:45.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795402+0000", 2024-04-20T01:47:45.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:45.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795401+0000", 2024-04-20T01:47:45.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:45.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:45.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795420+0000", 2024-04-20T01:47:45.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9117e-05 2024-04-20T01:47:45.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:45.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795435+0000", 2024-04-20T01:47:45.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4659e-05 2024-04-20T01:47:45.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:45.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795444+0000", 2024-04-20T01:47:45.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4520000000000003e-06 2024-04-20T01:47:45.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:45.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795529+0000", 2024-04-20T01:47:45.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4759000000000006e-05 2024-04-20T01:47:45.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:45.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795538+0000", 2024-04-20T01:47:45.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0969999999999995e-06 2024-04-20T01:47:45.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:45.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799376+0000", 2024-04-20T01:47:45.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0038379030000000001 2024-04-20T01:47:45.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:45.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799387+0000", 2024-04-20T01:47:45.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1039999999999999e-05 2024-04-20T01:47:45.128 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.128 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.128 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.128 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 38/30 ECSubWrite(tid=22, reqid=osd.3.0:22, at_version=38'12, trim_to=0'0, roll_forward_to=37'10))", 2024-04-20T01:47:45.128 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.784004+0000", 2024-04-20T01:47:45.128 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.3001930289999999, 2024-04-20T01:47:45.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003379637, 2024-04-20T01:47:45.129 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:45.129 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:45.129 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:45.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:45.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.784004+0000", 2024-04-20T01:47:45.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:45.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.784004+0000", 2024-04-20T01:47:45.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:45.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.784004+0000", 2024-04-20T01:47:45.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:45.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:45.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.784029+0000", 2024-04-20T01:47:45.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5233e-05 2024-04-20T01:47:45.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:45.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.784042+0000", 2024-04-20T01:47:45.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2425e-05 2024-04-20T01:47:45.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:45.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.784054+0000", 2024-04-20T01:47:45.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.293e-05 2024-04-20T01:47:45.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:45.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.784115+0000", 2024-04-20T01:47:45.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0504999999999998e-05 2024-04-20T01:47:45.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:45.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.784126+0000", 2024-04-20T01:47:45.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0784999999999999e-05 2024-04-20T01:47:45.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:45.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.787343+0000", 2024-04-20T01:47:45.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0032176240000000001 2024-04-20T01:47:45.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:45.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.787384+0000", 2024-04-20T01:47:45.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0638e-05 2024-04-20T01:47:45.132 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.132 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.132 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.132 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.ds3 38/30 ECSubWrite(tid=18, reqid=osd.6.0:18, at_version=38'10, trim_to=0'0, roll_forward_to=37'8))", 2024-04-20T01:47:45.132 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.786868+0000", 2024-04-20T01:47:45.132 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.2973292509999999, 2024-04-20T01:47:45.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039226649999999997, 2024-04-20T01:47:45.133 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:45.133 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:45.133 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:45.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:45.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.786868+0000", 2024-04-20T01:47:45.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:45.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.786868+0000", 2024-04-20T01:47:45.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:45.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.786867+0000", 2024-04-20T01:47:45.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:45.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:45.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.786894+0000", 2024-04-20T01:47:45.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6239000000000001e-05 2024-04-20T01:47:45.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:45.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.786904+0000", 2024-04-20T01:47:45.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0001e-05 2024-04-20T01:47:45.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:45.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.786912+0000", 2024-04-20T01:47:45.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4489999999999999e-06 2024-04-20T01:47:45.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:45.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.786972+0000", 2024-04-20T01:47:45.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0336e-05 2024-04-20T01:47:45.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:45.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.786984+0000", 2024-04-20T01:47:45.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2121e-05 2024-04-20T01:47:45.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:45.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.790759+0000", 2024-04-20T01:47:45.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037747599999999998 2024-04-20T01:47:45.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:45.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.790791+0000", 2024-04-20T01:47:45.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1652999999999997e-05 2024-04-20T01:47:45.136 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.136 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.136 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.136 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 38/30 ECSubWrite(tid=23, reqid=osd.3.0:22, at_version=0'0, trim_to=0'0, roll_forward_to=38'12))", 2024-04-20T01:47:45.136 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.788077+0000", 2024-04-20T01:47:45.136 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.2961208830000004, 2024-04-20T01:47:45.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057844480000000002, 2024-04-20T01:47:45.137 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:45.137 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:45.137 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:45.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:45.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788077+0000", 2024-04-20T01:47:45.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:45.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788077+0000", 2024-04-20T01:47:45.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:45.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788076+0000", 2024-04-20T01:47:45.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:47:45.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:45.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788088+0000", 2024-04-20T01:47:45.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2672e-05 2024-04-20T01:47:45.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:45.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788101+0000", 2024-04-20T01:47:45.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2468e-05 2024-04-20T01:47:45.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:45.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788121+0000", 2024-04-20T01:47:45.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0315000000000001e-05 2024-04-20T01:47:45.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:45.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788181+0000", 2024-04-20T01:47:45.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0257999999999998e-05 2024-04-20T01:47:45.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:45.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788194+0000", 2024-04-20T01:47:45.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2714000000000001e-05 2024-04-20T01:47:45.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:45.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.793811+0000", 2024-04-20T01:47:45.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056169649999999998 2024-04-20T01:47:45.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:45.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.793861+0000", 2024-04-20T01:47:45.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9978000000000002e-05 2024-04-20T01:47:45.140 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.140 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.140 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.140 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 38/30 ECSubWrite(tid=46, reqid=osd.8.0:46, at_version=38'34, trim_to=0'0, roll_forward_to=37'32))", 2024-04-20T01:47:45.140 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.791125+0000", 2024-04-20T01:47:45.140 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.2930721720000005, 2024-04-20T01:47:45.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046488060000000001, 2024-04-20T01:47:45.140 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:45.141 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:45.141 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:45.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:45.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791125+0000", 2024-04-20T01:47:45.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:45.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791125+0000", 2024-04-20T01:47:45.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:45.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791125+0000", 2024-04-20T01:47:45.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:45.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:45.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791147+0000", 2024-04-20T01:47:45.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2269999999999999e-05 2024-04-20T01:47:45.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:45.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791160+0000", 2024-04-20T01:47:45.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2492e-05 2024-04-20T01:47:45.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:45.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791172+0000", 2024-04-20T01:47:45.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2079000000000001e-05 2024-04-20T01:47:45.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:45.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791235+0000", 2024-04-20T01:47:45.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2957e-05 2024-04-20T01:47:45.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:45.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791246+0000", 2024-04-20T01:47:45.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1236999999999999e-05 2024-04-20T01:47:45.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:45.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.795741+0000", 2024-04-20T01:47:45.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.004495265 2024-04-20T01:47:45.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:45.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.795774+0000", 2024-04-20T01:47:45.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2775999999999999e-05 2024-04-20T01:47:45.144 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.144 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.144 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.144 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 42/30 ECSubWrite(tid=64, reqid=client.4643.0:318, at_version=42'51, trim_to=0'0, roll_forward_to=41'48))", 2024-04-20T01:47:45.144 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:43.938535+0000", 2024-04-20T01:47:45.144 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.1456620630000001, 2024-04-20T01:47:45.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043950170000000002, 2024-04-20T01:47:45.144 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:45.144 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:45.145 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:45.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:45.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.938535+0000", 2024-04-20T01:47:45.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:45.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.938535+0000", 2024-04-20T01:47:45.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:45.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.938535+0000", 2024-04-20T01:47:45.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:45.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:45.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.938673+0000", 2024-04-20T01:47:45.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013837800000000001 2024-04-20T01:47:45.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:45.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.938684+0000", 2024-04-20T01:47:45.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1233000000000001e-05 2024-04-20T01:47:45.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:45.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.938693+0000", 2024-04-20T01:47:45.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4470000000000006e-06 2024-04-20T01:47:45.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:45.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.939397+0000", 2024-04-20T01:47:45.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00070402599999999996 2024-04-20T01:47:45.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:45.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.939404+0000", 2024-04-20T01:47:45.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2309999999999999e-06 2024-04-20T01:47:45.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:45.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.942907+0000", 2024-04-20T01:47:45.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003502654 2024-04-20T01:47:45.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:45.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.942930+0000", 2024-04-20T01:47:45.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3728000000000001e-05 2024-04-20T01:47:45.148 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.148 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.148 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.148 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 42/30 ECSubWrite(tid=65, reqid=client.4643.0:319, at_version=42'52, trim_to=0'0, roll_forward_to=41'48))", 2024-04-20T01:47:45.148 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:43.939561+0000", 2024-04-20T01:47:45.148 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.1446364870000001, 2024-04-20T01:47:45.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0036640940000000001, 2024-04-20T01:47:45.148 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:47:45.148 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:47:45.149 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:47:45.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:47:45.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.939561+0000", 2024-04-20T01:47:45.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:47:45.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.939561+0000", 2024-04-20T01:47:45.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:47:45.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:47:45.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.939560+0000", 2024-04-20T01:47:45.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:45.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:47:45.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.939580+0000", 2024-04-20T01:47:45.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0100999999999999e-05 2024-04-20T01:47:45.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:47:45.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.939592+0000", 2024-04-20T01:47:45.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1409999999999999e-05 2024-04-20T01:47:45.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:47:45.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.939600+0000", 2024-04-20T01:47:45.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4780000000000008e-06 2024-04-20T01:47:45.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:47:45.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.939876+0000", 2024-04-20T01:47:45.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00027578100000000001 2024-04-20T01:47:45.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:47:45.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.939885+0000", 2024-04-20T01:47:45.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0569999999999999e-06 2024-04-20T01:47:45.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:47:45.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.943206+0000", 2024-04-20T01:47:45.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003321329 2024-04-20T01:47:45.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:47:45.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:47:45.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:47:45.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.943225+0000", 2024-04-20T01:47:45.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8644999999999999e-05 2024-04-20T01:47:45.152 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.152 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.152 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.152 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:47:45.152 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:47:45.152 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:47:45.154 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-20T01:47:45.154 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:47:45.168+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:47:45.255 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:45.269+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:47:45.300 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:45.300 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:47:45.300 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-04-20T01:47:45.300 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:45.311 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-20T01:47:45.356 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:45.371+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:45.367 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-20T01:47:45.457 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:47:45.472+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:47:45.460 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:45.460 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:47:45.460 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-04-20T01:47:45.460 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-04-20T01:47:45.460 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:45.472 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-20T01:47:45.533 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:47:45.554 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:47:45.558 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:47:45.572+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:47:45.575 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:47:45.597 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:47:45.617 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:47:45.619 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:45.619 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:45.633 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-20T01:47:45.638 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:47:45.659 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:45.674+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:47:45.659 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:47:45.680 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:47:45.701 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:47:45.722 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:47:45.743 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:47:45.760 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:45.775+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:47:45.764 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:47:45.782 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:45.782 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:45.785 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:47:45.795 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-20T01:47:45.802 INFO:tasks.rados.rados.0.smithi165.stdout:140: done (0 left) 2024-04-20T01:47:45.802 INFO:tasks.rados.rados.0.smithi165.stdout:141: read oid 12 snap 5 2024-04-20T01:47:45.802 INFO:tasks.rados.rados.0.smithi165.stdout:141: expect (ObjNum 11 snap 0 seq_num 11) 2024-04-20T01:47:45.802 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 41 current snap is 7 2024-04-20T01:47:45.802 INFO:tasks.rados.rados.0.smithi165.stdout:142: seq_num 75 ranges {688128=491520,1179648=770048} 2024-04-20T01:47:45.805 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.11 to 1 (10000) 2024-04-20T01:47:45.808 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:47:45.808 INFO:tasks.rados.rados.0.smithi165.stdout:142: writing smithi16579500-41 from 688128 to 1179648 tid 1 2024-04-20T01:47:45.813 INFO:tasks.rados.rados.0.smithi165.stdout:142: writing smithi16579500-41 from 1179648 to 1949696 tid 2 2024-04-20T01:47:45.813 INFO:tasks.rados.rados.0.smithi165.stdout:143: setattr oid 30 current snap is 7 2024-04-20T01:47:45.813 INFO:tasks.rados.rados.0.smithi165.stdout:141: expect (ObjNum 11 snap 0 seq_num 11) 2024-04-20T01:47:45.819 INFO:tasks.rados.rados.0.smithi165.stdout:141: done (2 left) 2024-04-20T01:47:45.819 INFO:tasks.rados.rados.0.smithi165.stdout:144: read oid 25 snap -1 2024-04-20T01:47:45.819 INFO:tasks.rados.rados.0.smithi165.stdout:144: expect (ObjNum 24 snap 0 seq_num 24) 2024-04-20T01:47:45.819 INFO:tasks.rados.rados.0.smithi165.stdout:145: rollback oid 28 current snap is 7 2024-04-20T01:47:45.819 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 28 to 5 2024-04-20T01:47:45.819 INFO:tasks.rados.rados.0.smithi165.stdout:146: delete oid 33 current snap is 7 2024-04-20T01:47:45.820 INFO:tasks.rados.rados.0.smithi165.stdout:142: finishing write tid 1 to smithi16579500-41 2024-04-20T01:47:45.823 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 12 1 2024-04-20T01:47:45.825 INFO:tasks.rados.rados.0.smithi165.stdout:144: expect (ObjNum 24 snap 0 seq_num 24) 2024-04-20T01:47:45.838 INFO:tasks.rados.rados.0.smithi165.stdout:142: finishing write tid 2 to smithi16579500-41 2024-04-20T01:47:45.838 INFO:tasks.rados.rados.0.smithi165.stdout:142: finishing write tid 3 to smithi16579500-41 2024-04-20T01:47:45.838 INFO:tasks.rados.rados.0.smithi165.stdout:142: finishing write tid 4 to smithi16579500-41 2024-04-20T01:47:45.838 INFO:tasks.rados.rados.0.smithi165.stdout:142: oid 41 updating version 0 to 54 2024-04-20T01:47:45.839 INFO:tasks.rados.rados.0.smithi165.stdout:142: oid 41 updating version 54 to 55 2024-04-20T01:47:45.839 INFO:tasks.rados.rados.0.smithi165.stdout:142: oid 41 updating version 55 to 56 2024-04-20T01:47:45.839 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 41 v 56 (ObjNum 75 snap 7 seq_num 75) dirty exists 2024-04-20T01:47:45.839 INFO:tasks.rados.rados.0.smithi165.stdout:142: left oid 41 (ObjNum 75 snap 7 seq_num 75) 2024-04-20T01:47:45.839 INFO:tasks.rados.rados.0.smithi165.stdout:145: finishing rollback tid 0 to smithi16579500-28 2024-04-20T01:47:45.839 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 30 v 17 (ObjNum 289 snap 0 seq_num 2584501104) dirty exists 2024-04-20T01:47:45.839 INFO:tasks.rados.rados.0.smithi165.stdout:145: finishing rollback tid 1 to smithi16579500-28 2024-04-20T01:47:45.839 INFO:tasks.rados.rados.0.smithi165.stdout:142: done (4 left) 2024-04-20T01:47:45.839 INFO:tasks.rados.rados.0.smithi165.stdout:143: done (3 left) 2024-04-20T01:47:45.839 INFO:tasks.rados.rados.0.smithi165.stdout:144: done (2 left) 2024-04-20T01:47:45.839 INFO:tasks.rados.rados.0.smithi165.stdout:146: done (1 left) 2024-04-20T01:47:45.839 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 26 current snap is 7 2024-04-20T01:47:45.839 INFO:tasks.rados.rados.0.smithi165.stdout:145: finishing rollback tid 2 to smithi16579500-28 2024-04-20T01:47:45.839 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 28 v 18 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-04-20T01:47:45.839 INFO:tasks.rados.rados.0.smithi165.stdout:147: seq_num 77 ranges {425984=524288,950272=524288,1474560=245760} 2024-04-20T01:47:45.840 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:47:45.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:47:45.852+0000 7f1a47fff700 0 -- 172.21.15.44:0/3488520891 >> [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] conn(0x7f1a48221400 msgr2=0x7f1a482237f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-20T01:47:45.842 INFO:tasks.rados.rados.0.smithi165.stdout:147: writing smithi16579500-26 from 425984 to 950272 tid 1 2024-04-20T01:47:45.845 INFO:tasks.rados.rados.0.smithi165.stdout:147: writing smithi16579500-26 from 950272 to 1474560 tid 2 2024-04-20T01:47:45.846 INFO:tasks.rados.rados.0.smithi165.stdout:147: writing smithi16579500-26 from 1474560 to 1720320 tid 3 2024-04-20T01:47:45.846 INFO:tasks.rados.rados.0.smithi165.stdout:145: done (1 left) 2024-04-20T01:47:45.847 INFO:tasks.rados.rados.0.smithi165.stdout:148: rollback oid 43 current snap is 7 2024-04-20T01:47:45.847 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 43 to 5 2024-04-20T01:47:45.847 INFO:tasks.rados.rados.0.smithi165.stdout:149: delete oid 40 current snap is 7 2024-04-20T01:47:45.847 INFO:tasks.rados.rados.0.smithi165.stdout:148: finishing rollback tid 1 to smithi16579500-43 2024-04-20T01:47:45.847 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 43 v 0 (ObjNum 145 snap 0 seq_num 2584251104) dirty dne 2024-04-20T01:47:45.849 INFO:tasks.rados.rados.0.smithi165.stdout:148: done (2 left) 2024-04-20T01:47:45.849 INFO:tasks.rados.rados.0.smithi165.stdout:149: done (1 left) 2024-04-20T01:47:45.850 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 19 current snap is 7 2024-04-20T01:47:45.850 INFO:tasks.rados.rados.0.smithi165.stdout:150: seq_num 79 ranges {2424832=573440,2998272=425984,3424256=344064} 2024-04-20T01:47:45.861 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:45.875+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:47:45.866 INFO:tasks.rados.rados.0.smithi165.stdout:150: writing smithi16579500-19 from 2424832 to 2998272 tid 1 2024-04-20T01:47:45.868 INFO:tasks.rados.rados.0.smithi165.stdout:150: writing smithi16579500-19 from 2998272 to 3424256 tid 2 2024-04-20T01:47:45.871 INFO:tasks.rados.rados.0.smithi165.stdout:150: writing smithi16579500-19 from 3424256 to 3768320 tid 3 2024-04-20T01:47:45.871 INFO:tasks.rados.rados.0.smithi165.stdout:151: snap_create 2024-04-20T01:47:45.871 INFO:tasks.rados.rados.0.smithi165.stdout:147: finishing write tid 1 to smithi16579500-26 2024-04-20T01:47:45.871 INFO:tasks.rados.rados.0.smithi165.stdout:147: finishing write tid 2 to smithi16579500-26 2024-04-20T01:47:45.871 INFO:tasks.rados.rados.0.smithi165.stdout:147: finishing write tid 3 to smithi16579500-26 2024-04-20T01:47:45.871 INFO:tasks.rados.rados.0.smithi165.stdout:147: finishing write tid 4 to smithi16579500-26 2024-04-20T01:47:45.871 INFO:tasks.rados.rados.0.smithi165.stdout:147: finishing write tid 5 to smithi16579500-26 2024-04-20T01:47:45.871 INFO:tasks.rados.rados.0.smithi165.stdout:147: oid 26 updating version 0 to 43 2024-04-20T01:47:45.871 INFO:tasks.rados.rados.0.smithi165.stdout:147: oid 26 version 43 is already newer than 40 2024-04-20T01:47:45.871 INFO:tasks.rados.rados.0.smithi165.stdout:147: oid 26 version 43 is already newer than 41 2024-04-20T01:47:45.871 INFO:tasks.rados.rados.0.smithi165.stdout:147: oid 26 version 43 is already newer than 42 2024-04-20T01:47:45.871 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 26 v 43 (ObjNum 77 snap 7 seq_num 77) dirty exists 2024-04-20T01:47:45.872 INFO:tasks.rados.rados.0.smithi165.stdout:147: left oid 26 (ObjNum 77 snap 7 seq_num 77) 2024-04-20T01:47:45.872 INFO:tasks.rados.rados.0.smithi165.stdout:150: finishing write tid 1 to smithi16579500-19 2024-04-20T01:47:45.874 INFO:tasks.rados.rados.0.smithi165.stdout:150: finishing write tid 2 to smithi16579500-19 2024-04-20T01:47:45.876 INFO:tasks.rados.rados.0.smithi165.stdout:150: finishing write tid 3 to smithi16579500-19 2024-04-20T01:47:45.876 INFO:tasks.rados.rados.0.smithi165.stdout:150: finishing write tid 4 to smithi16579500-19 2024-04-20T01:47:45.878 INFO:tasks.rados.rados.0.smithi165.stdout:150: finishing write tid 5 to smithi16579500-19 2024-04-20T01:47:45.878 INFO:tasks.rados.rados.0.smithi165.stdout:150: oid 19 updating version 0 to 59 2024-04-20T01:47:45.878 INFO:tasks.rados.rados.0.smithi165.stdout:150: oid 19 version 59 is already newer than 58 2024-04-20T01:47:45.878 INFO:tasks.rados.rados.0.smithi165.stdout:150: oid 19 updating version 59 to 61 2024-04-20T01:47:45.878 INFO:tasks.rados.rados.0.smithi165.stdout:150: oid 19 version 61 is already newer than 60 2024-04-20T01:47:45.878 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 19 v 61 (ObjNum 79 snap 7 seq_num 79) dirty exists 2024-04-20T01:47:45.878 INFO:tasks.rados.rados.0.smithi165.stdout:150: left oid 19 (ObjNum 79 snap 7 seq_num 79) 2024-04-20T01:47:45.962 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:45.976+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:45.967 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:45.967 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:45.980 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-20T01:47:46.060 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:47:46.062 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:47:46.062 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:47:46.062 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:47:46.062 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:47:46.062 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:47:46.062 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:47:46.062 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:47:46.062 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:47:46.062 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:47:46.063 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:47:46.063 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:47:46.063 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:47:46.063 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:47:46.063 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:47:46.063 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:47:46.063 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:47:46.064 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:46.077+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:47:46.130 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:46.130 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:46.142 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-20T01:47:46.187 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:47:46.201+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:47:46.264 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:46.279+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:47:46.285 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:46.285 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:46.297 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-20T01:47:46.365 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:46.379+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:47:46.465 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:47:46.480+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:47:46.484 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:46.484 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:46.496 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-20T01:47:46.567 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:46.582+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:46.668 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:46.683+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:47:46.683 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:46.683 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:46.695 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-20T01:47:46.769 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:46.783+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:46.804 INFO:tasks.rados.rados.0.smithi165.stdout:147: done (2 left) 2024-04-20T01:47:46.804 INFO:tasks.rados.rados.0.smithi165.stdout:150: done (1 left) 2024-04-20T01:47:46.804 INFO:tasks.rados.rados.0.smithi165.stdout:151: done (0 left) 2024-04-20T01:47:46.804 INFO:tasks.rados.rados.0.smithi165.stdout:152: snap_remove snap 7 2024-04-20T01:47:46.806 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.12 to 1 (10000) 2024-04-20T01:47:46.820 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 13 1 2024-04-20T01:47:46.820 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-20T01:47:46.846 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:46.847 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:46.861 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-20T01:47:46.869 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:46.884+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:46.970 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:46.985+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:47.064 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:47.064 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:47.071 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:47:47.085+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:47:47.076 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-20T01:47:47.172 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:47.187+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:47.261 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:47.261 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:47.273 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-20T01:47:47.274 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:47.288+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:47.375 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:47.390+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:47:47.413 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:47.413 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:47.424 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-20T01:47:47.476 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:47.490+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:47:47.551 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:47.551 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:47.562 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-20T01:47:47.576 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:47.591+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:47:47.677 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:47.691+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:47.690 INFO:tasks.rados.rados.0.smithi165.stdout:152: done (0 left) 2024-04-20T01:47:47.690 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 47 current snap is 8 2024-04-20T01:47:47.690 INFO:tasks.rados.rados.0.smithi165.stdout:153: seq_num 80 ranges {1671168=589824,2260992=573440,2834432=196608} 2024-04-20T01:47:47.690 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:47.691 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:47.694 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is unset 2024-04-20T01:47:47.694 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.13 to 1 (10000) 2024-04-20T01:47:47.701 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-20T01:47:47.702 INFO:tasks.rados.rados.0.smithi165.stdout:153: writing smithi16579500-47 from 1671168 to 2260992 tid 1 2024-04-20T01:47:47.705 INFO:tasks.rados.rados.0.smithi165.stdout:153: writing smithi16579500-47 from 2260992 to 2834432 tid 2 2024-04-20T01:47:47.706 INFO:tasks.rados.rados.0.smithi165.stdout:153: writing smithi16579500-47 from 2834432 to 3031040 tid 3 2024-04-20T01:47:47.706 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 24 current snap is 8 2024-04-20T01:47:47.706 INFO:tasks.rados.rados.0.smithi165.stdout:154: seq_num 81 ranges {1769472=606208,2375680=720896,3096576=49152} 2024-04-20T01:47:47.711 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 14 1 2024-04-20T01:47:47.720 INFO:tasks.rados.rados.0.smithi165.stdout:154: writing smithi16579500-24 from 1769472 to 2375680 tid 1 2024-04-20T01:47:47.724 INFO:tasks.rados.rados.0.smithi165.stdout:154: writing smithi16579500-24 from 2375680 to 3096576 tid 2 2024-04-20T01:47:47.725 INFO:tasks.rados.rados.0.smithi165.stdout:154: writing smithi16579500-24 from 3096576 to 3145728 tid 3 2024-04-20T01:47:47.725 INFO:tasks.rados.rados.0.smithi165.stdout:155: rmattr oid 11 current snap is 8 2024-04-20T01:47:47.725 INFO:tasks.rados.rados.0.smithi165.stdout:155: done (2 left) 2024-04-20T01:47:47.725 INFO:tasks.rados.rados.0.smithi165.stdout:156: read oid 12 snap 6 2024-04-20T01:47:47.725 INFO:tasks.rados.rados.0.smithi165.stdout:156: expect (ObjNum 11 snap 0 seq_num 11) 2024-04-20T01:47:47.725 INFO:tasks.rados.rados.0.smithi165.stdout:157: read oid 3 snap -1 2024-04-20T01:47:47.725 INFO:tasks.rados.rados.0.smithi165.stdout:157: expect (ObjNum 25 snap 0 seq_num 25) 2024-04-20T01:47:47.725 INFO:tasks.rados.rados.0.smithi165.stdout:158: snap_remove snap 4 2024-04-20T01:47:47.778 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:47:47.792+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:47:47.869 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:47.869 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:47.879 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-20T01:47:47.879 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:47.893+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:47.979 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:47.994+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:47:48.048 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:48.048 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:48.059 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-20T01:47:48.080 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:48.094+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:47:48.081 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-20T01:47:48.181 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:47:48.196+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:47:48.187 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:48.187 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:48.197 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-20T01:47:48.243 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:47:48.265 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:47:48.281 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:48.296+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:48.286 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:47:48.308 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:47:48.327 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:48.327 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:48.328 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:47:48.337 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-20T01:47:48.349 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:47:48.370 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:47:48.391 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:47:48.412 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:47:48.421 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:48.436+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:48.433 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:47:48.454 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:47:48.475 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:47:48.483 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:47:48.498+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:47:48.496 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:47:48.506 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:48.506 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:47:48.506 INFO:teuthology.orchestra.run.smithi121.stdout: "complaint_time": 30, 2024-04-20T01:47:48.506 INFO:teuthology.orchestra.run.smithi121.stdout: "num_blocked_ops": 0 2024-04-20T01:47:48.506 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:48.516 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-20T01:47:48.517 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:47:48.538 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:47:48.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:47:48.553+0000 7f3f5491a700 0 -- 172.21.15.44:0/825444044 >> [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] conn(0x7f3f5021f560 msgr2=0x7f3f50221950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-20T01:47:48.583 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:47:48.598+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:47:48.684 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:48.699+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:47:48.687 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:48.687 INFO:teuthology.orchestra.run.smithi121.stdout: "size": 20, 2024-04-20T01:47:48.687 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 600, 2024-04-20T01:47:48.687 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-04-20T01:47:48.687 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.687 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.6s6 33/30 ECSubWrite(tid=0, reqid=client.4643.0:22, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:48.687 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.759301+0000", 2024-04-20T01:47:48.687 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 13.941005766, 2024-04-20T01:47:48.687 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0067130840000000002, 2024-04-20T01:47:48.687 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:48.687 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:48.687 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:48.687 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.688 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:48.688 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.759301+0000", 2024-04-20T01:47:48.688 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.688 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.688 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.688 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:48.688 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.759301+0000", 2024-04-20T01:47:48.688 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.688 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.688 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.688 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:48.688 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.759300+0000", 2024-04-20T01:47:48.688 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:48.688 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.688 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.689 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:48.689 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.759499+0000", 2024-04-20T01:47:48.689 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000198471 2024-04-20T01:47:48.689 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.689 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.689 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:48.689 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.759538+0000", 2024-04-20T01:47:48.689 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.9181999999999999e-05 2024-04-20T01:47:48.689 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.689 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.689 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:48.689 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.759554+0000", 2024-04-20T01:47:48.689 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.592e-05 2024-04-20T01:47:48.689 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.689 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.690 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:48.690 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.759728+0000", 2024-04-20T01:47:48.690 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00017346399999999999 2024-04-20T01:47:48.690 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.690 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.690 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:48.690 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.759745+0000", 2024-04-20T01:47:48.690 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7442999999999999e-05 2024-04-20T01:47:48.690 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.690 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.690 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:48.690 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.765987+0000", 2024-04-20T01:47:48.690 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0062419279999999999 2024-04-20T01:47:48.690 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.693 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.694 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:48.694 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766014+0000", 2024-04-20T01:47:48.694 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6971e-05 2024-04-20T01:47:48.694 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.694 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.694 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.694 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.694 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.694 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.6s6 33/30 ECSubWrite(tid=1, reqid=client.4643.0:23, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:48.694 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.762401+0000", 2024-04-20T01:47:48.694 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 13.937905546, 2024-04-20T01:47:48.694 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0036366409999999999, 2024-04-20T01:47:48.694 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:48.694 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:48.695 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:48.695 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.695 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:48.695 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.762401+0000", 2024-04-20T01:47:48.695 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.695 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.695 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.695 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:48.695 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.762401+0000", 2024-04-20T01:47:48.696 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.696 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.696 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.696 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:48.696 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.762400+0000", 2024-04-20T01:47:48.696 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:48.696 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.697 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.697 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:48.697 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.762432+0000", 2024-04-20T01:47:48.697 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.1528e-05 2024-04-20T01:47:48.697 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.697 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.697 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:48.697 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.762455+0000", 2024-04-20T01:47:48.697 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3351000000000001e-05 2024-04-20T01:47:48.697 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.697 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.697 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:48.697 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.762476+0000", 2024-04-20T01:47:48.697 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1426000000000001e-05 2024-04-20T01:47:48.697 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.698 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.698 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:48.698 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.762569+0000", 2024-04-20T01:47:48.698 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2276999999999998e-05 2024-04-20T01:47:48.698 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.698 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.698 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:48.698 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.762580+0000", 2024-04-20T01:47:48.698 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1496e-05 2024-04-20T01:47:48.698 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.698 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.698 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:48.698 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766023+0000", 2024-04-20T01:47:48.698 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0034432 2024-04-20T01:47:48.698 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.699 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.699 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:48.699 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766038+0000", 2024-04-20T01:47:48.699 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4182000000000001e-05 2024-04-20T01:47:48.699 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.699 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.699 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.699 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.699 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.699 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 33/30 ECSubWrite(tid=0, reqid=client.4643.0:37, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:48.699 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.765612+0000", 2024-04-20T01:47:48.699 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 13.934694078, 2024-04-20T01:47:48.699 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.006884584, 2024-04-20T01:47:48.699 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:48.700 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:48.700 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:48.700 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.700 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:48.700 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.765612+0000", 2024-04-20T01:47:48.700 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.700 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.700 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.700 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:48.700 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.765612+0000", 2024-04-20T01:47:48.700 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.700 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.700 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.700 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:48.700 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.765612+0000", 2024-04-20T01:47:48.701 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:48.701 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.701 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.701 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:48.701 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766107+0000", 2024-04-20T01:47:48.701 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00049509399999999996 2024-04-20T01:47:48.701 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.701 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.701 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:48.701 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766119+0000", 2024-04-20T01:47:48.701 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.168e-05 2024-04-20T01:47:48.701 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.701 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.701 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:48.701 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766128+0000", 2024-04-20T01:47:48.701 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.1169999999999993e-06 2024-04-20T01:47:48.702 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.702 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.702 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:48.702 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766186+0000", 2024-04-20T01:47:48.702 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.8236000000000003e-05 2024-04-20T01:47:48.702 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.702 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.702 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:48.702 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766196+0000", 2024-04-20T01:47:48.702 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3680000000000008e-06 2024-04-20T01:47:48.702 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.702 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.702 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:48.702 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.772468+0000", 2024-04-20T01:47:48.703 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0062727529999999998 2024-04-20T01:47:48.703 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.703 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.703 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:48.703 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.772497+0000", 2024-04-20T01:47:48.703 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8759999999999999e-05 2024-04-20T01:47:48.703 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.703 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.703 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.703 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.704 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.704 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 33/30 ECSubWrite(tid=1, reqid=client.4643.0:38, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:48.704 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.767018+0000", 2024-04-20T01:47:48.704 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 13.933289030999999, 2024-04-20T01:47:48.704 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0055026670000000001, 2024-04-20T01:47:48.704 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:48.704 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:48.704 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:48.704 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.704 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:48.704 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.767018+0000", 2024-04-20T01:47:48.704 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.704 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.704 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.704 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:48.705 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.767018+0000", 2024-04-20T01:47:48.705 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.705 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.705 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.705 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:48.705 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.767017+0000", 2024-04-20T01:47:48.705 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:48.705 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.705 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.705 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:48.705 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.767046+0000", 2024-04-20T01:47:48.705 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9116999999999999e-05 2024-04-20T01:47:48.705 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.705 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.705 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:48.706 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.767065+0000", 2024-04-20T01:47:48.706 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8819000000000001e-05 2024-04-20T01:47:48.706 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.707 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.707 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:48.707 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.767080+0000", 2024-04-20T01:47:48.707 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5055000000000001e-05 2024-04-20T01:47:48.707 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.707 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.707 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:48.707 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.767136+0000", 2024-04-20T01:47:48.707 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6174000000000001e-05 2024-04-20T01:47:48.707 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.707 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.707 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:48.707 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.767146+0000", 2024-04-20T01:47:48.707 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0003999999999999e-05 2024-04-20T01:47:48.708 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.708 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.708 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:48.708 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.772506+0000", 2024-04-20T01:47:48.708 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0053601109999999999 2024-04-20T01:47:48.708 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.708 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.708 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:48.708 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.772520+0000", 2024-04-20T01:47:48.708 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4250000000000001e-05 2024-04-20T01:47:48.708 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.708 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.708 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.708 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.708 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.709 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.6s6 33/30 ECSubWrite(tid=2, reqid=client.4643.0:23, at_version=0'0, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:47:48.709 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.769473+0000", 2024-04-20T01:47:48.709 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 13.930833529999999, 2024-04-20T01:47:48.709 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0026127820000000001, 2024-04-20T01:47:48.709 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:48.709 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:48.709 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:48.709 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.709 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:48.709 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.769473+0000", 2024-04-20T01:47:48.709 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.709 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.709 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.709 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:48.709 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.769473+0000", 2024-04-20T01:47:48.709 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.710 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.710 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.710 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:48.710 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.769472+0000", 2024-04-20T01:47:48.710 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:48.710 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.710 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.710 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:48.710 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.769487+0000", 2024-04-20T01:47:48.710 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5296000000000001e-05 2024-04-20T01:47:48.710 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.711 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.711 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:48.711 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.769506+0000", 2024-04-20T01:47:48.711 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8501000000000001e-05 2024-04-20T01:47:48.711 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.711 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.711 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:48.711 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.769516+0000", 2024-04-20T01:47:48.711 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0294e-05 2024-04-20T01:47:48.711 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.711 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.711 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:48.711 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.769633+0000", 2024-04-20T01:47:48.711 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00011742 2024-04-20T01:47:48.711 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.712 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.712 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:48.712 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.769644+0000", 2024-04-20T01:47:48.712 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0353999999999999e-05 2024-04-20T01:47:48.712 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.712 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.712 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:48.712 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.772047+0000", 2024-04-20T01:47:48.712 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0024028930000000001 2024-04-20T01:47:48.712 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.712 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.712 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:48.712 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.772086+0000", 2024-04-20T01:47:48.712 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.909e-05 2024-04-20T01:47:48.712 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.712 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.713 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.713 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.713 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.713 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 33/30 ECSubWrite(tid=2, reqid=client.4643.0:38, at_version=0'0, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:47:48.713 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.778467+0000", 2024-04-20T01:47:48.713 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 13.921839110000001, 2024-04-20T01:47:48.713 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0011701940000000001, 2024-04-20T01:47:48.713 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:48.713 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:48.713 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:48.713 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.714 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:48.714 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.778467+0000", 2024-04-20T01:47:48.714 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.714 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.714 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.714 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:48.714 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.778467+0000", 2024-04-20T01:47:48.714 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.714 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.714 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.714 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:48.714 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.778466+0000", 2024-04-20T01:47:48.714 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:48.714 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.714 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.715 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:48.715 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.778482+0000", 2024-04-20T01:47:48.715 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6044000000000001e-05 2024-04-20T01:47:48.715 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.715 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.715 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:48.715 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.778500+0000", 2024-04-20T01:47:48.715 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7533999999999998e-05 2024-04-20T01:47:48.715 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.715 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.715 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:48.715 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.778513+0000", 2024-04-20T01:47:48.715 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.312e-05 2024-04-20T01:47:48.715 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.715 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.716 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:48.716 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.778564+0000", 2024-04-20T01:47:48.716 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.1365e-05 2024-04-20T01:47:48.716 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.716 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.716 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:48.716 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.778577+0000", 2024-04-20T01:47:48.716 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.31e-05 2024-04-20T01:47:48.716 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.716 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.716 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:48.716 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.779605+0000", 2024-04-20T01:47:48.717 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001027109 2024-04-20T01:47:48.717 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.717 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.717 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:48.717 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.779638+0000", 2024-04-20T01:47:48.717 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.3093000000000003e-05 2024-04-20T01:47:48.717 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.717 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.717 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.717 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.717 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.717 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.1s2 36/30 ECSubWrite(tid=55, reqid=osd.4.0:55, at_version=36'8, trim_to=0'0, roll_forward_to=35'6))", 2024-04-20T01:47:48.718 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:37.766954+0000", 2024-04-20T01:47:48.718 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 10.933352095, 2024-04-20T01:47:48.718 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001328206, 2024-04-20T01:47:48.718 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:48.718 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:48.718 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:48.718 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.718 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:48.718 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.766954+0000", 2024-04-20T01:47:48.718 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.718 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.718 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.718 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:48.718 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.766954+0000", 2024-04-20T01:47:48.719 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.719 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.719 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.719 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:48.719 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.766954+0000", 2024-04-20T01:47:48.719 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:47:48.719 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.719 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.719 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:48.719 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.766980+0000", 2024-04-20T01:47:48.719 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6001000000000001e-05 2024-04-20T01:47:48.719 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.719 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.719 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:48.719 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.766990+0000", 2024-04-20T01:47:48.720 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.8970000000000003e-06 2024-04-20T01:47:48.720 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.720 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.720 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:48.720 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.766999+0000", 2024-04-20T01:47:48.720 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.0399999999999998e-06 2024-04-20T01:47:48.720 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.720 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.720 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:48.720 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.767059+0000", 2024-04-20T01:47:48.720 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.9673999999999998e-05 2024-04-20T01:47:48.720 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.720 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.720 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:48.721 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.767068+0000", 2024-04-20T01:47:48.721 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4949999999999997e-06 2024-04-20T01:47:48.721 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.721 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.721 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:48.721 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.768253+0000", 2024-04-20T01:47:48.721 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0011844570000000001 2024-04-20T01:47:48.721 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.721 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.721 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:48.721 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.768283+0000", 2024-04-20T01:47:48.722 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9813999999999999e-05 2024-04-20T01:47:48.722 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.722 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.722 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.722 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.722 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.722 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s2 36/30 ECSubWrite(tid=57, reqid=client.4643.0:210, at_version=36'7, trim_to=0'0, roll_forward_to=33'5))", 2024-04-20T01:47:48.722 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:37.778635+0000", 2024-04-20T01:47:48.722 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 10.921671286, 2024-04-20T01:47:48.722 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0022454100000000002, 2024-04-20T01:47:48.722 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:48.722 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:48.722 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:48.722 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.722 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:48.723 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.778635+0000", 2024-04-20T01:47:48.723 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.723 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.723 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.723 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:48.723 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.778635+0000", 2024-04-20T01:47:48.723 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.723 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.723 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.723 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:48.723 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.778634+0000", 2024-04-20T01:47:48.723 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:48.723 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.723 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.723 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:48.724 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.779077+0000", 2024-04-20T01:47:48.724 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00044289399999999999 2024-04-20T01:47:48.724 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.724 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.724 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:48.724 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.779092+0000", 2024-04-20T01:47:48.724 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.437e-05 2024-04-20T01:47:48.724 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.724 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.724 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:48.724 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.779103+0000", 2024-04-20T01:47:48.724 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1749000000000001e-05 2024-04-20T01:47:48.724 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.725 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.725 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:48.725 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.779162+0000", 2024-04-20T01:47:48.725 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.8870999999999999e-05 2024-04-20T01:47:48.725 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.725 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.725 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:48.725 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.779173+0000", 2024-04-20T01:47:48.725 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0618999999999999e-05 2024-04-20T01:47:48.725 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.725 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.725 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:48.725 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.780852+0000", 2024-04-20T01:47:48.725 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001679238 2024-04-20T01:47:48.725 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.726 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.726 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:48.726 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:37.780881+0000", 2024-04-20T01:47:48.726 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8459e-05 2024-04-20T01:47:48.726 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.726 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.726 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.726 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.726 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.726 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 37/30 ECSubWrite(tid=64, reqid=client.4643.0:225, at_version=37'16, trim_to=0'0, roll_forward_to=36'14))", 2024-04-20T01:47:48.726 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.791284+0000", 2024-04-20T01:47:48.726 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 9.9090227370000008, 2024-04-20T01:47:48.726 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0047182170000000002, 2024-04-20T01:47:48.727 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:48.727 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:48.727 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:48.727 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.727 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:48.727 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791284+0000", 2024-04-20T01:47:48.727 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.727 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.727 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.728 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:48.728 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791284+0000", 2024-04-20T01:47:48.728 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.728 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.728 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.728 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:48.728 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791284+0000", 2024-04-20T01:47:48.728 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:47:48.728 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.728 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.728 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:48.728 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791637+0000", 2024-04-20T01:47:48.728 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000353075 2024-04-20T01:47:48.728 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.728 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.728 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:48.729 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791652+0000", 2024-04-20T01:47:48.729 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5625e-05 2024-04-20T01:47:48.729 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.729 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.729 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:48.729 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791665+0000", 2024-04-20T01:47:48.729 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2814e-05 2024-04-20T01:47:48.729 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.729 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.729 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:48.729 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791738+0000", 2024-04-20T01:47:48.729 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.2700999999999994e-05 2024-04-20T01:47:48.729 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.729 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.729 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:48.730 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791748+0000", 2024-04-20T01:47:48.730 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.9899999999999992e-06 2024-04-20T01:47:48.730 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.730 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.730 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:48.730 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795975+0000", 2024-04-20T01:47:48.730 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0042274510000000001 2024-04-20T01:47:48.730 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.730 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.730 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:48.730 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796002+0000", 2024-04-20T01:47:48.730 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6766000000000001e-05 2024-04-20T01:47:48.730 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.731 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.731 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.731 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.731 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.731 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 37/30 ECSubWrite(tid=65, reqid=client.4643.0:226, at_version=37'17, trim_to=0'0, roll_forward_to=36'14))", 2024-04-20T01:47:48.731 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.791683+0000", 2024-04-20T01:47:48.731 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 9.9086234019999999, 2024-04-20T01:47:48.731 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0043386420000000002, 2024-04-20T01:47:48.731 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:48.731 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:48.731 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:48.731 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.731 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:48.732 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791683+0000", 2024-04-20T01:47:48.732 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.732 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.732 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.732 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:48.732 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791683+0000", 2024-04-20T01:47:48.732 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.732 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.732 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.732 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:48.732 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791683+0000", 2024-04-20T01:47:48.732 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:48.732 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.732 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.732 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:48.733 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791807+0000", 2024-04-20T01:47:48.733 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00012426399999999999 2024-04-20T01:47:48.733 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.733 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.733 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:48.733 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791817+0000", 2024-04-20T01:47:48.733 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0079e-05 2024-04-20T01:47:48.733 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.733 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.733 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:48.733 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791823+0000", 2024-04-20T01:47:48.733 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.3609999999999997e-06 2024-04-20T01:47:48.733 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.733 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.734 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:48.734 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793245+0000", 2024-04-20T01:47:48.734 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0014213730000000001 2024-04-20T01:47:48.734 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.734 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.734 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:48.734 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793255+0000", 2024-04-20T01:47:48.734 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0278999999999999e-05 2024-04-20T01:47:48.734 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.734 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.734 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:48.734 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796010+0000", 2024-04-20T01:47:48.734 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0027550259999999998 2024-04-20T01:47:48.734 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.734 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796022+0000", 2024-04-20T01:47:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1754e-05 2024-04-20T01:47:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 37/30 ECSubWrite(tid=66, reqid=client.4643.0:227, at_version=37'18, trim_to=0'0, roll_forward_to=36'14))", 2024-04-20T01:47:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.791832+0000", 2024-04-20T01:47:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 9.9084742440000007, 2024-04-20T01:47:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0042145630000000002, 2024-04-20T01:47:48.735 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791832+0000", 2024-04-20T01:47:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791832+0000", 2024-04-20T01:47:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791832+0000", 2024-04-20T01:47:48.736 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:48.737 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.737 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.737 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:48.737 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791856+0000", 2024-04-20T01:47:48.737 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4837999999999998e-05 2024-04-20T01:47:48.737 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.737 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.737 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:48.737 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791866+0000", 2024-04-20T01:47:48.737 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.0110000000000006e-06 2024-04-20T01:47:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791890+0000", 2024-04-20T01:47:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4936999999999999e-05 2024-04-20T01:47:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793767+0000", 2024-04-20T01:47:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001876827 2024-04-20T01:47:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793779+0000", 2024-04-20T01:47:48.738 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1620000000000001e-05 2024-04-20T01:47:48.739 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.739 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.739 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:48.739 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796029+0000", 2024-04-20T01:47:48.739 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00225012 2024-04-20T01:47:48.739 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.739 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.739 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:48.739 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796047+0000", 2024-04-20T01:47:48.739 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7844000000000001e-05 2024-04-20T01:47:48.739 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.739 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.739 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.739 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.739 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.739 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 38/30 ECSubWrite(tid=73, reqid=osd.4.0:73, at_version=38'22, trim_to=0'0, roll_forward_to=37'18))", 2024-04-20T01:47:48.740 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:39.792847+0000", 2024-04-20T01:47:48.740 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 8.9074599980000002, 2024-04-20T01:47:48.740 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0061022330000000003, 2024-04-20T01:47:48.740 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:48.740 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:48.740 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:48.740 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.740 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:48.740 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.792847+0000", 2024-04-20T01:47:48.740 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.740 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.740 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.740 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:48.740 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.792847+0000", 2024-04-20T01:47:48.740 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.741 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.741 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.741 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:48.741 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.792846+0000", 2024-04-20T01:47:48.741 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:48.741 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.741 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.741 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:48.741 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.792867+0000", 2024-04-20T01:47:48.741 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1022999999999998e-05 2024-04-20T01:47:48.741 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.741 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.741 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:48.741 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.792879+0000", 2024-04-20T01:47:48.742 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2242000000000001e-05 2024-04-20T01:47:48.742 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.742 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.742 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:48.742 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.792894+0000", 2024-04-20T01:47:48.742 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4515000000000001e-05 2024-04-20T01:47:48.742 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.742 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.742 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:48.742 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.792940+0000", 2024-04-20T01:47:48.742 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.6592999999999999e-05 2024-04-20T01:47:48.742 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.742 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.742 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:48.742 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.792956+0000", 2024-04-20T01:47:48.743 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5962999999999999e-05 2024-04-20T01:47:48.743 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.743 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.743 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:48.743 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.798920+0000", 2024-04-20T01:47:48.743 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0059632720000000004 2024-04-20T01:47:48.743 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.743 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.743 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:48.743 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.798949+0000", 2024-04-20T01:47:48.743 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9196999999999998e-05 2024-04-20T01:47:48.743 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.743 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.743 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.743 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.743 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.744 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 38/30 ECSubWrite(tid=74, reqid=osd.4.0:73, at_version=0'0, trim_to=0'0, roll_forward_to=38'22))", 2024-04-20T01:47:48.744 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:39.799475+0000", 2024-04-20T01:47:48.744 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 8.9008316080000007, 2024-04-20T01:47:48.744 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.004862295, 2024-04-20T01:47:48.744 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:48.744 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:48.744 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:48.744 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.744 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:48.744 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.799475+0000", 2024-04-20T01:47:48.744 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.744 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.744 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.744 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:48.745 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.799475+0000", 2024-04-20T01:47:48.745 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.745 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.745 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.745 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:48.745 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.799474+0000", 2024-04-20T01:47:48.745 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:48.745 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.745 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.745 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:48.745 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.799483+0000", 2024-04-20T01:47:48.745 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.1619999999999997e-06 2024-04-20T01:47:48.745 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.745 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.745 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:48.746 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.799498+0000", 2024-04-20T01:47:48.746 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4168999999999999e-05 2024-04-20T01:47:48.746 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.746 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.746 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:48.746 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.799507+0000", 2024-04-20T01:47:48.746 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2499999999999995e-06 2024-04-20T01:47:48.746 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.746 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.746 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:48.746 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.799544+0000", 2024-04-20T01:47:48.746 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7020999999999997e-05 2024-04-20T01:47:48.746 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.746 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.746 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:48.747 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.799553+0000", 2024-04-20T01:47:48.747 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2089999999999994e-06 2024-04-20T01:47:48.747 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.747 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.747 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:48.747 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.804307+0000", 2024-04-20T01:47:48.747 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0047538260000000001 2024-04-20T01:47:48.747 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.747 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.747 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:48.747 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.804337+0000", 2024-04-20T01:47:48.748 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0272000000000001e-05 2024-04-20T01:47:48.748 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.748 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.748 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.748 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.748 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.748 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s2 38/30 ECSubWrite(tid=75, reqid=client.4643.0:254, at_version=38'10, trim_to=0'0, roll_forward_to=36'9))", 2024-04-20T01:47:48.748 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:39.805339+0000", 2024-04-20T01:47:48.748 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 8.8949677099999995, 2024-04-20T01:47:48.748 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0016987910000000001, 2024-04-20T01:47:48.748 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:48.748 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:48.748 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:48.748 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.748 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:48.749 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805339+0000", 2024-04-20T01:47:48.749 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.749 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.749 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.749 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:48.749 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805339+0000", 2024-04-20T01:47:48.749 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.749 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.749 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.749 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:48.749 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805334+0000", 2024-04-20T01:47:48.749 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999948 2024-04-20T01:47:48.749 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.749 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.749 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:48.750 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805552+0000", 2024-04-20T01:47:48.750 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00021822700000000001 2024-04-20T01:47:48.750 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.750 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.750 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:48.750 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805569+0000", 2024-04-20T01:47:48.750 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6847e-05 2024-04-20T01:47:48.750 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.750 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.750 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:48.750 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805577+0000", 2024-04-20T01:47:48.750 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.755e-06 2024-04-20T01:47:48.750 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.750 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.751 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:48.751 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805624+0000", 2024-04-20T01:47:48.751 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.6826999999999997e-05 2024-04-20T01:47:48.751 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.751 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.751 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:48.751 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.805636+0000", 2024-04-20T01:47:48.751 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1482e-05 2024-04-20T01:47:48.751 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.751 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.751 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:48.751 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.807018+0000", 2024-04-20T01:47:48.751 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001382557 2024-04-20T01:47:48.751 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.751 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.752 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:48.752 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.807038+0000", 2024-04-20T01:47:48.752 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9295000000000001e-05 2024-04-20T01:47:48.752 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.752 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.752 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.752 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.752 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.752 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 42/30 ECSubWrite(tid=91, reqid=client.4643.0:288, at_version=42'32, trim_to=0'0, roll_forward_to=41'31))", 2024-04-20T01:47:48.752 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.808712+0000", 2024-04-20T01:47:48.752 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 4.8915948880000002, 2024-04-20T01:47:48.752 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0015617210000000001, 2024-04-20T01:47:48.752 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:48.752 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:48.752 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:48.753 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.753 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:48.753 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808712+0000", 2024-04-20T01:47:48.753 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.753 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.753 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.753 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:48.753 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808712+0000", 2024-04-20T01:47:48.753 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.753 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.753 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.753 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:48.753 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808711+0000", 2024-04-20T01:47:48.753 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:48.753 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.753 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.754 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:48.754 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808891+0000", 2024-04-20T01:47:48.754 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00017941499999999999 2024-04-20T01:47:48.754 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.754 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.754 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:48.754 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808907+0000", 2024-04-20T01:47:48.754 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6356000000000001e-05 2024-04-20T01:47:48.754 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.754 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.754 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:48.754 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808919+0000", 2024-04-20T01:47:48.754 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1379000000000001e-05 2024-04-20T01:47:48.754 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.754 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.755 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:48.755 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808975+0000", 2024-04-20T01:47:48.755 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6811000000000002e-05 2024-04-20T01:47:48.755 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.755 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.755 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:48.755 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808984+0000", 2024-04-20T01:47:48.755 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8859999999999993e-06 2024-04-20T01:47:48.755 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.755 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.755 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:48.755 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.810244+0000", 2024-04-20T01:47:48.755 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001259584 2024-04-20T01:47:48.755 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.755 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.756 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:48.756 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.810273+0000", 2024-04-20T01:47:48.756 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9558999999999999e-05 2024-04-20T01:47:48.756 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.756 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.756 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.756 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.756 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.756 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 42/30 ECSubWrite(tid=92, reqid=client.4643.0:297, at_version=42'33, trim_to=0'0, roll_forward_to=41'31))", 2024-04-20T01:47:48.756 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.810386+0000", 2024-04-20T01:47:48.756 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 4.8899202299999995, 2024-04-20T01:47:48.756 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0049351079999999997, 2024-04-20T01:47:48.756 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:48.756 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:48.756 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:48.757 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.757 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:48.757 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.810386+0000", 2024-04-20T01:47:48.757 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.757 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.757 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.757 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:48.757 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.810386+0000", 2024-04-20T01:47:48.757 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.757 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.757 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.757 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:48.758 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.810385+0000", 2024-04-20T01:47:48.758 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:48.758 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.758 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.758 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:48.758 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.810471+0000", 2024-04-20T01:47:48.758 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.5270000000000002e-05 2024-04-20T01:47:48.758 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.758 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.758 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:48.758 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.810497+0000", 2024-04-20T01:47:48.758 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6508000000000001e-05 2024-04-20T01:47:48.758 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.758 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.759 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:48.759 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.810507+0000", 2024-04-20T01:47:48.759 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0399e-05 2024-04-20T01:47:48.759 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.759 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.759 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:48.759 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.811310+0000", 2024-04-20T01:47:48.759 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00080240200000000002 2024-04-20T01:47:48.759 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.759 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.759 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:48.759 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.811320+0000", 2024-04-20T01:47:48.759 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0536999999999999e-05 2024-04-20T01:47:48.759 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.759 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.760 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:48.760 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.815282+0000", 2024-04-20T01:47:48.760 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0039620690000000004 2024-04-20T01:47:48.760 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.760 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.760 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:48.760 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.815321+0000", 2024-04-20T01:47:48.760 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.8971999999999999e-05 2024-04-20T01:47:48.760 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.760 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.760 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.760 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.760 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.760 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 42/30 ECSubWrite(tid=94, reqid=client.4643.0:297, at_version=0'0, trim_to=0'0, roll_forward_to=42'33))", 2024-04-20T01:47:48.761 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.816307+0000", 2024-04-20T01:47:48.761 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 4.8839997630000003, 2024-04-20T01:47:48.761 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0026243590000000002, 2024-04-20T01:47:48.761 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:48.761 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:48.761 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:48.761 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.761 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:48.761 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.816307+0000", 2024-04-20T01:47:48.761 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.761 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.761 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.761 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:48.761 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.816307+0000", 2024-04-20T01:47:48.761 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.762 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.762 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.762 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:48.762 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.816305+0000", 2024-04-20T01:47:48.762 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:48.762 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.762 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.762 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:48.762 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.816320+0000", 2024-04-20T01:47:48.762 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4335000000000001e-05 2024-04-20T01:47:48.762 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.762 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.762 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:48.762 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.816336+0000", 2024-04-20T01:47:48.762 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6685999999999999e-05 2024-04-20T01:47:48.763 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.763 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.763 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:48.763 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.816347+0000", 2024-04-20T01:47:48.763 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0217000000000001e-05 2024-04-20T01:47:48.763 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.763 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.763 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:48.763 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.816415+0000", 2024-04-20T01:47:48.763 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.8292000000000003e-05 2024-04-20T01:47:48.763 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.763 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.763 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:48.763 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.816427+0000", 2024-04-20T01:47:48.763 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2061e-05 2024-04-20T01:47:48.764 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.764 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.764 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:48.764 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.818899+0000", 2024-04-20T01:47:48.764 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0024714649999999999 2024-04-20T01:47:48.764 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.764 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.764 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:48.764 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.818931+0000", 2024-04-20T01:47:48.764 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.2607999999999997e-05 2024-04-20T01:47:48.764 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.764 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.764 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.764 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.765 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.765 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.es1 42/30 ECSubWrite(tid=30, reqid=client.4643.0:309, at_version=42'20, trim_to=0'0, roll_forward_to=41'18))", 2024-04-20T01:47:48.765 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.824449+0000", 2024-04-20T01:47:48.765 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 4.8758578740000003, 2024-04-20T01:47:48.765 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0027442870000000002, 2024-04-20T01:47:48.765 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:48.765 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:48.765 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:48.765 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.765 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:48.765 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.824449+0000", 2024-04-20T01:47:48.765 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.765 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.765 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.765 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:48.766 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.824449+0000", 2024-04-20T01:47:48.766 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.766 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.766 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.766 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:48.766 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.824448+0000", 2024-04-20T01:47:48.766 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:48.766 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.766 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.766 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:48.766 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.824472+0000", 2024-04-20T01:47:48.766 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3934000000000002e-05 2024-04-20T01:47:48.767 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.767 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.767 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:48.767 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.824488+0000", 2024-04-20T01:47:48.767 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6364000000000001e-05 2024-04-20T01:47:48.767 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.767 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.767 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:48.767 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.824499+0000", 2024-04-20T01:47:48.767 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1031e-05 2024-04-20T01:47:48.767 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.767 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.767 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:48.767 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.824562+0000", 2024-04-20T01:47:48.768 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.2695999999999997e-05 2024-04-20T01:47:48.768 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.768 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.768 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:48.768 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.824573+0000", 2024-04-20T01:47:48.768 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0873000000000001e-05 2024-04-20T01:47:48.768 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.768 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.768 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:48.768 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.827165+0000", 2024-04-20T01:47:48.768 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0025917919999999999 2024-04-20T01:47:48.768 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.769 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.769 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:48.769 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.827193+0000", 2024-04-20T01:47:48.769 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8127999999999999e-05 2024-04-20T01:47:48.769 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.769 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.769 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.769 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.769 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.769 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 42/30 ECSubWrite(tid=96, reqid=client.4643.0:316, at_version=42'35, trim_to=0'0, roll_forward_to=42'33))", 2024-04-20T01:47:48.769 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.924091+0000", 2024-04-20T01:47:48.769 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 4.7762157270000003, 2024-04-20T01:47:48.769 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0015012630000000001, 2024-04-20T01:47:48.769 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:48.769 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:48.770 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:48.770 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.770 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:48.770 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.924091+0000", 2024-04-20T01:47:48.770 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.770 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.770 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.770 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:48.770 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.924091+0000", 2024-04-20T01:47:48.770 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.770 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.770 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.770 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:48.770 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.924089+0000", 2024-04-20T01:47:48.770 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:48.771 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.771 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.771 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:48.771 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.924200+0000", 2024-04-20T01:47:48.771 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000110794 2024-04-20T01:47:48.771 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.771 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.771 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:48.771 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.924220+0000", 2024-04-20T01:47:48.771 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9734000000000001e-05 2024-04-20T01:47:48.771 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.771 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.771 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:48.771 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.924233+0000", 2024-04-20T01:47:48.771 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3055e-05 2024-04-20T01:47:48.772 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.772 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.772 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:48.772 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.924323+0000", 2024-04-20T01:47:48.772 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.9721000000000004e-05 2024-04-20T01:47:48.772 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.772 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.772 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:48.772 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.924336+0000", 2024-04-20T01:47:48.772 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2948999999999999e-05 2024-04-20T01:47:48.772 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.772 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.772 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:48.772 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.925565+0000", 2024-04-20T01:47:48.772 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0012294669999999999 2024-04-20T01:47:48.772 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.773 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.773 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:48.773 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.925592+0000", 2024-04-20T01:47:48.773 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7093999999999998e-05 2024-04-20T01:47:48.773 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.773 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.773 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.773 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.773 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.773 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 42/30 ECSubWrite(tid=98, reqid=client.4643.0:321, at_version=42'36, trim_to=0'0, roll_forward_to=42'35))", 2024-04-20T01:47:48.773 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.944425+0000", 2024-04-20T01:47:48.773 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 4.7558813549999996, 2024-04-20T01:47:48.773 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0013637759999999999, 2024-04-20T01:47:48.773 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:48.773 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:48.774 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:48.774 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.774 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:48.774 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.944425+0000", 2024-04-20T01:47:48.774 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.774 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.774 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.774 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:48.774 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.944425+0000", 2024-04-20T01:47:48.774 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:48.774 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.774 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.774 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:48.774 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.944424+0000", 2024-04-20T01:47:48.774 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:48.775 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.775 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.775 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:48.775 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.944649+0000", 2024-04-20T01:47:48.775 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000224818 2024-04-20T01:47:48.775 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.775 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.775 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:48.775 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.944663+0000", 2024-04-20T01:47:48.775 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3818000000000001e-05 2024-04-20T01:47:48.775 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.775 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.775 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:48.775 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.944670+0000", 2024-04-20T01:47:48.775 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.7919999999999993e-06 2024-04-20T01:47:48.776 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.776 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.776 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:48.776 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.944725+0000", 2024-04-20T01:47:48.776 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.4552999999999998e-05 2024-04-20T01:47:48.776 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.776 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.776 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:48.776 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.944737+0000", 2024-04-20T01:47:48.776 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2174000000000001e-05 2024-04-20T01:47:48.776 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.776 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.776 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:48.776 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.945758+0000", 2024-04-20T01:47:48.776 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0010212680000000001 2024-04-20T01:47:48.777 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:48.777 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:48.777 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:48.777 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.945789+0000", 2024-04-20T01:47:48.777 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0499000000000001e-05 2024-04-20T01:47:48.777 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.777 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.777 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.777 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:48.777 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:48.777 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:48.780 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.14 to 1 (10000) 2024-04-20T01:47:48.780 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-20T01:47:48.781 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 15 1 2024-04-20T01:47:48.781 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:47:48.783 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:47:48.783 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:47:48.783 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:47:48.783 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:47:48.783 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:47:48.783 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:47:48.783 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:47:48.783 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:47:48.783 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:47:48.783 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:47:48.783 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:47:48.784 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:47:48.784 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:47:48.784 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:47:48.784 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:47:48.784 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:47:48.784 INFO:tasks.rados.rados.0.smithi165.stdout:158: done (4 left) 2024-04-20T01:47:48.784 INFO:tasks.rados.rados.0.smithi165.stdout:159: copy_from oid 21 from oid 14 current snap is 8 2024-04-20T01:47:48.784 INFO:tasks.rados.rados.0.smithi165.stdout:153: finishing write tid 1 to smithi16579500-47 2024-04-20T01:47:48.784 INFO:tasks.rados.rados.0.smithi165.stdout:153: finishing write tid 2 to smithi16579500-47 2024-04-20T01:47:48.785 INFO:tasks.rados.rados.0.smithi165.stdout:153: finishing write tid 3 to smithi16579500-47 2024-04-20T01:47:48.785 INFO:tasks.rados.rados.0.smithi165.stdout:153: finishing write tid 4 to smithi16579500-47 2024-04-20T01:47:48.785 INFO:tasks.rados.rados.0.smithi165.stdout:153: finishing write tid 5 to smithi16579500-47 2024-04-20T01:47:48.785 INFO:tasks.rados.rados.0.smithi165.stdout:153: oid 47 updating version 0 to 27 2024-04-20T01:47:48.785 INFO:tasks.rados.rados.0.smithi165.stdout:153: oid 47 updating version 27 to 28 2024-04-20T01:47:48.785 INFO:tasks.rados.rados.0.smithi165.stdout:153: oid 47 version 28 is already newer than 26 2024-04-20T01:47:48.785 INFO:tasks.rados.rados.0.smithi165.stdout:153: oid 47 updating version 28 to 29 2024-04-20T01:47:48.785 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 47 v 29 (ObjNum 80 snap 8 seq_num 80) dirty exists 2024-04-20T01:47:48.785 INFO:tasks.rados.rados.0.smithi165.stdout:153: left oid 47 (ObjNum 80 snap 8 seq_num 80) 2024-04-20T01:47:48.785 INFO:tasks.rados.rados.0.smithi165.stdout:154: finishing write tid 1 to smithi16579500-24 2024-04-20T01:47:48.785 INFO:tasks.rados.rados.0.smithi165.stdout:156: expect (ObjNum 11 snap 0 seq_num 11) 2024-04-20T01:47:48.785 INFO:tasks.rados.rados.0.smithi165.stdout:157: expect (ObjNum 25 snap 0 seq_num 25) 2024-04-20T01:47:48.785 INFO:tasks.rados.rados.0.smithi165.stdout:154: finishing write tid 2 to smithi16579500-24 2024-04-20T01:47:48.785 INFO:tasks.rados.rados.0.smithi165.stdout:154: finishing write tid 3 to smithi16579500-24 2024-04-20T01:47:48.786 INFO:tasks.rados.rados.0.smithi165.stdout:154: finishing write tid 4 to smithi16579500-24 2024-04-20T01:47:48.786 INFO:tasks.rados.rados.0.smithi165.stdout:154: finishing write tid 5 to smithi16579500-24 2024-04-20T01:47:48.786 INFO:tasks.rados.rados.0.smithi165.stdout:154: oid 24 updating version 0 to 22 2024-04-20T01:47:48.786 INFO:tasks.rados.rados.0.smithi165.stdout:154: oid 24 updating version 22 to 23 2024-04-20T01:47:48.786 INFO:tasks.rados.rados.0.smithi165.stdout:154: oid 24 updating version 23 to 25 2024-04-20T01:47:48.786 INFO:tasks.rados.rados.0.smithi165.stdout:154: oid 24 version 25 is already newer than 24 2024-04-20T01:47:48.786 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 24 v 25 (ObjNum 81 snap 8 seq_num 81) dirty exists 2024-04-20T01:47:48.786 INFO:tasks.rados.rados.0.smithi165.stdout:154: left oid 24 (ObjNum 81 snap 8 seq_num 81) 2024-04-20T01:47:48.786 INFO:tasks.rados.rados.0.smithi165.stdout:159: finishing copy_from racing read to smithi16579500-21 2024-04-20T01:47:48.786 INFO:tasks.rados.rados.0.smithi165.stdout:159: finishing copy_from to smithi16579500-21 2024-04-20T01:47:48.786 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 21 v 2 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-20T01:47:48.786 INFO:tasks.rados.rados.0.smithi165.stdout:153: done (4 left) 2024-04-20T01:47:48.786 INFO:tasks.rados.rados.0.smithi165.stdout:154: done (3 left) 2024-04-20T01:47:48.786 INFO:tasks.rados.rados.0.smithi165.stdout:156: done (2 left) 2024-04-20T01:47:48.786 INFO:tasks.rados.rados.0.smithi165.stdout:157: done (1 left) 2024-04-20T01:47:48.787 INFO:tasks.rados.rados.0.smithi165.stdout:159: done (0 left) 2024-04-20T01:47:48.787 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 17 current snap is 8 2024-04-20T01:47:48.787 INFO:tasks.rados.rados.0.smithi165.stdout:160: seq_num 82 ranges {786432=622592,1409024=475136,1884160=294912} 2024-04-20T01:47:48.787 INFO:tasks.rados.rados.0.smithi165.stdout:160: writing smithi16579500-17 from 786432 to 1409024 tid 1 2024-04-20T01:47:48.787 INFO:tasks.rados.rados.0.smithi165.stdout:160: writing smithi16579500-17 from 1409024 to 1884160 tid 2 2024-04-20T01:47:48.787 INFO:tasks.rados.rados.0.smithi165.stdout:160: writing smithi16579500-17 from 1884160 to 2179072 tid 3 2024-04-20T01:47:48.787 INFO:tasks.rados.rados.0.smithi165.stdout:161: rmattr oid 20 current snap is 8 2024-04-20T01:47:48.787 INFO:tasks.rados.rados.0.smithi165.stdout:162: read oid 44 snap 5 2024-04-20T01:47:48.787 INFO:tasks.rados.rados.0.smithi165.stdout:162: expect (ObjNum 57 snap 2 seq_num 57) 2024-04-20T01:47:48.787 INFO:tasks.rados.rados.0.smithi165.stdout:163: rmattr oid 36 current snap is 8 2024-04-20T01:47:48.787 INFO:tasks.rados.rados.0.smithi165.stdout:163: done (3 left) 2024-04-20T01:47:48.787 INFO:tasks.rados.rados.0.smithi165.stdout:164: copy_from oid 1 from oid 39 current snap is 8 2024-04-20T01:47:48.787 INFO:tasks.rados.rados.0.smithi165.stdout:165: snap_remove snap 6 2024-04-20T01:47:48.788 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:48.803+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:48.889 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:48.904+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:47:48.918 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:48.918 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:47:48.918 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 0 2024-04-20T01:47:48.918 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:48.928 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-20T01:47:48.990 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:49.005+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:49.056 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:49.057 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:47:49.057 INFO:teuthology.orchestra.run.smithi121.stdout: "complaint_time": 30, 2024-04-20T01:47:49.057 INFO:teuthology.orchestra.run.smithi121.stdout: "num_blocked_ops": 0 2024-04-20T01:47:49.057 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:49.067 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-20T01:47:49.091 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:49.106+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:49.192 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:49.207+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:49.197 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:49.197 INFO:teuthology.orchestra.run.smithi121.stdout: "size": 20, 2024-04-20T01:47:49.197 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 600, 2024-04-20T01:47:49.197 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-04-20T01:47:49.197 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.197 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-20T01:47:49.197 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:17.004323+0000", 2024-04-20T01:47:49.198 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 32.205892738000003, 2024-04-20T01:47:49.198 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.29931750600000001, 2024-04-20T01:47:49.198 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:49.198 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:49.198 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:49.198 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:47:49.198 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:47:49.198 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 1 2024-04-20T01:47:49.198 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.198 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:49.198 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.199 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:49.199 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004323+0000", 2024-04-20T01:47:49.199 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.199 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.199 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.199 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:49.199 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004323+0000", 2024-04-20T01:47:49.199 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.199 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.199 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.200 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:49.200 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004321+0000", 2024-04-20T01:47:49.200 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:49.200 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.200 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.200 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:49.200 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004337+0000", 2024-04-20T01:47:49.201 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5866999999999999e-05 2024-04-20T01:47:49.201 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.201 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.201 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:49.201 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004349+0000", 2024-04-20T01:47:49.201 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1802e-05 2024-04-20T01:47:49.201 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.201 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.202 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.202 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004360+0000", 2024-04-20T01:47:49.202 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1348999999999999e-05 2024-04-20T01:47:49.202 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.202 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.202 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.202 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004411+0000", 2024-04-20T01:47:49.202 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.0385e-05 2024-04-20T01:47:49.202 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.202 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.202 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for peered", 2024-04-20T01:47:49.203 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004419+0000", 2024-04-20T01:47:49.203 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4430000000000003e-06 2024-04-20T01:47:49.203 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.203 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.203 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.203 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.302330+0000", 2024-04-20T01:47:49.203 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.29791090199999998 2024-04-20T01:47:49.203 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.204 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.204 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:49.204 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.302488+0000", 2024-04-20T01:47:49.204 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00015830799999999999 2024-04-20T01:47:49.204 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.204 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.204 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:47:49.204 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.302650+0000", 2024-04-20T01:47:49.204 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00016196900000000001 2024-04-20T01:47:49.204 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.205 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.205 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:47:49.205 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.303204+0000", 2024-04-20T01:47:49.205 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00055385000000000005 2024-04-20T01:47:49.206 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.206 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.206 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:47:49.206 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.303574+0000", 2024-04-20T01:47:49.206 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00037030800000000002 2024-04-20T01:47:49.206 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.206 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.206 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:49.206 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.303614+0000", 2024-04-20T01:47:49.206 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.9161e-05 2024-04-20T01:47:49.206 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.207 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.207 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:49.207 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.303640+0000", 2024-04-20T01:47:49.207 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6752999999999999e-05 2024-04-20T01:47:49.207 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.207 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.207 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.207 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.208 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.208 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:16 3.3s0 3:cb777d78:::smithi16579500-6:head [append 0~98304,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:49.208 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.755788+0000", 2024-04-20T01:47:49.208 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.454427326999999, 2024-04-20T01:47:49.208 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010286419, 2024-04-20T01:47:49.208 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:49.208 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:49.208 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:49.208 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:47:49.208 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.209 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 16 2024-04-20T01:47:49.209 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.209 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:49.209 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.209 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:49.209 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.755788+0000", 2024-04-20T01:47:49.209 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.209 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.209 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.209 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:49.209 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.755788+0000", 2024-04-20T01:47:49.209 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.209 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.209 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.210 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:49.210 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.755787+0000", 2024-04-20T01:47:49.210 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999981 2024-04-20T01:47:49.210 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.210 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.210 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:49.210 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756003+0000", 2024-04-20T01:47:49.210 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00021687200000000001 2024-04-20T01:47:49.210 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.210 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.210 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:49.210 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756019+0000", 2024-04-20T01:47:49.210 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5962000000000001e-05 2024-04-20T01:47:49.210 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.211 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.211 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.211 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756028+0000", 2024-04-20T01:47:49.211 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.2160000000000006e-06 2024-04-20T01:47:49.211 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.211 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.211 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.211 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756081+0000", 2024-04-20T01:47:49.211 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3650000000000003e-05 2024-04-20T01:47:49.211 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.211 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.211 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:49.211 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756204+0000", 2024-04-20T01:47:49.211 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00012253200000000001 2024-04-20T01:47:49.212 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.212 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.212 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.212 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.757017+0000", 2024-04-20T01:47:49.212 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000812929 2024-04-20T01:47:49.212 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.212 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.212 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.212 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.761633+0000", 2024-04-20T01:47:49.212 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.004616411 2024-04-20T01:47:49.212 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.213 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.213 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:49.213 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.765990+0000", 2024-04-20T01:47:49.213 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.004357278 2024-04-20T01:47:49.213 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.213 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.213 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:49.213 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766075+0000", 2024-04-20T01:47:49.213 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4269000000000002e-05 2024-04-20T01:47:49.213 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.213 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.213 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.213 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.213 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.214 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:18 3.3s0 3:cb777d78:::smithi16579500-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:49.214 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.756072+0000", 2024-04-20T01:47:49.214 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.454143330999999, 2024-04-20T01:47:49.214 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013705281, 2024-04-20T01:47:49.214 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:49.214 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:47:49.214 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:49.214 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:47:49.214 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.214 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 18 2024-04-20T01:47:49.214 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.214 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:49.215 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.215 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:49.215 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756072+0000", 2024-04-20T01:47:49.215 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.215 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.215 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.215 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:49.215 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756072+0000", 2024-04-20T01:47:49.215 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.216 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.216 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.216 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:49.216 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756072+0000", 2024-04-20T01:47:49.216 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:49.216 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.216 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.216 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:49.217 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756074+0000", 2024-04-20T01:47:49.217 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.148e-06 2024-04-20T01:47:49.217 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.217 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.217 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:49.217 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756079+0000", 2024-04-20T01:47:49.217 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.4790000000000004e-06 2024-04-20T01:47:49.217 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.217 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.217 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.217 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756083+0000", 2024-04-20T01:47:49.217 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.4910000000000001e-06 2024-04-20T01:47:49.217 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.217 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.218 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.218 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.758130+0000", 2024-04-20T01:47:49.218 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00204725 2024-04-20T01:47:49.218 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.218 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.218 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:47:49.218 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.758200+0000", 2024-04-20T01:47:49.218 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0246000000000005e-05 2024-04-20T01:47:49.218 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.218 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.218 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.218 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766738+0000", 2024-04-20T01:47:49.218 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0085377939999999996 2024-04-20T01:47:49.218 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.219 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.219 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:49.219 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766828+0000", 2024-04-20T01:47:49.219 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.9677000000000002e-05 2024-04-20T01:47:49.219 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.219 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.219 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:49.219 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.769777+0000", 2024-04-20T01:47:49.219 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0029495369999999999 2024-04-20T01:47:49.219 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.219 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.219 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.219 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.219 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.219 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:48 3.2s0 3:4efae9ea:::smithi16579500-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:49.220 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.768007+0000", 2024-04-20T01:47:49.220 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.442208187, 2024-04-20T01:47:49.220 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010953058, 2024-04-20T01:47:49.220 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:49.220 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:47:49.220 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:49.220 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:47:49.220 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.220 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 48 2024-04-20T01:47:49.220 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.220 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:49.220 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.220 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:49.220 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768007+0000", 2024-04-20T01:47:49.221 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.221 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.221 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.221 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:49.221 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768007+0000", 2024-04-20T01:47:49.221 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.221 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.221 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.221 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:49.221 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768007+0000", 2024-04-20T01:47:49.221 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:49.221 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.221 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.221 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:49.221 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768011+0000", 2024-04-20T01:47:49.222 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2899999999999996e-06 2024-04-20T01:47:49.222 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.222 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.222 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:49.222 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768018+0000", 2024-04-20T01:47:49.222 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0060000000000003e-06 2024-04-20T01:47:49.222 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.222 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.222 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.222 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768025+0000", 2024-04-20T01:47:49.222 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.8390000000000001e-06 2024-04-20T01:47:49.222 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.222 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.222 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.223 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.772765+0000", 2024-04-20T01:47:49.223 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0047401409999999998 2024-04-20T01:47:49.223 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.223 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.223 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:47:49.223 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.772861+0000", 2024-04-20T01:47:49.223 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6513000000000002e-05 2024-04-20T01:47:49.223 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.223 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.223 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.223 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.776375+0000", 2024-04-20T01:47:49.223 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0035133679999999998 2024-04-20T01:47:49.223 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.223 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.224 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:49.224 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.776459+0000", 2024-04-20T01:47:49.224 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4368000000000003e-05 2024-04-20T01:47:49.224 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.224 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.224 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:49.224 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.778960+0000", 2024-04-20T01:47:49.224 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0025012910000000001 2024-04-20T01:47:49.224 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.224 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.224 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.224 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.224 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.224 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:67 3.3s0 3:c86e2dd3:::smithi16579500-23:head [append 0~376832,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:49.224 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.783364+0000", 2024-04-20T01:47:49.225 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.426851397, 2024-04-20T01:47:49.225 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013527099000000001, 2024-04-20T01:47:49.225 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:49.225 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:49.225 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:49.225 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:47:49.225 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.225 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 67 2024-04-20T01:47:49.225 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.225 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:49.226 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.226 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:49.226 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783364+0000", 2024-04-20T01:47:49.226 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.226 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.226 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.226 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:49.226 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783364+0000", 2024-04-20T01:47:49.226 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.227 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.227 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.227 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:49.227 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783355+0000", 2024-04-20T01:47:49.227 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999909 2024-04-20T01:47:49.227 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.227 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.227 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:49.227 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783704+0000", 2024-04-20T01:47:49.227 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00034900399999999998 2024-04-20T01:47:49.227 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.227 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.227 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:49.227 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783720+0000", 2024-04-20T01:47:49.228 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5464e-05 2024-04-20T01:47:49.228 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.228 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.228 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.228 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783727+0000", 2024-04-20T01:47:49.228 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.6199999999999999e-06 2024-04-20T01:47:49.228 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.228 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.228 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.228 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783773+0000", 2024-04-20T01:47:49.228 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5237000000000001e-05 2024-04-20T01:47:49.228 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.228 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.228 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:49.228 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783881+0000", 2024-04-20T01:47:49.229 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00010799200000000001 2024-04-20T01:47:49.229 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.229 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.229 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.229 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.784468+0000", 2024-04-20T01:47:49.229 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00058749400000000004 2024-04-20T01:47:49.229 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.229 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.229 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.229 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.785849+0000", 2024-04-20T01:47:49.229 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0013810420000000001 2024-04-20T01:47:49.229 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.229 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.229 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:49.230 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.796778+0000", 2024-04-20T01:47:49.230 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010928955000000001 2024-04-20T01:47:49.230 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.230 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.230 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:49.230 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.796891+0000", 2024-04-20T01:47:49.230 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00011315399999999999 2024-04-20T01:47:49.230 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.230 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.230 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.230 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.230 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.230 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:68 3.3s0 3:c86e2dd3:::smithi16579500-23:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:49.231 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.783739+0000", 2024-04-20T01:47:49.231 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.426476123, 2024-04-20T01:47:49.231 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013359042, 2024-04-20T01:47:49.231 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:49.231 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:49.231 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:49.231 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:47:49.231 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.231 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 68 2024-04-20T01:47:49.231 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.232 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:49.232 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.232 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:49.232 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783739+0000", 2024-04-20T01:47:49.232 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.232 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.232 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.232 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:49.232 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783739+0000", 2024-04-20T01:47:49.232 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.232 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.232 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.232 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:49.232 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783739+0000", 2024-04-20T01:47:49.233 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:49.233 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.233 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.233 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:49.233 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783744+0000", 2024-04-20T01:47:49.233 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.7690000000000004e-06 2024-04-20T01:47:49.233 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.233 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.233 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:49.233 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783750+0000", 2024-04-20T01:47:49.233 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.0719999999999996e-06 2024-04-20T01:47:49.233 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.233 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.233 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.234 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783754+0000", 2024-04-20T01:47:49.234 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.6450000000000004e-06 2024-04-20T01:47:49.234 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.234 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.234 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.234 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.784916+0000", 2024-04-20T01:47:49.234 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001161477 2024-04-20T01:47:49.234 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.234 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.234 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:49.234 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.785022+0000", 2024-04-20T01:47:49.234 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000106153 2024-04-20T01:47:49.234 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.234 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.235 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.235 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.785477+0000", 2024-04-20T01:47:49.235 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00045462699999999998 2024-04-20T01:47:49.235 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.235 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.235 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.235 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.787709+0000", 2024-04-20T01:47:49.235 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0022322309999999999 2024-04-20T01:47:49.235 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.235 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.235 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:49.236 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797021+0000", 2024-04-20T01:47:49.236 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0093120730000000006 2024-04-20T01:47:49.236 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.236 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.236 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:49.236 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797098+0000", 2024-04-20T01:47:49.236 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.7526000000000003e-05 2024-04-20T01:47:49.236 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.236 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.236 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.236 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.236 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.236 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:69 3.3s0 3:c86e2dd3:::smithi16579500-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:49.236 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.783758+0000", 2024-04-20T01:47:49.236 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.426457576000001, 2024-04-20T01:47:49.237 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.017903170999999999, 2024-04-20T01:47:49.237 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:49.237 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:47:49.237 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:49.237 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:47:49.237 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.237 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 69 2024-04-20T01:47:49.237 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.237 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:49.237 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.237 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:49.237 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783758+0000", 2024-04-20T01:47:49.237 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.237 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.237 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.238 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:49.238 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783758+0000", 2024-04-20T01:47:49.238 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.238 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.238 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.238 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:49.238 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783758+0000", 2024-04-20T01:47:49.238 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:49.238 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.238 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.238 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:49.238 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783759+0000", 2024-04-20T01:47:49.238 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.734e-06 2024-04-20T01:47:49.238 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.239 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.239 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:49.239 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783763+0000", 2024-04-20T01:47:49.239 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.5820000000000002e-06 2024-04-20T01:47:49.239 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.239 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.239 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.239 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783766+0000", 2024-04-20T01:47:49.239 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0469999999999998e-06 2024-04-20T01:47:49.239 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.239 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.239 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.239 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.785726+0000", 2024-04-20T01:47:49.239 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0019601079999999999 2024-04-20T01:47:49.239 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.239 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.240 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:47:49.240 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.785810+0000", 2024-04-20T01:47:49.240 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.3522000000000004e-05 2024-04-20T01:47:49.240 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.240 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.240 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.240 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797485+0000", 2024-04-20T01:47:49.240 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011675207 2024-04-20T01:47:49.240 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.240 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.240 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:49.240 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797573+0000", 2024-04-20T01:47:49.240 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8146000000000006e-05 2024-04-20T01:47:49.240 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.240 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.241 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:49.241 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.801661+0000", 2024-04-20T01:47:49.241 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0040880969999999997 2024-04-20T01:47:49.241 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.241 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.241 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.241 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.241 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.241 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:75 3.2s0 3:48443997:::smithi16579500-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:49.241 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.789051+0000", 2024-04-20T01:47:49.241 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.421164041000001, 2024-04-20T01:47:49.241 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013741625, 2024-04-20T01:47:49.241 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:49.241 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:47:49.241 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:49.242 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:47:49.242 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.242 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 75 2024-04-20T01:47:49.242 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.242 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:49.242 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.242 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:49.242 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789051+0000", 2024-04-20T01:47:49.242 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.242 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.242 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.242 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:49.242 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789051+0000", 2024-04-20T01:47:49.242 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.242 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.243 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.243 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:49.243 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789051+0000", 2024-04-20T01:47:49.243 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:49.243 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.243 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.243 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:49.243 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789054+0000", 2024-04-20T01:47:49.243 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7499999999999999e-06 2024-04-20T01:47:49.243 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.243 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.243 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:49.243 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789060+0000", 2024-04-20T01:47:49.243 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.567e-06 2024-04-20T01:47:49.243 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.244 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.244 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.244 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789066+0000", 2024-04-20T01:47:49.244 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.1499999999999998e-06 2024-04-20T01:47:49.244 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.244 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.244 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.244 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.790905+0000", 2024-04-20T01:47:49.244 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0018397000000000001 2024-04-20T01:47:49.244 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.244 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.244 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:47:49.244 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.790994+0000", 2024-04-20T01:47:49.244 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8679999999999998e-05 2024-04-20T01:47:49.244 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.245 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.245 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.245 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797798+0000", 2024-04-20T01:47:49.245 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0068040899999999996 2024-04-20T01:47:49.245 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.245 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.245 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:49.245 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797879+0000", 2024-04-20T01:47:49.245 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.1318999999999998e-05 2024-04-20T01:47:49.245 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.245 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.245 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:49.245 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.802793+0000", 2024-04-20T01:47:49.245 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.004913789 2024-04-20T01:47:49.245 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.246 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.246 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.246 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.246 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.246 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:82 3.3s0 3:cb8bd5ef:::smithi16579500-28:head [append 0~458752,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:49.246 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.797057+0000", 2024-04-20T01:47:49.246 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.413158053, 2024-04-20T01:47:49.246 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010776908999999999, 2024-04-20T01:47:49.246 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:49.246 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:49.246 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:49.246 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:47:49.246 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.246 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 82 2024-04-20T01:47:49.246 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.247 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:49.247 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.247 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:49.247 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797057+0000", 2024-04-20T01:47:49.247 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.247 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.247 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.247 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:49.247 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797057+0000", 2024-04-20T01:47:49.247 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.247 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.247 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.247 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:49.247 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797056+0000", 2024-04-20T01:47:49.248 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:49.248 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.248 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.248 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:49.248 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797396+0000", 2024-04-20T01:47:49.248 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00033930700000000001 2024-04-20T01:47:49.248 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.248 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.248 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:49.248 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797408+0000", 2024-04-20T01:47:49.248 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2362e-05 2024-04-20T01:47:49.248 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.248 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.248 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.248 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797417+0000", 2024-04-20T01:47:49.249 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.9209999999999998e-06 2024-04-20T01:47:49.249 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.249 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.249 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.249 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797863+0000", 2024-04-20T01:47:49.249 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00044603200000000001 2024-04-20T01:47:49.249 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.249 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.249 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:49.249 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.798011+0000", 2024-04-20T01:47:49.249 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00014856900000000001 2024-04-20T01:47:49.249 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.249 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.249 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.249 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.798944+0000", 2024-04-20T01:47:49.249 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000933054 2024-04-20T01:47:49.250 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.250 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.250 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.250 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.801368+0000", 2024-04-20T01:47:49.250 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0024231980000000001 2024-04-20T01:47:49.250 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.250 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.250 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:49.250 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.807752+0000", 2024-04-20T01:47:49.250 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0063840559999999999 2024-04-20T01:47:49.250 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.250 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.250 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:49.250 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.807834+0000", 2024-04-20T01:47:49.251 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.2693000000000003e-05 2024-04-20T01:47:49.251 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.251 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.251 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.251 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.251 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.251 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:83 3.3s0 3:cb8bd5ef:::smithi16579500-28:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:49.251 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.797426+0000", 2024-04-20T01:47:49.251 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.41278926, 2024-04-20T01:47:49.251 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010760049000000001, 2024-04-20T01:47:49.251 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:49.251 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:49.251 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:49.251 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:47:49.251 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.251 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 83 2024-04-20T01:47:49.252 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.252 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:49.252 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.252 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:49.252 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797426+0000", 2024-04-20T01:47:49.252 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.252 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.252 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.252 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:49.252 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797426+0000", 2024-04-20T01:47:49.252 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.252 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.252 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.252 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:49.252 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797426+0000", 2024-04-20T01:47:49.253 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:49.253 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.253 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.253 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:49.253 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797430+0000", 2024-04-20T01:47:49.253 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.3159999999999998e-06 2024-04-20T01:47:49.253 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.253 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.253 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:49.253 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797437+0000", 2024-04-20T01:47:49.253 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.8959999999999997e-06 2024-04-20T01:47:49.253 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.253 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.253 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.253 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797442+0000", 2024-04-20T01:47:49.254 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6520000000000003e-06 2024-04-20T01:47:49.254 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.254 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.254 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.254 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.799583+0000", 2024-04-20T01:47:49.254 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0021408070000000002 2024-04-20T01:47:49.254 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.254 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.254 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:49.254 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.799685+0000", 2024-04-20T01:47:49.254 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00010137900000000001 2024-04-20T01:47:49.254 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.254 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.254 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.254 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.800122+0000", 2024-04-20T01:47:49.255 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00043781400000000002 2024-04-20T01:47:49.255 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.255 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.255 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.255 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.805542+0000", 2024-04-20T01:47:49.255 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0054195340000000002 2024-04-20T01:47:49.255 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.255 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.255 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:49.255 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.807962+0000", 2024-04-20T01:47:49.255 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0024197139999999999 2024-04-20T01:47:49.255 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.255 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.255 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:49.255 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.808186+0000", 2024-04-20T01:47:49.256 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00022459699999999999 2024-04-20T01:47:49.256 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.256 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.256 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.256 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.256 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.256 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:84 3.3s0 3:cb8bd5ef:::smithi16579500-28:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:49.256 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.797448+0000", 2024-04-20T01:47:49.256 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.412767579, 2024-04-20T01:47:49.256 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.01109836, 2024-04-20T01:47:49.256 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:49.256 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:49.256 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:49.256 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:47:49.257 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.257 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 84 2024-04-20T01:47:49.257 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.257 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:49.257 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.257 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:49.257 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797448+0000", 2024-04-20T01:47:49.257 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.257 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.257 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.257 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:49.257 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797448+0000", 2024-04-20T01:47:49.257 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.257 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.257 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.258 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:49.258 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797448+0000", 2024-04-20T01:47:49.258 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:49.258 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.258 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.258 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:49.258 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797450+0000", 2024-04-20T01:47:49.258 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7999999999999999e-06 2024-04-20T01:47:49.258 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.258 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.258 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:49.258 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797458+0000", 2024-04-20T01:47:49.258 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.2590000000000003e-06 2024-04-20T01:47:49.258 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.258 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.259 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.259 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797464+0000", 2024-04-20T01:47:49.259 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.2979999999999997e-06 2024-04-20T01:47:49.259 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.259 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.259 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.259 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.801705+0000", 2024-04-20T01:47:49.259 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0042409819999999999 2024-04-20T01:47:49.259 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.259 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.259 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:47:49.259 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.801761+0000", 2024-04-20T01:47:49.259 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.5696999999999999e-05 2024-04-20T01:47:49.259 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.259 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.260 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.260 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.808533+0000", 2024-04-20T01:47:49.260 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0067724639999999997 2024-04-20T01:47:49.260 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.260 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.260 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:49.260 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.808546+0000", 2024-04-20T01:47:49.260 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3276e-05 2024-04-20T01:47:49.260 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.260 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.260 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.260 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.260 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.260 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:232 3.3s0 3:cb777d78:::smithi16579500-6:head [copy-from ver 6] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:47:49.260 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.787663+0000", 2024-04-20T01:47:49.261 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 10.422552975, 2024-04-20T01:47:49.261 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.019650832, 2024-04-20T01:47:49.261 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:49.261 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:49.261 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:49.261 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:47:49.261 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.261 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 232 2024-04-20T01:47:49.261 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.261 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:49.261 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.261 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:49.261 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787663+0000", 2024-04-20T01:47:49.261 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.262 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.262 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.262 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:49.262 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787663+0000", 2024-04-20T01:47:49.262 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.262 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.262 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.262 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:49.262 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787661+0000", 2024-04-20T01:47:49.262 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:49.262 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.262 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.262 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:49.262 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787673+0000", 2024-04-20T01:47:49.262 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.181e-05 2024-04-20T01:47:49.263 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.263 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.263 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:49.263 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787689+0000", 2024-04-20T01:47:49.263 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6731999999999998e-05 2024-04-20T01:47:49.263 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.263 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.263 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.263 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787706+0000", 2024-04-20T01:47:49.263 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6739e-05 2024-04-20T01:47:49.263 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.263 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.263 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.263 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787795+0000", 2024-04-20T01:47:49.264 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.9226000000000003e-05 2024-04-20T01:47:49.264 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.264 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.264 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:49.264 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787904+0000", 2024-04-20T01:47:49.264 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000108452 2024-04-20T01:47:49.264 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.264 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.264 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.264 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.797712+0000", 2024-04-20T01:47:49.264 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0098079599999999992 2024-04-20T01:47:49.264 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.264 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.264 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.264 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.799543+0000", 2024-04-20T01:47:49.265 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001831596 2024-04-20T01:47:49.265 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.265 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.265 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:49.265 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807224+0000", 2024-04-20T01:47:49.265 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0076805249999999997 2024-04-20T01:47:49.265 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.265 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.265 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:49.265 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807313+0000", 2024-04-20T01:47:49.265 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.9431999999999993e-05 2024-04-20T01:47:49.265 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.265 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.265 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.265 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.266 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.266 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:233 3.3s0 3:cb777d78:::smithi16579500-6:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:47:49.266 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.787721+0000", 2024-04-20T01:47:49.266 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 10.422494363, 2024-04-20T01:47:49.266 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.01143623, 2024-04-20T01:47:49.266 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:49.266 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:47:49.266 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:49.266 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:47:49.266 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.266 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 233 2024-04-20T01:47:49.266 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.266 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:49.266 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.266 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:49.267 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787721+0000", 2024-04-20T01:47:49.267 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.267 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.267 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.267 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:49.267 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787721+0000", 2024-04-20T01:47:49.267 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.267 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.267 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.267 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:49.267 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787721+0000", 2024-04-20T01:47:49.267 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:49.267 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.267 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.267 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:49.268 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787743+0000", 2024-04-20T01:47:49.268 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2540000000000001e-05 2024-04-20T01:47:49.268 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.268 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.268 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:49.268 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787753+0000", 2024-04-20T01:47:49.268 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0101e-05 2024-04-20T01:47:49.268 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.268 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.268 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.268 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787760+0000", 2024-04-20T01:47:49.268 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.3729999999999998e-06 2024-04-20T01:47:49.268 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.268 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.268 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.268 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.788051+0000", 2024-04-20T01:47:49.269 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00029180999999999999 2024-04-20T01:47:49.269 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.269 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.269 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for blocked object", 2024-04-20T01:47:49.269 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.788092+0000", 2024-04-20T01:47:49.269 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.0309000000000003e-05 2024-04-20T01:47:49.269 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.269 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.269 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.269 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.798980+0000", 2024-04-20T01:47:49.269 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010888070999999999 2024-04-20T01:47:49.269 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.269 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.269 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:49.270 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.799065+0000", 2024-04-20T01:47:49.270 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.5241999999999995e-05 2024-04-20T01:47:49.270 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.270 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.270 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:49.270 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.799157+0000", 2024-04-20T01:47:49.270 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2388999999999999e-05 2024-04-20T01:47:49.270 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.270 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.270 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.270 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.270 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.270 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:295 3.2s0 3:44d9544b:::smithi16579500-38:head [read 0~1474560,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:47:49.270 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.807427+0000", 2024-04-20T01:47:49.270 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.4027886489999997, 2024-04-20T01:47:49.271 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010585898999999999, 2024-04-20T01:47:49.271 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:49.271 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:47:49.271 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:49.271 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:47:49.271 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.271 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 295 2024-04-20T01:47:49.271 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.271 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:49.271 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.271 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:49.271 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807427+0000", 2024-04-20T01:47:49.271 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.271 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.271 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.272 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:49.272 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807427+0000", 2024-04-20T01:47:49.272 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.272 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.272 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.272 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:49.272 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807426+0000", 2024-04-20T01:47:49.272 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:49.272 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.272 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.272 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:49.272 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807430+0000", 2024-04-20T01:47:49.272 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.6490000000000001e-06 2024-04-20T01:47:49.272 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.273 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.273 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:49.273 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807435+0000", 2024-04-20T01:47:49.273 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.4220000000000002e-06 2024-04-20T01:47:49.273 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.273 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.273 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.273 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807438+0000", 2024-04-20T01:47:49.273 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.269e-06 2024-04-20T01:47:49.273 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.273 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.273 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.273 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807579+0000", 2024-04-20T01:47:49.273 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000141691 2024-04-20T01:47:49.273 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.274 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.274 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:47:49.274 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807589+0000", 2024-04-20T01:47:49.274 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3190000000000001e-06 2024-04-20T01:47:49.274 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.274 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.274 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.274 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.810618+0000", 2024-04-20T01:47:49.274 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0030287859999999999 2024-04-20T01:47:49.274 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.274 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.274 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:49.274 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.810717+0000", 2024-04-20T01:47:49.274 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.9780999999999999e-05 2024-04-20T01:47:49.274 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.275 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.275 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:49.275 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.818013+0000", 2024-04-20T01:47:49.275 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0072954220000000002 2024-04-20T01:47:49.275 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.275 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.275 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.275 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.275 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.275 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:296 3.2s0 3:44d9544b:::smithi16579500-38:head [read 0~1474560,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:47:49.275 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.807470+0000", 2024-04-20T01:47:49.275 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.4027453899999998, 2024-04-20T01:47:49.275 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011565101, 2024-04-20T01:47:49.275 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:49.275 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:47:49.275 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:49.276 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:47:49.276 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.276 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 296 2024-04-20T01:47:49.276 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.276 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:49.276 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.276 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:49.276 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807470+0000", 2024-04-20T01:47:49.276 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.276 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.276 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.276 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:49.276 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807470+0000", 2024-04-20T01:47:49.276 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.276 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.277 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.277 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:49.277 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807470+0000", 2024-04-20T01:47:49.277 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:49.277 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.277 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.277 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:49.277 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807473+0000", 2024-04-20T01:47:49.277 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9809999999999999e-06 2024-04-20T01:47:49.277 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.277 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.277 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:49.277 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807479+0000", 2024-04-20T01:47:49.277 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.1970000000000001e-06 2024-04-20T01:47:49.277 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.278 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.278 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.278 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807483+0000", 2024-04-20T01:47:49.278 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.053e-06 2024-04-20T01:47:49.278 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.278 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.278 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.278 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807633+0000", 2024-04-20T01:47:49.278 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00015004999999999999 2024-04-20T01:47:49.278 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.278 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.278 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:47:49.278 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807643+0000", 2024-04-20T01:47:49.278 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7049999999999994e-06 2024-04-20T01:47:49.278 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.279 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.279 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.279 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.811287+0000", 2024-04-20T01:47:49.279 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0036436699999999999 2024-04-20T01:47:49.279 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.279 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.279 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:49.279 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.811372+0000", 2024-04-20T01:47:49.279 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.5251000000000001e-05 2024-04-20T01:47:49.279 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.279 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.279 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:49.279 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819035+0000", 2024-04-20T01:47:49.279 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.007663464 2024-04-20T01:47:49.279 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.280 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.280 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.280 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.280 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.280 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:298 3.2s0 3:4302a124:::smithi16579500-33:head [append 0~0] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:47:49.280 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.808446+0000", 2024-04-20T01:47:49.280 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.4017693099999997, 2024-04-20T01:47:49.280 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.01090315, 2024-04-20T01:47:49.280 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:49.280 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:49.280 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:49.280 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:47:49.280 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.280 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 298 2024-04-20T01:47:49.280 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.281 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:49.281 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.281 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:49.281 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808446+0000", 2024-04-20T01:47:49.281 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.281 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.281 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.281 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:49.281 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808446+0000", 2024-04-20T01:47:49.281 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.281 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.281 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.281 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:49.281 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808445+0000", 2024-04-20T01:47:49.281 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:49.282 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.282 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.282 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:49.282 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808451+0000", 2024-04-20T01:47:49.282 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.9710000000000001e-06 2024-04-20T01:47:49.282 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.282 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.282 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:49.282 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808466+0000", 2024-04-20T01:47:49.282 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4945e-05 2024-04-20T01:47:49.282 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.282 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.282 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.282 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808473+0000", 2024-04-20T01:47:49.282 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.8789999999999997e-06 2024-04-20T01:47:49.283 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.283 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.283 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.283 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808579+0000", 2024-04-20T01:47:49.283 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00010635 2024-04-20T01:47:49.283 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.283 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.283 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:47:49.283 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808587+0000", 2024-04-20T01:47:49.283 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.4000000000000003e-06 2024-04-20T01:47:49.283 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.283 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.283 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.283 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.813433+0000", 2024-04-20T01:47:49.283 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0048458549999999996 2024-04-20T01:47:49.284 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.284 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.284 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:49.284 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.813595+0000", 2024-04-20T01:47:49.284 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00016224300000000001 2024-04-20T01:47:49.284 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.284 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.284 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.284 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.814543+0000", 2024-04-20T01:47:49.284 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00094800199999999998 2024-04-20T01:47:49.284 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.284 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.284 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.284 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.818083+0000", 2024-04-20T01:47:49.284 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.003539824 2024-04-20T01:47:49.285 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.285 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.285 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:49.285 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819295+0000", 2024-04-20T01:47:49.285 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001212634 2024-04-20T01:47:49.285 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.285 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.285 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:49.285 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819349+0000", 2024-04-20T01:47:49.285 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3949999999999997e-05 2024-04-20T01:47:49.285 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.285 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.285 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.285 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.285 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.286 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:299 3.2s0 3:4302a124:::smithi16579500-33:head [rollback a] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:47:49.286 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.808485+0000", 2024-04-20T01:47:49.286 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.4017306190000003, 2024-04-20T01:47:49.286 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011156045, 2024-04-20T01:47:49.286 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:49.286 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:49.286 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:49.286 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:47:49.286 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.286 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 299 2024-04-20T01:47:49.286 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.286 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:49.286 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.286 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:49.286 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808485+0000", 2024-04-20T01:47:49.287 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.287 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.287 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.287 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:49.287 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808485+0000", 2024-04-20T01:47:49.287 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.287 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.287 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.287 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:49.287 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808484+0000", 2024-04-20T01:47:49.287 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:49.287 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.288 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.288 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:49.288 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808488+0000", 2024-04-20T01:47:49.288 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.3529999999999999e-06 2024-04-20T01:47:49.288 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.288 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.288 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:49.288 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808495+0000", 2024-04-20T01:47:49.288 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.4390000000000003e-06 2024-04-20T01:47:49.288 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.288 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.288 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.288 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808501+0000", 2024-04-20T01:47:49.288 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.4480000000000002e-06 2024-04-20T01:47:49.289 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.289 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.289 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.289 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808634+0000", 2024-04-20T01:47:49.289 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000133461 2024-04-20T01:47:49.289 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.289 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.289 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:47:49.289 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808650+0000", 2024-04-20T01:47:49.289 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6271999999999999e-05 2024-04-20T01:47:49.289 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.289 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.289 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.289 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.815215+0000", 2024-04-20T01:47:49.289 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0065644509999999998 2024-04-20T01:47:49.290 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.290 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.290 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:49.290 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.815328+0000", 2024-04-20T01:47:49.290 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000112754 2024-04-20T01:47:49.290 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.290 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.290 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.290 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.815818+0000", 2024-04-20T01:47:49.290 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00049083000000000004 2024-04-20T01:47:49.290 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.291 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.291 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.291 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819209+0000", 2024-04-20T01:47:49.291 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0033910139999999999 2024-04-20T01:47:49.291 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.291 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.291 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:49.291 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819596+0000", 2024-04-20T01:47:49.291 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00038603299999999998 2024-04-20T01:47:49.291 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.291 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.291 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:49.291 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819641+0000", 2024-04-20T01:47:49.291 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5442999999999998e-05 2024-04-20T01:47:49.291 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.292 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.292 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.292 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.292 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.292 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:300 3.2s0 3:4302a124:::smithi16579500-33:head [append 0~0] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:47:49.292 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.808516+0000", 2024-04-20T01:47:49.292 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.4016994819999997, 2024-04-20T01:47:49.292 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.016176293000000001, 2024-04-20T01:47:49.292 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:49.292 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:49.292 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:49.292 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:47:49.292 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.292 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 300 2024-04-20T01:47:49.292 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.293 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:49.293 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.293 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:49.293 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808516+0000", 2024-04-20T01:47:49.293 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.293 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.293 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.293 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:49.293 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808516+0000", 2024-04-20T01:47:49.293 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.293 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.293 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.293 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:49.293 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808515+0000", 2024-04-20T01:47:49.293 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:49.294 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.294 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.294 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:49.294 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808521+0000", 2024-04-20T01:47:49.294 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.8919999999999997e-06 2024-04-20T01:47:49.294 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.294 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.294 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:49.294 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808528+0000", 2024-04-20T01:47:49.294 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0190000000000001e-06 2024-04-20T01:47:49.294 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.294 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.294 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.294 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808534+0000", 2024-04-20T01:47:49.294 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.891e-06 2024-04-20T01:47:49.295 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.295 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.295 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.295 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.809751+0000", 2024-04-20T01:47:49.295 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0012163580000000001 2024-04-20T01:47:49.295 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.295 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.295 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:47:49.295 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.809758+0000", 2024-04-20T01:47:49.295 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.6580000000000002e-06 2024-04-20T01:47:49.295 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.295 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.295 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.295 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819973+0000", 2024-04-20T01:47:49.295 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010214983 2024-04-20T01:47:49.296 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.296 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.296 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:49.296 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.820076+0000", 2024-04-20T01:47:49.296 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00010318 2024-04-20T01:47:49.296 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.296 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.296 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.296 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.820519+0000", 2024-04-20T01:47:49.296 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00044209799999999998 2024-04-20T01:47:49.296 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.296 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.296 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.296 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.821591+0000", 2024-04-20T01:47:49.297 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001072677 2024-04-20T01:47:49.297 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.297 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.297 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:49.297 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.824619+0000", 2024-04-20T01:47:49.297 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0030279170000000002 2024-04-20T01:47:49.297 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.297 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.297 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:49.297 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.824692+0000", 2024-04-20T01:47:49.297 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.3220999999999996e-05 2024-04-20T01:47:49.297 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.297 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.297 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.297 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.298 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.298 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:302 3.2s0 3:4cb85a84:::smithi16579500-29:head [copy-from ver 10] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:47:49.298 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.823551+0000", 2024-04-20T01:47:49.298 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.3866644260000003, 2024-04-20T01:47:49.298 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.014229169999999999, 2024-04-20T01:47:49.298 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:49.298 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:49.298 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:49.298 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:47:49.298 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.298 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 302 2024-04-20T01:47:49.298 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.298 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:49.298 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.299 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:49.299 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823551+0000", 2024-04-20T01:47:49.299 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.299 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.299 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.299 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:49.299 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823551+0000", 2024-04-20T01:47:49.299 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.299 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.299 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.299 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:49.299 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823550+0000", 2024-04-20T01:47:49.299 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:49.299 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.299 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.300 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:49.300 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823556+0000", 2024-04-20T01:47:49.300 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.455e-06 2024-04-20T01:47:49.300 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.300 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.300 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:49.300 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823568+0000", 2024-04-20T01:47:49.300 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1921e-05 2024-04-20T01:47:49.300 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.300 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.300 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.300 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823575+0000", 2024-04-20T01:47:49.300 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.1169999999999998e-06 2024-04-20T01:47:49.300 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.300 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.301 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.301 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823617+0000", 2024-04-20T01:47:49.301 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2012999999999999e-05 2024-04-20T01:47:49.301 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.301 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.301 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:49.301 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823741+0000", 2024-04-20T01:47:49.301 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000123454 2024-04-20T01:47:49.301 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.301 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.301 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.301 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.829454+0000", 2024-04-20T01:47:49.301 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0057129570000000003 2024-04-20T01:47:49.301 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.301 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.302 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.302 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.831037+0000", 2024-04-20T01:47:49.302 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0015832559999999999 2024-04-20T01:47:49.302 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.302 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.302 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:47:49.302 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.837719+0000", 2024-04-20T01:47:49.302 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.006681953 2024-04-20T01:47:49.302 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.302 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.302 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:49.302 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.837780+0000", 2024-04-20T01:47:49.302 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.1561000000000002e-05 2024-04-20T01:47:49.302 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.302 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.303 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.303 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.303 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.303 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:391 3.2s0 3:4303e4bf:::smithi16579500-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-04-20T01:47:49.303 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.746204+0000", 2024-04-20T01:47:49.303 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.46401193800000001, 2024-04-20T01:47:49.303 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010330114, 2024-04-20T01:47:49.303 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:49.303 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:47:49.303 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:47:49.303 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:47:49.303 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.303 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 391 2024-04-20T01:47:49.303 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.303 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:49.305 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.305 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:49.306 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.746204+0000", 2024-04-20T01:47:49.306 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.306 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.306 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.306 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:49.306 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.746204+0000", 2024-04-20T01:47:49.306 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:49.306 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.306 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.306 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:49.306 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.746203+0000", 2024-04-20T01:47:49.306 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:49.306 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.306 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.307 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:49.307 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.746206+0000", 2024-04-20T01:47:49.307 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4200000000000001e-06 2024-04-20T01:47:49.307 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.307 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.307 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:49.307 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.746211+0000", 2024-04-20T01:47:49.307 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.9390000000000002e-06 2024-04-20T01:47:49.307 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.307 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.307 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.307 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.746215+0000", 2024-04-20T01:47:49.307 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.8029999999999998e-06 2024-04-20T01:47:49.307 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.307 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.307 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.308 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.748247+0000", 2024-04-20T01:47:49.308 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0020318570000000002 2024-04-20T01:47:49.308 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.308 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.308 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:47:49.308 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.748300+0000", 2024-04-20T01:47:49.308 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3044999999999998e-05 2024-04-20T01:47:49.308 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.308 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.308 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:49.308 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.753501+0000", 2024-04-20T01:47:49.308 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.005201153 2024-04-20T01:47:49.308 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.308 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.308 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:47:49.309 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.753589+0000", 2024-04-20T01:47:49.309 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.7305000000000005e-05 2024-04-20T01:47:49.309 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:49.309 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:49.309 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:49.309 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.756534+0000", 2024-04-20T01:47:49.309 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.002945044 2024-04-20T01:47:49.309 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.309 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.309 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.309 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:49.309 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:49.309 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:49.311 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-20T01:47:49.311 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:49.325+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:49.412 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:49.427+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:49.455 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:49.455 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:47:49.455 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:47:49.455 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:49.465 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-20T01:47:49.513 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:49.526+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:47:49.613 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:49.628+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:47:49.637 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:49.638 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:47:49.638 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 30, 2024-04-20T01:47:49.638 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-20T01:47:49.638 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:49.647 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-20T01:47:49.695 INFO:tasks.rados.rados.0.smithi165.stdout:160: finishing write tid 1 to smithi16579500-17 2024-04-20T01:47:49.695 INFO:tasks.rados.rados.0.smithi165.stdout:160: finishing write tid 2 to smithi16579500-17 2024-04-20T01:47:49.695 INFO:tasks.rados.rados.0.smithi165.stdout:160: finishing write tid 3 to smithi16579500-17 2024-04-20T01:47:49.695 INFO:tasks.rados.rados.0.smithi165.stdout:160: finishing write tid 4 to smithi16579500-17 2024-04-20T01:47:49.695 INFO:tasks.rados.rados.0.smithi165.stdout:164: finishing copy_from racing read to smithi16579500-1 2024-04-20T01:47:49.696 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 20 v 49 (ObjNum 129 snap 0 seq_num 2584250992) dirty exists 2024-04-20T01:47:49.696 INFO:tasks.rados.rados.0.smithi165.stdout:160: finishing write tid 5 to smithi16579500-17 2024-04-20T01:47:49.696 INFO:tasks.rados.rados.0.smithi165.stdout:160: oid 17 updating version 0 to 40 2024-04-20T01:47:49.696 INFO:tasks.rados.rados.0.smithi165.stdout:160: oid 17 version 40 is already newer than 37 2024-04-20T01:47:49.696 INFO:tasks.rados.rados.0.smithi165.stdout:160: oid 17 version 40 is already newer than 39 2024-04-20T01:47:49.696 INFO:tasks.rados.rados.0.smithi165.stdout:160: oid 17 version 40 is already newer than 38 2024-04-20T01:47:49.696 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 17 v 40 (ObjNum 82 snap 8 seq_num 82) dirty exists 2024-04-20T01:47:49.696 INFO:tasks.rados.rados.0.smithi165.stdout:160: left oid 17 (ObjNum 82 snap 8 seq_num 82) 2024-04-20T01:47:49.696 INFO:tasks.rados.rados.0.smithi165.stdout:164: finishing copy_from to smithi16579500-1 2024-04-20T01:47:49.696 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 1 v 32 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-04-20T01:47:49.696 INFO:tasks.rados.rados.0.smithi165.stdout:160: done (4 left) 2024-04-20T01:47:49.696 INFO:tasks.rados.rados.0.smithi165.stdout:161: done (3 left) 2024-04-20T01:47:49.696 INFO:tasks.rados.rados.0.smithi165.stdout:164: done (2 left) 2024-04-20T01:47:49.696 INFO:tasks.rados.rados.0.smithi165.stdout:165: done (1 left) 2024-04-20T01:47:49.696 INFO:tasks.rados.rados.0.smithi165.stdout:166: rollback oid 10 current snap is 8 2024-04-20T01:47:49.697 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 10 to 5 2024-04-20T01:47:49.697 INFO:tasks.rados.rados.0.smithi165.stdout:162: expect (ObjNum 57 snap 2 seq_num 57) 2024-04-20T01:47:49.702 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.15 to 1 (10000) 2024-04-20T01:47:49.713 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-20T01:47:49.715 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:47:49.729+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:47:49.721 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-20T01:47:49.721 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T01:47:49.756 INFO:tasks.rados.rados.0.smithi165.stdout:166: finishing rollback tid 0 to smithi16579500-10 2024-04-20T01:47:49.756 INFO:tasks.rados.rados.0.smithi165.stdout:166: finishing rollback tid 1 to smithi16579500-10 2024-04-20T01:47:49.756 INFO:tasks.rados.rados.0.smithi165.stdout:166: finishing rollback tid 2 to smithi16579500-10 2024-04-20T01:47:49.756 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 10 v 73 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-04-20T01:47:49.756 INFO:tasks.rados.rados.0.smithi165.stdout:162: done (1 left) 2024-04-20T01:47:49.756 INFO:tasks.rados.rados.0.smithi165.stdout:166: done (0 left) 2024-04-20T01:47:49.756 INFO:tasks.rados.rados.0.smithi165.stdout:167: setattr oid 47 current snap is 8 2024-04-20T01:47:49.757 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 38 current snap is 8 2024-04-20T01:47:49.757 INFO:tasks.rados.rados.0.smithi165.stdout:168: seq_num 83 ranges {1474560=638976,2113536=622592,2736128=147456} 2024-04-20T01:47:49.768 INFO:tasks.rados.rados.0.smithi165.stdout:168: writing smithi16579500-38 from 1474560 to 2113536 tid 1 2024-04-20T01:47:49.772 INFO:tasks.rados.rados.0.smithi165.stdout:168: writing smithi16579500-38 from 2113536 to 2736128 tid 2 2024-04-20T01:47:49.774 INFO:tasks.rados.rados.0.smithi165.stdout:168: writing smithi16579500-38 from 2736128 to 2883584 tid 3 2024-04-20T01:47:49.774 INFO:tasks.rados.rados.0.smithi165.stdout:169: read oid 16 snap -1 2024-04-20T01:47:49.775 INFO:tasks.rados.rados.0.smithi165.stdout:169: expect (ObjNum 15 snap 0 seq_num 15) 2024-04-20T01:47:49.775 INFO:tasks.rados.rados.0.smithi165.stdout:170: read oid 22 snap -1 2024-04-20T01:47:49.775 INFO:tasks.rados.rados.0.smithi165.stdout:170: expect (ObjNum 21 snap 0 seq_num 21) 2024-04-20T01:47:49.775 INFO:tasks.rados.rados.0.smithi165.stdout:171: copy_from oid 31 from oid 7 current snap is 8 2024-04-20T01:47:49.775 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 47 v 36 (ObjNum 80 snap 8 seq_num 80) dirty exists 2024-04-20T01:47:49.775 INFO:tasks.rados.rados.0.smithi165.stdout:167: done (4 left) 2024-04-20T01:47:49.775 INFO:tasks.rados.rados.0.smithi165.stdout:172: delete oid 3 current snap is 8 2024-04-20T01:47:49.775 INFO:tasks.rados.rados.0.smithi165.stdout:171: finishing copy_from to smithi16579500-31 2024-04-20T01:47:49.775 INFO:tasks.rados.rados.0.smithi165.stdout:171: got expected ENOENT (src dne) 2024-04-20T01:47:49.775 INFO:tasks.rados.rados.0.smithi165.stdout:171: finishing copy_from racing read to smithi16579500-31 2024-04-20T01:47:49.777 INFO:tasks.rados.rados.0.smithi165.stdout:171: done (4 left) 2024-04-20T01:47:49.777 INFO:tasks.rados.rados.0.smithi165.stdout:172: done (3 left) 2024-04-20T01:47:49.777 INFO:tasks.rados.rados.0.smithi165.stdout:173: snap_create 2024-04-20T01:47:49.778 INFO:tasks.rados.rados.0.smithi165.stdout:170: expect (ObjNum 21 snap 0 seq_num 21) 2024-04-20T01:47:49.788 INFO:tasks.rados.rados.0.smithi165.stdout:168: finishing write tid 1 to smithi16579500-38 2024-04-20T01:47:49.789 INFO:tasks.rados.rados.0.smithi165.stdout:169: expect (ObjNum 15 snap 0 seq_num 15) 2024-04-20T01:47:49.790 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:49.790 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-20T01:47:49.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-20T01:47:49.790 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-20T01:47:49.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.790 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:229 3.fs0 3:fc9748e2:::smithi16579500-32:head [read 0~524288,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:47:49.790 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.787590+0000", 2024-04-20T01:47:49.790 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 11.015198960999999, 2024-04-20T01:47:49.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047169430000000004, 2024-04-20T01:47:49.790 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:49.791 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:47:49.791 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:47:49.791 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:47:49.791 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.791 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 229 2024-04-20T01:47:49.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.791 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:49.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:49.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787590+0000", 2024-04-20T01:47:49.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:49.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787590+0000", 2024-04-20T01:47:49.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:49.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787589+0000", 2024-04-20T01:47:49.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999981 2024-04-20T01:47:49.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:49.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787599+0000", 2024-04-20T01:47:49.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0658e-05 2024-04-20T01:47:49.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:49.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787618+0000", 2024-04-20T01:47:49.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9066999999999999e-05 2024-04-20T01:47:49.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787629+0000", 2024-04-20T01:47:49.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0604999999999999e-05 2024-04-20T01:47:49.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787703+0000", 2024-04-20T01:47:49.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4544999999999999e-05 2024-04-20T01:47:49.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op must wait for map", 2024-04-20T01:47:49.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787713+0000", 2024-04-20T01:47:49.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9930000000000007e-06 2024-04-20T01:47:49.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.789878+0000", 2024-04-20T01:47:49.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021646809999999999 2024-04-20T01:47:49.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:47:49.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.789941+0000", 2024-04-20T01:47:49.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2558000000000007e-05 2024-04-20T01:47:49.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:49.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.792307+0000", 2024-04-20T01:47:49.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023666920000000001 2024-04-20T01:47:49.795 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.795 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.795 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.795 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:230 3.fs0 3:fc9748e2:::smithi16579500-32:head [read 0~524288,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:47:49.795 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.787639+0000", 2024-04-20T01:47:49.795 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 11.015150508, 2024-04-20T01:47:49.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052033440000000004, 2024-04-20T01:47:49.795 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:49.795 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:47:49.796 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:47:49.796 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:47:49.796 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.796 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 230 2024-04-20T01:47:49.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.796 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:49.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:49.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787639+0000", 2024-04-20T01:47:49.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:49.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787639+0000", 2024-04-20T01:47:49.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:49.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787638+0000", 2024-04-20T01:47:49.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:49.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:49.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787642+0000", 2024-04-20T01:47:49.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4539999999999999e-06 2024-04-20T01:47:49.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:49.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787646+0000", 2024-04-20T01:47:49.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1269999999999996e-06 2024-04-20T01:47:49.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787650+0000", 2024-04-20T01:47:49.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.467e-06 2024-04-20T01:47:49.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787796+0000", 2024-04-20T01:47:49.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014651000000000001 2024-04-20T01:47:49.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:47:49.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787801+0000", 2024-04-20T01:47:49.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5059999999999997e-06 2024-04-20T01:47:49.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.790292+0000", 2024-04-20T01:47:49.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024917329999999999 2024-04-20T01:47:49.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:47:49.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.790354+0000", 2024-04-20T01:47:49.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1407000000000003e-05 2024-04-20T01:47:49.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:49.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.792842+0000", 2024-04-20T01:47:49.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024885139999999998 2024-04-20T01:47:49.800 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.800 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.800 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.800 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:231 3.fs0 3:fc9748e2:::smithi16579500-32:head [read 0~524288,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:47:49.800 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.787695+0000", 2024-04-20T01:47:49.800 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 11.015094275999999, 2024-04-20T01:47:49.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0078083780000000004, 2024-04-20T01:47:49.800 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:49.800 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:47:49.800 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:47:49.800 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:47:49.801 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:49.801 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 231 2024-04-20T01:47:49.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.801 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:49.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:49.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787695+0000", 2024-04-20T01:47:49.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:49.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787695+0000", 2024-04-20T01:47:49.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:49.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787694+0000", 2024-04-20T01:47:49.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:49.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:49.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787699+0000", 2024-04-20T01:47:49.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8439999999999997e-06 2024-04-20T01:47:49.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:49.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787708+0000", 2024-04-20T01:47:49.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2679999999999994e-06 2024-04-20T01:47:49.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.803 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.803 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787715+0000", 2024-04-20T01:47:49.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8290000000000008e-06 2024-04-20T01:47:49.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.803 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.803 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787839+0000", 2024-04-20T01:47:49.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012379700000000001 2024-04-20T01:47:49.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.803 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:47:49.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787845+0000", 2024-04-20T01:47:49.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1249999999999998e-06 2024-04-20T01:47:49.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.804 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.793431+0000", 2024-04-20T01:47:49.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055859880000000001 2024-04-20T01:47:49.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.804 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:47:49.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.793524+0000", 2024-04-20T01:47:49.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2943999999999996e-05 2024-04-20T01:47:49.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.804 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:49.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.795503+0000", 2024-04-20T01:47:49.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019791520000000001 2024-04-20T01:47:49.805 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.805 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.805 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.805 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.ds1 37/30 ECSubWrite(tid=16, reqid=client.4643.0:243, at_version=37'8, trim_to=0'0, roll_forward_to=34'6))", 2024-04-20T01:47:49.805 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.788528+0000", 2024-04-20T01:47:49.805 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 11.014261827, 2024-04-20T01:47:49.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001988461, 2024-04-20T01:47:49.805 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:49.805 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:49.805 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:49.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:49.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788528+0000", 2024-04-20T01:47:49.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:49.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788528+0000", 2024-04-20T01:47:49.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:49.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788527+0000", 2024-04-20T01:47:49.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:49.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:49.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788568+0000", 2024-04-20T01:47:49.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0674999999999999e-05 2024-04-20T01:47:49.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:49.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788581+0000", 2024-04-20T01:47:49.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3444e-05 2024-04-20T01:47:49.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788605+0000", 2024-04-20T01:47:49.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3742000000000001e-05 2024-04-20T01:47:49.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788680+0000", 2024-04-20T01:47:49.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4637999999999999e-05 2024-04-20T01:47:49.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788693+0000", 2024-04-20T01:47:49.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3084999999999999e-05 2024-04-20T01:47:49.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.790486+0000", 2024-04-20T01:47:49.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017930240000000001 2024-04-20T01:47:49.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.809 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:49.809 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.790516+0000", 2024-04-20T01:47:49.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0275999999999999e-05 2024-04-20T01:47:49.809 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.809 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.809 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.809 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(osd.13.22:3 3.fs0 3:f3742432:::smithi16579500-42:head [assert-version v6,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:47:49.809 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.788849+0000", 2024-04-20T01:47:49.809 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 11.013940656000001, 2024-04-20T01:47:49.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004535295, 2024-04-20T01:47:49.810 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:49.810 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:47:49.810 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:49.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.810 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:49.810 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788849+0000", 2024-04-20T01:47:49.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.810 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:49.810 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788849+0000", 2024-04-20T01:47:49.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:49.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788847+0000", 2024-04-20T01:47:49.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:49.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:49.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788856+0000", 2024-04-20T01:47:49.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4139999999999994e-06 2024-04-20T01:47:49.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:49.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788868+0000", 2024-04-20T01:47:49.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2235999999999999e-05 2024-04-20T01:47:49.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788880+0000", 2024-04-20T01:47:49.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2122e-05 2024-04-20T01:47:49.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788930+0000", 2024-04-20T01:47:49.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0160000000000001e-05 2024-04-20T01:47:49.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op must wait for map", 2024-04-20T01:47:49.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788938+0000", 2024-04-20T01:47:49.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8269999999999998e-06 2024-04-20T01:47:49.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.791323+0000", 2024-04-20T01:47:49.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002384471 2024-04-20T01:47:49.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:47:49.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.791386+0000", 2024-04-20T01:47:49.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3707000000000005e-05 2024-04-20T01:47:49.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:49.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.793384+0000", 2024-04-20T01:47:49.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019975800000000001 2024-04-20T01:47:49.814 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.814 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.814 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.814 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpReadReply(3.fs0 36/30 ECSubReadReply(tid=21, attrs_read=0))", 2024-04-20T01:47:49.814 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.791264+0000", 2024-04-20T01:47:49.814 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 11.011525463, 2024-04-20T01:47:49.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015999829999999999, 2024-04-20T01:47:49.814 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:49.814 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:49.814 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:49.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.815 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:49.815 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.791264+0000", 2024-04-20T01:47:49.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.815 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:49.815 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.791264+0000", 2024-04-20T01:47:49.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.815 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:49.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.791264+0000", 2024-04-20T01:47:49.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:49.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:49.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.791444+0000", 2024-04-20T01:47:49.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018027200000000001 2024-04-20T01:47:49.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:49.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.791455+0000", 2024-04-20T01:47:49.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0669999999999999e-05 2024-04-20T01:47:49.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.791458+0000", 2024-04-20T01:47:49.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8820000000000003e-06 2024-04-20T01:47:49.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.792374+0000", 2024-04-20T01:47:49.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000915495 2024-04-20T01:47:49.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:49.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.792864+0000", 2024-04-20T01:47:49.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00048994700000000002 2024-04-20T01:47:49.818 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.818 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.818 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.818 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s1 37/30 ECSubWrite(tid=202, reqid=client.4643.0:232, at_version=37'10, trim_to=0'0, roll_forward_to=33'8))", 2024-04-20T01:47:49.818 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.797639+0000", 2024-04-20T01:47:49.818 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 11.005150617, 2024-04-20T01:47:49.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023990209999999999, 2024-04-20T01:47:49.818 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:49.818 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:49.818 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:49.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:49.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.797639+0000", 2024-04-20T01:47:49.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:49.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.797639+0000", 2024-04-20T01:47:49.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:49.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.797638+0000", 2024-04-20T01:47:49.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:49.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:49.820 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.797826+0000", 2024-04-20T01:47:49.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018806300000000001 2024-04-20T01:47:49.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.820 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:49.820 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.797840+0000", 2024-04-20T01:47:49.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4167e-05 2024-04-20T01:47:49.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.820 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.820 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.797849+0000", 2024-04-20T01:47:49.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2959999999999999e-06 2024-04-20T01:47:49.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.820 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.820 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.797912+0000", 2024-04-20T01:47:49.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3171999999999998e-05 2024-04-20T01:47:49.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.821 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.797929+0000", 2024-04-20T01:47:49.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7198e-05 2024-04-20T01:47:49.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.821 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.800003+0000", 2024-04-20T01:47:49.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020742339999999999 2024-04-20T01:47:49.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:49.821 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.800038+0000", 2024-04-20T01:47:49.822 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4613999999999998e-05 2024-04-20T01:47:49.822 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.822 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.822 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.822 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.ds1 38/30 ECSubWrite(tid=19, reqid=osd.6.0:18, at_version=0'0, trim_to=0'0, roll_forward_to=38'10))", 2024-04-20T01:47:49.822 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:39.794291+0000", 2024-04-20T01:47:49.822 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.008498146999999, 2024-04-20T01:47:49.822 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068904140000000001, 2024-04-20T01:47:49.822 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:49.822 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:49.822 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:49.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.823 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:49.823 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794291+0000", 2024-04-20T01:47:49.823 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.823 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:49.823 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794291+0000", 2024-04-20T01:47:49.823 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.823 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:49.823 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794290+0000", 2024-04-20T01:47:49.823 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:49.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.824 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:49.824 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794317+0000", 2024-04-20T01:47:49.824 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7328999999999999e-05 2024-04-20T01:47:49.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.824 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:49.824 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794331+0000", 2024-04-20T01:47:49.824 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4076999999999999e-05 2024-04-20T01:47:49.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.824 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.824 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794339+0000", 2024-04-20T01:47:49.824 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6629999999999993e-06 2024-04-20T01:47:49.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794383+0000", 2024-04-20T01:47:49.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4106000000000001e-05 2024-04-20T01:47:49.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794390+0000", 2024-04-20T01:47:49.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5710000000000003e-06 2024-04-20T01:47:49.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.801146+0000", 2024-04-20T01:47:49.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0067565300000000002 2024-04-20T01:47:49.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.826 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:49.826 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.801182+0000", 2024-04-20T01:47:49.826 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5472999999999997e-05 2024-04-20T01:47:49.826 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.826 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.826 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.827 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 42/30 ECSubWrite(tid=17, reqid=client.4643.0:307, at_version=42'10, trim_to=0'0, roll_forward_to=33'8))", 2024-04-20T01:47:49.827 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.830042+0000", 2024-04-20T01:47:49.827 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.972747096, 2024-04-20T01:47:49.827 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0099127780000000006, 2024-04-20T01:47:49.827 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:49.827 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:49.827 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:49.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.827 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:49.827 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830042+0000", 2024-04-20T01:47:49.827 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.827 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:49.828 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830042+0000", 2024-04-20T01:47:49.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:49.828 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830042+0000", 2024-04-20T01:47:49.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:49.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:49.828 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830306+0000", 2024-04-20T01:47:49.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026456199999999998 2024-04-20T01:47:49.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:49.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830326+0000", 2024-04-20T01:47:49.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9919e-05 2024-04-20T01:47:49.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830337+0000", 2024-04-20T01:47:49.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0693000000000001e-05 2024-04-20T01:47:49.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830414+0000", 2024-04-20T01:47:49.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6916999999999996e-05 2024-04-20T01:47:49.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830426+0000", 2024-04-20T01:47:49.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2231e-05 2024-04-20T01:47:49.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.839923+0000", 2024-04-20T01:47:49.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094965540000000008 2024-04-20T01:47:49.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:49.831 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.839955+0000", 2024-04-20T01:47:49.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2299999999999999e-05 2024-04-20T01:47:49.831 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.831 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.831 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.831 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 42/30 ECSubWrite(tid=20, reqid=client.4643.0:311, at_version=42'13, trim_to=0'0, roll_forward_to=42'10))", 2024-04-20T01:47:49.831 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.922687+0000", 2024-04-20T01:47:49.831 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.8801018799999998, 2024-04-20T01:47:49.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026580829999999999, 2024-04-20T01:47:49.831 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:49.831 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:49.832 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:49.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:49.832 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.922687+0000", 2024-04-20T01:47:49.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:49.832 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.922687+0000", 2024-04-20T01:47:49.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:49.832 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.922687+0000", 2024-04-20T01:47:49.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:49.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.833 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:49.833 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.922917+0000", 2024-04-20T01:47:49.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023064200000000001 2024-04-20T01:47:49.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.833 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:49.833 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.922933+0000", 2024-04-20T01:47:49.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5355000000000001e-05 2024-04-20T01:47:49.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.833 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.833 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.922942+0000", 2024-04-20T01:47:49.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5659999999999996e-06 2024-04-20T01:47:49.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.922985+0000", 2024-04-20T01:47:49.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2551e-05 2024-04-20T01:47:49.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.922991+0000", 2024-04-20T01:47:49.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3709999999999996e-06 2024-04-20T01:47:49.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.925313+0000", 2024-04-20T01:47:49.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023217759999999998 2024-04-20T01:47:49.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:49.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.925346+0000", 2024-04-20T01:47:49.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2469999999999999e-05 2024-04-20T01:47:49.835 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.835 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.835 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.835 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 42/30 ECSubWrite(tid=21, reqid=client.4643.0:312, at_version=42'14, trim_to=0'0, roll_forward_to=42'10))", 2024-04-20T01:47:49.836 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.923502+0000", 2024-04-20T01:47:49.836 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.8792873429999997, 2024-04-20T01:47:49.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021709120000000001, 2024-04-20T01:47:49.836 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:49.836 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:49.836 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:49.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.836 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:49.836 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.923502+0000", 2024-04-20T01:47:49.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.836 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:49.836 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.923502+0000", 2024-04-20T01:47:49.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.837 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:49.837 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.923501+0000", 2024-04-20T01:47:49.837 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:49.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.837 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:49.837 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.923531+0000", 2024-04-20T01:47:49.837 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9796999999999999e-05 2024-04-20T01:47:49.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.837 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:49.837 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.923547+0000", 2024-04-20T01:47:49.837 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5960999999999998e-05 2024-04-20T01:47:49.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.923557+0000", 2024-04-20T01:47:49.838 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0277e-05 2024-04-20T01:47:49.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.923593+0000", 2024-04-20T01:47:49.838 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5411999999999999e-05 2024-04-20T01:47:49.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.923604+0000", 2024-04-20T01:47:49.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0866999999999999e-05 2024-04-20T01:47:49.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.925644+0000", 2024-04-20T01:47:49.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020406819999999998 2024-04-20T01:47:49.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:49.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.925673+0000", 2024-04-20T01:47:49.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8697999999999999e-05 2024-04-20T01:47:49.839 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.839 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.839 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.840 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(osd.5.21:2 3.fs0 3:fc9748e2:::smithi16579500-32:head [assert-version v4,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:47:49.840 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.924970+0000", 2024-04-20T01:47:49.840 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.8778192220000003, 2024-04-20T01:47:49.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0018882440000000001, 2024-04-20T01:47:49.840 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:49.840 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:47:49.840 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:49.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.840 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:49.840 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.924970+0000", 2024-04-20T01:47:49.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:49.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.924970+0000", 2024-04-20T01:47:49.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:49.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.924969+0000", 2024-04-20T01:47:49.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:49.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.842 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:49.842 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.924976+0000", 2024-04-20T01:47:49.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3540000000000002e-06 2024-04-20T01:47:49.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.842 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:49.842 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.924987+0000", 2024-04-20T01:47:49.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1027e-05 2024-04-20T01:47:49.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.842 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.842 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.924997+0000", 2024-04-20T01:47:49.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5880000000000004e-06 2024-04-20T01:47:49.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.925057+0000", 2024-04-20T01:47:49.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0220999999999998e-05 2024-04-20T01:47:49.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:47:49.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.925146+0000", 2024-04-20T01:47:49.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9450999999999994e-05 2024-04-20T01:47:49.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:49.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.926858+0000", 2024-04-20T01:47:49.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00171211 2024-04-20T01:47:49.844 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.844 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.844 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.844 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 42/30 ECSubWrite(tid=22, reqid=client.4643.0:314, at_version=42'16, trim_to=0'0, roll_forward_to=42'12))", 2024-04-20T01:47:49.844 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.928764+0000", 2024-04-20T01:47:49.844 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.8740248629999998, 2024-04-20T01:47:49.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036527479999999999, 2024-04-20T01:47:49.844 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:49.844 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:49.844 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:49.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.844 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:49.844 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.928764+0000", 2024-04-20T01:47:49.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:49.845 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.928764+0000", 2024-04-20T01:47:49.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:49.845 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.928764+0000", 2024-04-20T01:47:49.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:49.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:49.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.929079+0000", 2024-04-20T01:47:49.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031541300000000001 2024-04-20T01:47:49.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.846 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:49.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.929097+0000", 2024-04-20T01:47:49.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.823e-05 2024-04-20T01:47:49.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.846 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.929108+0000", 2024-04-20T01:47:49.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0139000000000001e-05 2024-04-20T01:47:49.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.847 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.929158+0000", 2024-04-20T01:47:49.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.994e-05 2024-04-20T01:47:49.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.847 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.929169+0000", 2024-04-20T01:47:49.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1464000000000001e-05 2024-04-20T01:47:49.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.848 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.932389+0000", 2024-04-20T01:47:49.848 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032196299999999998 2024-04-20T01:47:49.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.848 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:49.848 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.932417+0000", 2024-04-20T01:47:49.848 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8620999999999999e-05 2024-04-20T01:47:49.848 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.848 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.848 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.848 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.6s5 42/30 ECSubWrite(tid=8, reqid=client.4643.0:331, at_version=42'4, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:47:49.849 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.957688+0000", 2024-04-20T01:47:49.849 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.8451012740000001, 2024-04-20T01:47:49.849 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022312410000000001, 2024-04-20T01:47:49.849 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:49.849 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:49.849 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:49.849 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.849 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:49.849 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957688+0000", 2024-04-20T01:47:49.849 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.849 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.849 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:49.849 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957688+0000", 2024-04-20T01:47:49.850 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.850 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.850 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:49.850 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957688+0000", 2024-04-20T01:47:49.850 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:47:49.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.850 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.850 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:49.850 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957707+0000", 2024-04-20T01:47:49.850 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9049e-05 2024-04-20T01:47:49.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.851 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:49.851 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957721+0000", 2024-04-20T01:47:49.851 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3932e-05 2024-04-20T01:47:49.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.851 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.851 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957732+0000", 2024-04-20T01:47:49.851 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1093e-05 2024-04-20T01:47:49.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.851 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957814+0000", 2024-04-20T01:47:49.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1751999999999999e-05 2024-04-20T01:47:49.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.852 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.852 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957829+0000", 2024-04-20T01:47:49.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5500000000000001e-05 2024-04-20T01:47:49.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.852 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.852 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.959893+0000", 2024-04-20T01:47:49.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020637419999999999 2024-04-20T01:47:49.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.853 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.853 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:49.853 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.959919+0000", 2024-04-20T01:47:49.853 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.635e-05 2024-04-20T01:47:49.853 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.853 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.853 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.853 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.853 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.6s5 42/30 ECSubWrite(tid=9, reqid=client.4643.0:331, at_version=0'0, trim_to=0'0, roll_forward_to=42'4))", 2024-04-20T01:47:49.853 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.963201+0000", 2024-04-20T01:47:49.853 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.8395887640000002, 2024-04-20T01:47:49.853 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0018501419999999999, 2024-04-20T01:47:49.854 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:49.854 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:49.854 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:49.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.854 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:49.854 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.963201+0000", 2024-04-20T01:47:49.854 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.854 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:49.855 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.963201+0000", 2024-04-20T01:47:49.855 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.855 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:49.855 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.963200+0000", 2024-04-20T01:47:49.855 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:49.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.855 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:49.855 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.963217+0000", 2024-04-20T01:47:49.855 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6926999999999999e-05 2024-04-20T01:47:49.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:49.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.963232+0000", 2024-04-20T01:47:49.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.552e-05 2024-04-20T01:47:49.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.963242+0000", 2024-04-20T01:47:49.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5470000000000002e-06 2024-04-20T01:47:49.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.963290+0000", 2024-04-20T01:47:49.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8508999999999999e-05 2024-04-20T01:47:49.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.963297+0000", 2024-04-20T01:47:49.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9639999999999997e-06 2024-04-20T01:47:49.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.965024+0000", 2024-04-20T01:47:49.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017264330000000001 2024-04-20T01:47:49.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:49.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.965051+0000", 2024-04-20T01:47:49.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7056e-05 2024-04-20T01:47:49.858 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.858 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.858 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.858 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 42/30 ECSubWrite(tid=26, reqid=client.4643.0:333, at_version=42'18, trim_to=0'0, roll_forward_to=42'16))", 2024-04-20T01:47:49.858 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.973550+0000", 2024-04-20T01:47:49.858 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.8292397239999998, 2024-04-20T01:47:49.858 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020938340000000001, 2024-04-20T01:47:49.858 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:49.858 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:49.858 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:49.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.858 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:49.859 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.973550+0000", 2024-04-20T01:47:49.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.859 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:49.859 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.973550+0000", 2024-04-20T01:47:49.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.859 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:49.859 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.973549+0000", 2024-04-20T01:47:49.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:49.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.860 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:49.860 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.973943+0000", 2024-04-20T01:47:49.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00039386199999999998 2024-04-20T01:47:49.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.860 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:49.860 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.973971+0000", 2024-04-20T01:47:49.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.832e-05 2024-04-20T01:47:49.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.860 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.860 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.973990+0000", 2024-04-20T01:47:49.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8813000000000001e-05 2024-04-20T01:47:49.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.974096+0000", 2024-04-20T01:47:49.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106344 2024-04-20T01:47:49.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.974121+0000", 2024-04-20T01:47:49.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5032999999999999e-05 2024-04-20T01:47:49.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.975617+0000", 2024-04-20T01:47:49.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014960939999999999 2024-04-20T01:47:49.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:49.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.975643+0000", 2024-04-20T01:47:49.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6129e-05 2024-04-20T01:47:49.862 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.862 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.862 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.862 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 47/30 ECSubWrite(tid=39, reqid=osd.5.0:39, at_version=47'27, trim_to=0'0, roll_forward_to=46'25))", 2024-04-20T01:47:49.862 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:48.718442+0000", 2024-04-20T01:47:49.862 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.084347014, 2024-04-20T01:47:49.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027161479999999998, 2024-04-20T01:47:49.862 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:49.862 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:49.863 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:49.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.863 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:49.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.718442+0000", 2024-04-20T01:47:49.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.863 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:49.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.718442+0000", 2024-04-20T01:47:49.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.863 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:49.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.718442+0000", 2024-04-20T01:47:49.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:49.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:49.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.718464+0000", 2024-04-20T01:47:49.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2195999999999999e-05 2024-04-20T01:47:49.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:49.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.718476+0000", 2024-04-20T01:47:49.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1761e-05 2024-04-20T01:47:49.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.718485+0000", 2024-04-20T01:47:49.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9490000000000002e-06 2024-04-20T01:47:49.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.718527+0000", 2024-04-20T01:47:49.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1631000000000002e-05 2024-04-20T01:47:49.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.718533+0000", 2024-04-20T01:47:49.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4189999999999999e-06 2024-04-20T01:47:49.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.721128+0000", 2024-04-20T01:47:49.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002594903 2024-04-20T01:47:49.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:49.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.721158+0000", 2024-04-20T01:47:49.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0550999999999999e-05 2024-04-20T01:47:49.866 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.866 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.866 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.866 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 47/30 ECSubWrite(tid=40, reqid=osd.5.0:40, at_version=47'29, trim_to=0'0, roll_forward_to=46'25))", 2024-04-20T01:47:49.866 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:48.719180+0000", 2024-04-20T01:47:49.866 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.0836096399999999, 2024-04-20T01:47:49.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002933055, 2024-04-20T01:47:49.866 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:49.867 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:49.867 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:49.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:49.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.719180+0000", 2024-04-20T01:47:49.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:49.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.719180+0000", 2024-04-20T01:47:49.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:49.868 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.719179+0000", 2024-04-20T01:47:49.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:49.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.868 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:49.868 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.719204+0000", 2024-04-20T01:47:49.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5051000000000001e-05 2024-04-20T01:47:49.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.868 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:49.868 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.719216+0000", 2024-04-20T01:47:49.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2361000000000001e-05 2024-04-20T01:47:49.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.868 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.719226+0000", 2024-04-20T01:47:49.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8449999999999998e-06 2024-04-20T01:47:49.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.719280+0000", 2024-04-20T01:47:49.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3532999999999997e-05 2024-04-20T01:47:49.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.719289+0000", 2024-04-20T01:47:49.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9160000000000007e-06 2024-04-20T01:47:49.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722091+0000", 2024-04-20T01:47:49.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028020089999999998 2024-04-20T01:47:49.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:49.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722113+0000", 2024-04-20T01:47:49.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2172999999999999e-05 2024-04-20T01:47:49.870 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.870 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.870 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.870 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s1 47/30 ECSubWrite(tid=241, reqid=osd.13.0:241, at_version=47'20, trim_to=0'0, roll_forward_to=44'18))", 2024-04-20T01:47:49.870 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:48.722207+0000", 2024-04-20T01:47:49.871 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.080582371, 2024-04-20T01:47:49.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002400545, 2024-04-20T01:47:49.871 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:49.871 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:49.871 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:49.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:49.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722207+0000", 2024-04-20T01:47:49.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:49.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722207+0000", 2024-04-20T01:47:49.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.872 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.872 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:49.872 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722207+0000", 2024-04-20T01:47:49.872 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:47:49.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.872 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.872 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:49.872 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722229+0000", 2024-04-20T01:47:49.872 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2589e-05 2024-04-20T01:47:49.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.872 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.872 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:49.872 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722243+0000", 2024-04-20T01:47:49.872 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.379e-05 2024-04-20T01:47:49.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.873 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.873 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722252+0000", 2024-04-20T01:47:49.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0729999999999994e-06 2024-04-20T01:47:49.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.873 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.873 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722306+0000", 2024-04-20T01:47:49.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3819000000000001e-05 2024-04-20T01:47:49.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.873 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.873 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722315+0000", 2024-04-20T01:47:49.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1770000000000004e-06 2024-04-20T01:47:49.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.724577+0000", 2024-04-20T01:47:49.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022613170000000001 2024-04-20T01:47:49.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:49.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.724608+0000", 2024-04-20T01:47:49.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1013000000000003e-05 2024-04-20T01:47:49.874 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.874 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.874 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.875 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.as6 47/30 ECSubWrite(tid=32, reqid=osd.6.0:31, at_version=0'0, trim_to=0'0, roll_forward_to=47'12))", 2024-04-20T01:47:49.875 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:48.722985+0000", 2024-04-20T01:47:49.875 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.0798039909999999, 2024-04-20T01:47:49.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022214370000000002, 2024-04-20T01:47:49.875 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:49.875 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:49.875 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:49.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:49.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722985+0000", 2024-04-20T01:47:49.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:49.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722985+0000", 2024-04-20T01:47:49.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:49.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:49.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722984+0000", 2024-04-20T01:47:49.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:49.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:49.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722999+0000", 2024-04-20T01:47:49.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4637999999999999e-05 2024-04-20T01:47:49.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:49.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.723013+0000", 2024-04-20T01:47:49.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3852000000000001e-05 2024-04-20T01:47:49.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.877 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:49.877 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.723025+0000", 2024-04-20T01:47:49.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1647e-05 2024-04-20T01:47:49.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.877 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:49.877 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.723304+0000", 2024-04-20T01:47:49.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000279695 2024-04-20T01:47:49.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.877 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:49.877 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.723315+0000", 2024-04-20T01:47:49.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0497e-05 2024-04-20T01:47:49.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:49.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.725175+0000", 2024-04-20T01:47:49.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001860443 2024-04-20T01:47:49.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:49.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:49.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:49.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.725207+0000", 2024-04-20T01:47:49.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1625999999999998e-05 2024-04-20T01:47:49.878 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.878 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.878 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.879 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:49.879 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:49.879 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:49.880 INFO:tasks.rados.rados.0.smithi165.stdout:168: finishing write tid 2 to smithi16579500-38 2024-04-20T01:47:49.880 INFO:tasks.rados.rados.0.smithi165.stdout:168: finishing write tid 3 to smithi16579500-38 2024-04-20T01:47:49.880 INFO:tasks.rados.rados.0.smithi165.stdout:168: finishing write tid 4 to smithi16579500-38 2024-04-20T01:47:49.880 INFO:tasks.rados.rados.0.smithi165.stdout:168: finishing write tid 5 to smithi16579500-38 2024-04-20T01:47:49.880 INFO:tasks.rados.rados.0.smithi165.stdout:168: oid 38 updating version 0 to 47 2024-04-20T01:47:49.880 INFO:tasks.rados.rados.0.smithi165.stdout:168: oid 38 updating version 47 to 48 2024-04-20T01:47:49.880 INFO:tasks.rados.rados.0.smithi165.stdout:168: oid 38 version 48 is already newer than 46 2024-04-20T01:47:49.880 INFO:tasks.rados.rados.0.smithi165.stdout:168: oid 38 updating version 48 to 49 2024-04-20T01:47:49.880 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 38 v 49 (ObjNum 83 snap 8 seq_num 83) dirty exists 2024-04-20T01:47:49.881 INFO:tasks.rados.rados.0.smithi165.stdout:168: left oid 38 (ObjNum 83 snap 8 seq_num 83) 2024-04-20T01:47:49.881 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:49.894+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:49.881 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-20T01:47:49.981 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:47:49.995+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:47:50.009 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:50.010 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:47:50.010 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:47:50.010 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:50.019 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-20T01:47:50.082 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:47:50.096+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:47:50.136 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:47:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T01:47:50.147 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:50.148 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:47:50.148 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 30, 2024-04-20T01:47:50.148 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-20T01:47:50.148 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:50.150 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2024-04-20T01:47:49.706398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":38732320,"num_objects":65,"num_object_clones":18,"num_object_copies":445,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":4,"num_read":360,"num_read_kb":30398,"num_write":592,"num_write_kb":46372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":463,"ondisk_log_size":463,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":501520,"kb_used_data":69092,"kb_used_omap":24,"kb_used_meta":432103,"kb_avail":1509447920,"statfs":{"total":1546188226560,"available":1545674670080,"internally_reserved":0,"allocated":70750208,"data_stored":76816481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25507,"internal_metadata":442473565},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8830976,"num_objects":-7,"num_object_clones":-2,"num_object_copies":-49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7,"num_whiteouts":-3,"num_read":47,"num_read_kb":12161,"num_write":188,"num_write_kb":13296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.891675"},"pg_stats":[{"pgid":"3.f","version":"41'12","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.807655+0000","last_change":"2024-04-20T01:47:42.818507+0000","last_active":"2024-04-20T01:47:44.807655+0000","last_peered":"2024-04-20T01:47:44.807655+0000","last_clean":"2024-04-20T01:47:44.807655+0000","last_became_active":"2024-04-20T01:47:32.867228+0000","last_became_peered":"2024-04-20T01:47:32.867228+0000","last_unstale":"2024-04-20T01:47:44.807655+0000","last_undegraded":"2024-04-20T01:47:44.807655+0000","last_fullsized":"2024-04-20T01:47:44.807655+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.328818+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015122383,"stat_sum":{"num_bytes":1228800,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":5142,"num_write":47,"num_write_kb":1488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,15,13,8,11,0],"acting":[2,3,15,13,8,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"3.e","version":"42'21","reported_seq":76,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.821067+0000","last_change":"2024-04-20T01:47:42.822202+0000","last_active":"2024-04-20T01:47:45.821067+0000","last_peered":"2024-04-20T01:47:45.821067+0000","last_clean":"2024-04-20T01:47:45.821067+0000","last_became_active":"2024-04-20T01:47:32.867409+0000","last_became_peered":"2024-04-20T01:47:32.867409+0000","last_unstale":"2024-04-20T01:47:45.821067+0000","last_undegraded":"2024-04-20T01:47:45.821067+0000","last_fullsized":"2024-04-20T01:47:45.821067+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:42.581217+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012353486,"stat_sum":{"num_bytes":1605632,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":24,"num_read_kb":1735,"num_write":13,"num_write_kb":3280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,1,10,8,3,12],"acting":[11,9,1,10,8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"3.d","version":"41'12","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.816124+0000","last_change":"2024-04-20T01:47:42.815568+0000","last_active":"2024-04-20T01:47:44.816124+0000","last_peered":"2024-04-20T01:47:44.816124+0000","last_clean":"2024-04-20T01:47:44.816124+0000","last_became_active":"2024-04-20T01:47:32.867954+0000","last_became_peered":"2024-04-20T01:47:32.867954+0000","last_unstale":"2024-04-20T01:47:44.816124+0000","last_undegraded":"2024-04-20T01:47:44.816124+0000","last_fullsized":"2024-04-20T01:47:44.816124+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.083554+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011117459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,11,15,7,8,4],"acting":[6,2,11,15,7,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"3.c","version":"42'24","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.816615+0000","last_change":"2024-04-20T01:47:42.812324+0000","last_active":"2024-04-20T01:47:44.816615+0000","last_peered":"2024-04-20T01:47:44.816615+0000","last_clean":"2024-04-20T01:47:44.816615+0000","last_became_active":"2024-04-20T01:47:32.867926+0000","last_became_peered":"2024-04-20T01:47:32.867926+0000","last_unstale":"2024-04-20T01:47:44.816615+0000","last_undegraded":"2024-04-20T01:47:44.816615+0000","last_fullsized":"2024-04-20T01:47:44.816615+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:54.374528+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012044350000000001,"stat_sum":{"num_bytes":4292608,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":16,"num_write_kb":4192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,8,6,3,7,12,5],"acting":[14,8,6,3,7,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"3.b","version":"44'43","reported_seq":105,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.868627+0000","last_change":"2024-04-20T01:47:42.837347+0000","last_active":"2024-04-20T01:47:45.868627+0000","last_peered":"2024-04-20T01:47:45.868627+0000","last_clean":"2024-04-20T01:47:45.868627+0000","last_became_active":"2024-04-20T01:47:32.867309+0000","last_became_peered":"2024-04-20T01:47:32.867309+0000","last_unstale":"2024-04-20T01:47:45.868627+0000","last_undegraded":"2024-04-20T01:47:45.868627+0000","last_fullsized":"2024-04-20T01:47:45.868627+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:01.227735+0000","scrub_duration":0,"objects_trimmed":5,"snaptrim_duration":0.024423878999999999,"stat_sum":{"num_bytes":6799360,"num_objects":8,"num_object_clones":2,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":35,"num_read_kb":5390,"num_write":87,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,11,10,9],"acting":[4,0,1,12,11,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"3.a","version":"44'10","reported_seq":56,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.839665+0000","last_change":"2024-04-20T01:47:42.819658+0000","last_active":"2024-04-20T01:47:45.839665+0000","last_peered":"2024-04-20T01:47:45.839665+0000","last_clean":"2024-04-20T01:47:45.839665+0000","last_became_active":"2024-04-20T01:47:32.868002+0000","last_became_peered":"2024-04-20T01:47:32.868002+0000","last_unstale":"2024-04-20T01:47:45.839665+0000","last_undegraded":"2024-04-20T01:47:45.839665+0000","last_fullsized":"2024-04-20T01:47:45.839665+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:35.716198+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.012917052,"stat_sum":{"num_bytes":1736704,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":8,"num_read_kb":1971,"num_write":6,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,1,14,3,15,2],"acting":[6,10,1,14,3,15,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"3.9","version":"42'6","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.815818+0000","last_change":"2024-04-20T01:47:42.807106+0000","last_active":"2024-04-20T01:47:44.815818+0000","last_peered":"2024-04-20T01:47:44.815818+0000","last_clean":"2024-04-20T01:47:44.815818+0000","last_became_active":"2024-04-20T01:47:32.866706+0000","last_became_peered":"2024-04-20T01:47:32.866706+0000","last_unstale":"2024-04-20T01:47:44.815818+0000","last_undegraded":"2024-04-20T01:47:44.815818+0000","last_fullsized":"2024-04-20T01:47:44.815818+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:42.776206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056749800000000005,"stat_sum":{"num_bytes":507904,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":5,"num_write_kb":496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0,11,14,8,3],"acting":[7,1,0,11,14,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"3.8","version":"42'20","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.816129+0000","last_change":"2024-04-20T01:47:42.804832+0000","last_active":"2024-04-20T01:47:44.816129+0000","last_peered":"2024-04-20T01:47:44.816129+0000","last_clean":"2024-04-20T01:47:44.816129+0000","last_became_active":"2024-04-20T01:47:32.718069+0000","last_became_peered":"2024-04-20T01:47:32.718069+0000","last_unstale":"2024-04-20T01:47:44.816129+0000","last_undegraded":"2024-04-20T01:47:44.816129+0000","last_fullsized":"2024-04-20T01:47:44.816129+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:58.489885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053838200000000001,"stat_sum":{"num_bytes":3997696,"num_objects":7,"num_object_clones":3,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":535,"num_write":17,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,11,3,14,2,7],"acting":[5,0,11,3,14,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.806979+0000","last_change":"2024-04-20T01:47:20.071988+0000","last_active":"2024-04-20T01:47:44.806979+0000","last_peered":"2024-04-20T01:47:44.806979+0000","last_clean":"2024-04-20T01:47:44.806979+0000","last_became_active":"2024-04-20T01:47:20.071615+0000","last_became_peered":"2024-04-20T01:47:20.071615+0000","last_unstale":"2024-04-20T01:47:44.806979+0000","last_undegraded":"2024-04-20T01:47:44.806979+0000","last_fullsized":"2024-04-20T01:47:44.806979+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:40.106777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"41'5","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.816765+0000","last_change":"2024-04-20T01:47:42.809817+0000","last_active":"2024-04-20T01:47:44.816765+0000","last_peered":"2024-04-20T01:47:44.816765+0000","last_clean":"2024-04-20T01:47:44.816765+0000","last_became_active":"2024-04-20T01:47:32.867180+0000","last_became_peered":"2024-04-20T01:47:32.867180+0000","last_unstale":"2024-04-20T01:47:44.816765+0000","last_undegraded":"2024-04-20T01:47:44.816765+0000","last_fullsized":"2024-04-20T01:47:44.816765+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:43.358783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003779077,"stat_sum":{"num_bytes":360448,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":5,"num_write_kb":416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,4,14,12,0,11],"acting":[10,8,4,14,12,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.807531+0000","last_change":"2024-04-20T01:47:20.075613+0000","last_active":"2024-04-20T01:47:44.807531+0000","last_peered":"2024-04-20T01:47:44.807531+0000","last_clean":"2024-04-20T01:47:44.807531+0000","last_became_active":"2024-04-20T01:47:20.075314+0000","last_became_peered":"2024-04-20T01:47:20.075314+0000","last_unstale":"2024-04-20T01:47:44.807531+0000","last_undegraded":"2024-04-20T01:47:44.807531+0000","last_fullsized":"2024-04-20T01:47:44.807531+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:35.956173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,12],"acting":[2,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"44'18","reported_seq":66,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.845417+0000","last_change":"2024-04-20T01:47:42.809609+0000","last_active":"2024-04-20T01:47:45.845417+0000","last_peered":"2024-04-20T01:47:45.845417+0000","last_clean":"2024-04-20T01:47:45.845417+0000","last_became_active":"2024-04-20T01:47:32.719136+0000","last_became_peered":"2024-04-20T01:47:32.719136+0000","last_unstale":"2024-04-20T01:47:45.845417+0000","last_undegraded":"2024-04-20T01:47:45.845417+0000","last_fullsized":"2024-04-20T01:47:45.845417+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:46.940575+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0089564480000000005,"stat_sum":{"num_bytes":1540096,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":3,"num_write":30,"num_write_kb":2128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,4,14,11,5,10],"acting":[13,2,4,14,11,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.829483+0000","last_change":"2024-04-20T01:47:20.039221+0000","last_active":"2024-04-20T01:47:45.829483+0000","last_peered":"2024-04-20T01:47:45.829483+0000","last_clean":"2024-04-20T01:47:45.829483+0000","last_became_active":"2024-04-20T01:47:20.038972+0000","last_became_peered":"2024-04-20T01:47:20.038972+0000","last_unstale":"2024-04-20T01:47:45.829483+0000","last_undegraded":"2024-04-20T01:47:45.829483+0000","last_fullsized":"2024-04-20T01:47:45.829483+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:24.167832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"44'17","reported_seq":67,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.828308+0000","last_change":"2024-04-20T01:47:42.819924+0000","last_active":"2024-04-20T01:47:45.828308+0000","last_peered":"2024-04-20T01:47:45.828308+0000","last_clean":"2024-04-20T01:47:45.828308+0000","last_became_active":"2024-04-20T01:47:32.868365+0000","last_became_peered":"2024-04-20T01:47:32.868365+0000","last_unstale":"2024-04-20T01:47:45.828308+0000","last_undegraded":"2024-04-20T01:47:45.828308+0000","last_fullsized":"2024-04-20T01:47:45.828308+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:41.198793+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.015482437999999999,"stat_sum":{"num_bytes":1343488,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":14,"num_read_kb":1685,"num_write":29,"num_write_kb":1312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,1,5,15,4],"acting":[3,7,6,1,5,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.816422+0000","last_change":"2024-04-20T01:47:20.039014+0000","last_active":"2024-04-20T01:47:44.816422+0000","last_peered":"2024-04-20T01:47:44.816422+0000","last_clean":"2024-04-20T01:47:44.816422+0000","last_became_active":"2024-04-20T01:47:20.038764+0000","last_became_peered":"2024-04-20T01:47:20.038764+0000","last_unstale":"2024-04-20T01:47:44.816422+0000","last_undegraded":"2024-04-20T01:47:44.816422+0000","last_fullsized":"2024-04-20T01:47:44.816422+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:26.112165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"25'157","reported_seq":225,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.829609+0000","last_change":"2024-04-20T01:47:17.303029+0000","last_active":"2024-04-20T01:47:45.829609+0000","last_peered":"2024-04-20T01:47:45.829609+0000","last_clean":"2024-04-20T01:47:45.829609+0000","last_became_active":"2024-04-20T01:47:17.302105+0000","last_became_peered":"2024-04-20T01:47:17.302105+0000","last_unstale":"2024-04-20T01:47:45.829609+0000","last_undegraded":"2024-04-20T01:47:45.829609+0000","last_fullsized":"2024-04-20T01:47:45.829609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_clean_scrub_stamp":"2024-04-20T01:47:15.986754+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:30.481506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,15],"acting":[13,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.2","version":"44'31","reported_seq":92,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.844195+0000","last_change":"2024-04-20T01:47:42.813164+0000","last_active":"2024-04-20T01:47:45.844195+0000","last_peered":"2024-04-20T01:47:45.844195+0000","last_clean":"2024-04-20T01:47:45.844195+0000","last_became_active":"2024-04-20T01:47:32.866509+0000","last_became_peered":"2024-04-20T01:47:32.866509+0000","last_unstale":"2024-04-20T01:47:45.844195+0000","last_undegraded":"2024-04-20T01:47:45.844195+0000","last_fullsized":"2024-04-20T01:47:45.844195+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.395078+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012506468999999999,"stat_sum":{"num_bytes":4505600,"num_objects":9,"num_object_clones":3,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":32,"num_read_kb":8603,"num_write":40,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,6,10,14,1,7],"acting":[13,3,6,10,14,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.807472+0000","last_change":"2024-04-20T01:47:20.069279+0000","last_active":"2024-04-20T01:47:44.807472+0000","last_peered":"2024-04-20T01:47:44.807472+0000","last_clean":"2024-04-20T01:47:44.807472+0000","last_became_active":"2024-04-20T01:47:20.069005+0000","last_became_peered":"2024-04-20T01:47:20.069005+0000","last_unstale":"2024-04-20T01:47:44.807472+0000","last_undegraded":"2024-04-20T01:47:44.807472+0000","last_fullsized":"2024-04-20T01:47:44.807472+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:22.648743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"41'10","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.807023+0000","last_change":"2024-04-20T01:47:42.820323+0000","last_active":"2024-04-20T01:47:44.807023+0000","last_peered":"2024-04-20T01:47:44.807023+0000","last_clean":"2024-04-20T01:47:44.807023+0000","last_became_active":"2024-04-20T01:47:32.722189+0000","last_became_peered":"2024-04-20T01:47:32.722189+0000","last_unstale":"2024-04-20T01:47:44.807023+0000","last_undegraded":"2024-04-20T01:47:44.807023+0000","last_fullsized":"2024-04-20T01:47:44.807023+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:43.443403+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.007645771,"stat_sum":{"num_bytes":344064,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":1011,"num_write":5,"num_write_kb":768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,14,9,5,6,3,13],"acting":[4,14,9,5,6,3,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.816006+0000","last_change":"2024-04-20T01:47:20.038409+0000","last_active":"2024-04-20T01:47:44.816006+0000","last_peered":"2024-04-20T01:47:44.816006+0000","last_clean":"2024-04-20T01:47:44.816006+0000","last_became_active":"2024-04-20T01:47:20.038107+0000","last_became_peered":"2024-04-20T01:47:20.038107+0000","last_unstale":"2024-04-20T01:47:44.816006+0000","last_undegraded":"2024-04-20T01:47:44.816006+0000","last_fullsized":"2024-04-20T01:47:44.816006+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:44.125577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"44'61","reported_seq":113,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.892029+0000","last_change":"2024-04-20T01:47:42.832597+0000","last_active":"2024-04-20T01:47:45.892029+0000","last_peered":"2024-04-20T01:47:45.892029+0000","last_clean":"2024-04-20T01:47:45.892029+0000","last_became_active":"2024-04-20T01:47:32.867693+0000","last_became_peered":"2024-04-20T01:47:32.867693+0000","last_unstale":"2024-04-20T01:47:45.892029+0000","last_undegraded":"2024-04-20T01:47:45.892029+0000","last_fullsized":"2024-04-20T01:47:45.892029+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:31.824121+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.032078015000000001,"stat_sum":{"num_bytes":7225344,"num_objects":9,"num_object_clones":3,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":28,"num_read_kb":3805,"num_write":52,"num_write_kb":7840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,15,4,1,0,3,13],"acting":[8,15,4,1,0,3,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"3.7","version":"41'12","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.806971+0000","last_change":"2024-04-20T01:47:42.826203+0000","last_active":"2024-04-20T01:47:44.806971+0000","last_peered":"2024-04-20T01:47:44.806971+0000","last_clean":"2024-04-20T01:47:44.806971+0000","last_became_active":"2024-04-20T01:47:32.722002+0000","last_became_peered":"2024-04-20T01:47:32.722002+0000","last_unstale":"2024-04-20T01:47:44.806971+0000","last_undegraded":"2024-04-20T01:47:44.806971+0000","last_fullsized":"2024-04-20T01:47:44.806971+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:43.990113+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.013542391000000001,"stat_sum":{"num_bytes":2523136,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":28,"num_write_kb":2464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,0,8,5,14],"acting":[4,11,9,0,8,5,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.819716+0000","last_change":"2024-04-20T01:47:20.110600+0000","last_active":"2024-04-20T01:47:44.819716+0000","last_peered":"2024-04-20T01:47:44.819716+0000","last_clean":"2024-04-20T01:47:44.819716+0000","last_became_active":"2024-04-20T01:47:20.110260+0000","last_became_peered":"2024-04-20T01:47:20.110260+0000","last_unstale":"2024-04-20T01:47:44.819716+0000","last_undegraded":"2024-04-20T01:47:44.819716+0000","last_fullsized":"2024-04-20T01:47:44.819716+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:47.718290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.816246+0000","last_change":"2024-04-20T01:47:20.036634+0000","last_active":"2024-04-20T01:47:44.816246+0000","last_peered":"2024-04-20T01:47:44.816246+0000","last_clean":"2024-04-20T01:47:44.816246+0000","last_became_active":"2024-04-20T01:47:20.036328+0000","last_became_peered":"2024-04-20T01:47:20.036328+0000","last_unstale":"2024-04-20T01:47:44.816246+0000","last_undegraded":"2024-04-20T01:47:44.816246+0000","last_fullsized":"2024-04-20T01:47:44.816246+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.385250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,13],"acting":[6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"42'4","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.819220+0000","last_change":"2024-04-20T01:47:42.820437+0000","last_active":"2024-04-20T01:47:44.819220+0000","last_peered":"2024-04-20T01:47:44.819220+0000","last_clean":"2024-04-20T01:47:44.819220+0000","last_became_active":"2024-04-20T01:47:32.866834+0000","last_became_peered":"2024-04-20T01:47:32.866834+0000","last_unstale":"2024-04-20T01:47:44.819220+0000","last_undegraded":"2024-04-20T01:47:44.819220+0000","last_fullsized":"2024-04-20T01:47:44.819220+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:54.424527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001992705,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":7,"num_read_kb":386,"num_write":3,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,3,6,2,9],"acting":[0,13,1,3,6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":38141952,"num_objects":63,"num_object_clones":18,"num_object_copies":441,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":4,"num_read":224,"num_read_kb":30280,"num_write":389,"num_write_kb":42912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":66748416,"data_stored":83148800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":112,"acting":112,"num_store_stats":16},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280521,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":4724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":4837376,"data_stored":5903706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]}]},{"osd":15,"up_from":21,"seq":90194313225,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30244,"kb_used_data":3220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341596,"statfs":{"total":96636764160,"available":96605794304,"internally_reserved":0,"allocated":3297280,"data_stored":3344371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]}]},{"osd":14,"up_from":21,"seq":90194313226,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":5368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":5496832,"data_stored":5548889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,4,5,6,7,8,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]}]},{"osd":12,"up_from":21,"seq":90194313225,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30760,"kb_used_data":3740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341080,"statfs":{"total":96636764160,"available":96605265920,"internally_reserved":0,"allocated":3829760,"data_stored":4515221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]}]},{"osd":11,"up_from":21,"seq":90194313224,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31780,"kb_used_data":4752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340060,"statfs":{"total":96636764160,"available":96604221440,"internally_reserved":0,"allocated":4866048,"data_stored":5340596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,8,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":10,"up_from":21,"seq":90194313226,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":4260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":4362240,"data_stored":4935092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,4,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.915}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3049999999999997}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]}]},{"osd":9,"up_from":21,"seq":90194313226,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30032,"kb_used_data":3004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341808,"statfs":{"total":96636764160,"available":96606011392,"internally_reserved":0,"allocated":3076096,"data_stored":3513780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,4,6,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]},{"osd":8,"up_from":21,"seq":90194313225,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31260,"kb_used_data":4232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340580,"statfs":{"total":96636764160,"available":96604753920,"internally_reserved":0,"allocated":4333568,"data_stored":4525492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,6,7,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9630000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]}]},{"osd":7,"up_from":21,"seq":90194313226,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31140,"kb_used_data":4112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340700,"statfs":{"total":96636764160,"available":96604876800,"internally_reserved":0,"allocated":4210688,"data_stored":4259252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,3,4,6,8,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":6,"up_from":21,"seq":90194313224,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30580,"kb_used_data":3552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341260,"statfs":{"total":96636764160,"available":96605450240,"internally_reserved":0,"allocated":3637248,"data_stored":3648948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]}]},{"osd":1,"up_from":21,"seq":90194313225,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":5408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":5537792,"data_stored":6426036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]}]},{"osd":0,"up_from":21,"seq":90194313226,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":5140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":5263360,"data_stored":6056810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]}]},{"osd":2,"up_from":21,"seq":90194313224,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29672,"kb_used_data":2644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342168,"statfs":{"total":96636764160,"available":96606380032,"internally_reserved":0,"allocated":2707456,"data_stored":2473396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]}]},{"osd":3,"up_from":21,"seq":90194313226,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33500,"kb_used_data":6472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338340,"statfs":{"total":96636764160,"available":96602460160,"internally_reserved":0,"allocated":6627328,"data_stored":6991284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,4,5,6,7,8,9,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]}]},{"osd":4,"up_from":21,"seq":90194313226,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31528,"kb_used_data":4500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":4608000,"data_stored":5361076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,5,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]}]},{"osd":5,"up_from":21,"seq":90194313225,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30992,"kb_used_data":3964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340848,"statfs":{"total":96636764160,"available":96605028352,"internally_reserved":0,"allocated":4059136,"data_stored":3972532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5693440,"data_stored":7639040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5963776,"data_stored":8007680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2158592,"data_stored":2310144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6729728,"data_stored":8142848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5033984,"data_stored":7057408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3510272,"data_stored":3809280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3088384,"data_stored":3371008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3661824,"data_stored":3981312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4435968,"data_stored":5677056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":2850816,"data_stored":3665920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4136960,"data_stored":5201920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4640768,"data_stored":5607424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":3264512,"data_stored":4227072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":3743744,"data_stored":5087232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":4952064,"data_stored":5386240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":2883584,"data_stored":3977216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T01:47:50.154 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T01:47:50.158 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-20T01:47:50.192 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:47:50.206+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:47:50.284 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:47:50.298+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:47:50.287 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:50.287 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-20T01:47:50.287 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-20T01:47:50.287 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-20T01:47:50.287 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.288 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.1s4 36/30 ECSubWrite(tid=56, reqid=osd.4.0:55, at_version=0'0, trim_to=0'0, roll_forward_to=36'8))", 2024-04-20T01:47:50.288 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:37.770698+0000", 2024-04-20T01:47:50.288 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 12.530254358000001, 2024-04-20T01:47:50.288 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051485360000000004, 2024-04-20T01:47:50.288 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.288 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.288 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.288 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.288 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.288 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:37.770698+0000", 2024-04-20T01:47:50.288 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.288 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.288 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.289 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:37.770698+0000", 2024-04-20T01:47:50.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.289 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.289 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:37.770697+0000", 2024-04-20T01:47:50.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:50.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.289 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.289 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:37.770715+0000", 2024-04-20T01:47:50.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7952e-05 2024-04-20T01:47:50.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.290 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:37.770736+0000", 2024-04-20T01:47:50.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0879000000000001e-05 2024-04-20T01:47:50.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.290 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:37.770748+0000", 2024-04-20T01:47:50.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2334000000000001e-05 2024-04-20T01:47:50.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.290 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:37.771313+0000", 2024-04-20T01:47:50.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00056510299999999996 2024-04-20T01:47:50.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:37.771320+0000", 2024-04-20T01:47:50.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0899999999999999e-06 2024-04-20T01:47:50.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:37.775828+0000", 2024-04-20T01:47:50.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045072549999999999 2024-04-20T01:47:50.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.292 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:37.775846+0000", 2024-04-20T01:47:50.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8743e-05 2024-04-20T01:47:50.292 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.292 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.292 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.292 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:243 3.ds0 3:b226b74f:::smithi16579500-7:head [delete] snapc 5=[5,2] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:47:50.292 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.787656+0000", 2024-04-20T01:47:50.292 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 11.513296240000001, 2024-04-20T01:47:50.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013205184, 2024-04-20T01:47:50.292 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.292 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:50.292 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:47:50.293 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:47:50.293 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:50.293 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 243 2024-04-20T01:47:50.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.293 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787656+0000", 2024-04-20T01:47:50.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787656+0000", 2024-04-20T01:47:50.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.294 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.294 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.294 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787654+0000", 2024-04-20T01:47:50.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:50.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.294 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.294 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.294 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787662+0000", 2024-04-20T01:47:50.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6599999999999995e-06 2024-04-20T01:47:50.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.294 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.294 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.294 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787680+0000", 2024-04-20T01:47:50.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7672999999999998e-05 2024-04-20T01:47:50.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787688+0000", 2024-04-20T01:47:50.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.405e-06 2024-04-20T01:47:50.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787739+0000", 2024-04-20T01:47:50.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0652999999999998e-05 2024-04-20T01:47:50.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:47:50.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787851+0000", 2024-04-20T01:47:50.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112172 2024-04-20T01:47:50.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.296 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.296 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.296 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788643+0000", 2024-04-20T01:47:50.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00079174300000000003 2024-04-20T01:47:50.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.296 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.296 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.296 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.789564+0000", 2024-04-20T01:47:50.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00092135199999999996 2024-04-20T01:47:50.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:47:50.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.800781+0000", 2024-04-20T01:47:50.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011217187999999999 2024-04-20T01:47:50.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.800861+0000", 2024-04-20T01:47:50.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9796999999999995e-05 2024-04-20T01:47:50.297 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.297 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.297 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.297 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.0s2 38/30 ECSubWrite(tid=23, reqid=osd.3.0:22, at_version=0'0, trim_to=0'0, roll_forward_to=38'12))", 2024-04-20T01:47:50.298 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:39.788115+0000", 2024-04-20T01:47:50.298 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.512837282, 2024-04-20T01:47:50.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002677792, 2024-04-20T01:47:50.298 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.298 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.298 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.788115+0000", 2024-04-20T01:47:50.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.788115+0000", 2024-04-20T01:47:50.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.299 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.299 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.788114+0000", 2024-04-20T01:47:50.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:50.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.299 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.299 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.788129+0000", 2024-04-20T01:47:50.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4934e-05 2024-04-20T01:47:50.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.299 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.299 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.788141+0000", 2024-04-20T01:47:50.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2415000000000001e-05 2024-04-20T01:47:50.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.300 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.300 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.788149+0000", 2024-04-20T01:47:50.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2450000000000001e-06 2024-04-20T01:47:50.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.300 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.300 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.788188+0000", 2024-04-20T01:47:50.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9201e-05 2024-04-20T01:47:50.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.788195+0000", 2024-04-20T01:47:50.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4749999999999996e-06 2024-04-20T01:47:50.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.790766+0000", 2024-04-20T01:47:50.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025703570000000001 2024-04-20T01:47:50.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.790793+0000", 2024-04-20T01:47:50.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6988999999999999e-05 2024-04-20T01:47:50.302 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.302 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.302 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.302 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 42/30 ECSubWrite(tid=226, reqid=client.4643.0:299, at_version=0'0, trim_to=0'0, roll_forward_to=42'26))", 2024-04-20T01:47:50.302 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.819820+0000", 2024-04-20T01:47:50.302 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.4811318599999996, 2024-04-20T01:47:50.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032324340000000002, 2024-04-20T01:47:50.302 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.302 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.302 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.819820+0000", 2024-04-20T01:47:50.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.819820+0000", 2024-04-20T01:47:50.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.819820+0000", 2024-04-20T01:47:50.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:50.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.819831+0000", 2024-04-20T01:47:50.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.102e-05 2024-04-20T01:47:50.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.819846+0000", 2024-04-20T01:47:50.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5739e-05 2024-04-20T01:47:50.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.819854+0000", 2024-04-20T01:47:50.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9160000000000001e-06 2024-04-20T01:47:50.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.819914+0000", 2024-04-20T01:47:50.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9987e-05 2024-04-20T01:47:50.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.819925+0000", 2024-04-20T01:47:50.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0567000000000001e-05 2024-04-20T01:47:50.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.823022+0000", 2024-04-20T01:47:50.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030975769999999998 2024-04-20T01:47:50.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.823053+0000", 2024-04-20T01:47:50.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0348000000000001e-05 2024-04-20T01:47:50.306 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.306 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.306 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.307 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 42/30 ECSubWrite(tid=227, reqid=client.4643.0:300, at_version=42'27, trim_to=0'0, roll_forward_to=42'26))", 2024-04-20T01:47:50.307 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.820573+0000", 2024-04-20T01:47:50.307 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.4803790049999996, 2024-04-20T01:47:50.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028957610000000002, 2024-04-20T01:47:50.307 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.307 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.307 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.820573+0000", 2024-04-20T01:47:50.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.820573+0000", 2024-04-20T01:47:50.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.820572+0000", 2024-04-20T01:47:50.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:50.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.820590+0000", 2024-04-20T01:47:50.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7805999999999999e-05 2024-04-20T01:47:50.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.820602+0000", 2024-04-20T01:47:50.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1643e-05 2024-04-20T01:47:50.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.820608+0000", 2024-04-20T01:47:50.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7499999999999997e-06 2024-04-20T01:47:50.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.820663+0000", 2024-04-20T01:47:50.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4116000000000002e-05 2024-04-20T01:47:50.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.820675+0000", 2024-04-20T01:47:50.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.242e-05 2024-04-20T01:47:50.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.823439+0000", 2024-04-20T01:47:50.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027643360000000001 2024-04-20T01:47:50.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.823469+0000", 2024-04-20T01:47:50.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9575000000000001e-05 2024-04-20T01:47:50.311 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.311 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.311 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.311 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 42/30 ECSubWrite(tid=229, reqid=client.4643.0:302, at_version=42'29, trim_to=0'0, roll_forward_to=42'27))", 2024-04-20T01:47:50.311 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.829719+0000", 2024-04-20T01:47:50.311 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.4712332730000002, 2024-04-20T01:47:50.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045205979999999998, 2024-04-20T01:47:50.311 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.311 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.311 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.829719+0000", 2024-04-20T01:47:50.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.829719+0000", 2024-04-20T01:47:50.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.829718+0000", 2024-04-20T01:47:50.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:50.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830290+0000", 2024-04-20T01:47:50.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00057253800000000004 2024-04-20T01:47:50.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830321+0000", 2024-04-20T01:47:50.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0368000000000001e-05 2024-04-20T01:47:50.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830349+0000", 2024-04-20T01:47:50.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8342000000000001e-05 2024-04-20T01:47:50.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830435+0000", 2024-04-20T01:47:50.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6215e-05 2024-04-20T01:47:50.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830446+0000", 2024-04-20T01:47:50.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0883999999999999e-05 2024-04-20T01:47:50.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.834215+0000", 2024-04-20T01:47:50.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003768815 2024-04-20T01:47:50.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.834239+0000", 2024-04-20T01:47:50.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4433000000000001e-05 2024-04-20T01:47:50.315 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.315 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.315 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.315 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.cs2 42/30 ECSubWrite(tid=28, reqid=client.4643.0:327, at_version=42'23, trim_to=0'0, roll_forward_to=41'19))", 2024-04-20T01:47:50.315 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.955477+0000", 2024-04-20T01:47:50.315 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.3454747469999999, 2024-04-20T01:47:50.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044838229999999996, 2024-04-20T01:47:50.316 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.316 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.316 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.316 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.955477+0000", 2024-04-20T01:47:50.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.955477+0000", 2024-04-20T01:47:50.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.955476+0000", 2024-04-20T01:47:50.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:50.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.955553+0000", 2024-04-20T01:47:50.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6977000000000005e-05 2024-04-20T01:47:50.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.955566+0000", 2024-04-20T01:47:50.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2724999999999999e-05 2024-04-20T01:47:50.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.955576+0000", 2024-04-20T01:47:50.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5910000000000002e-06 2024-04-20T01:47:50.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.955618+0000", 2024-04-20T01:47:50.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2230000000000001e-05 2024-04-20T01:47:50.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.955628+0000", 2024-04-20T01:47:50.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.815e-06 2024-04-20T01:47:50.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.959940+0000", 2024-04-20T01:47:50.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043127019999999999 2024-04-20T01:47:50.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.959961+0000", 2024-04-20T01:47:50.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0869999999999998e-05 2024-04-20T01:47:50.320 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.320 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.320 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.320 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.6s4 42/30 ECSubWrite(tid=8, reqid=client.4643.0:331, at_version=42'4, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:47:50.320 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.957689+0000", 2024-04-20T01:47:50.320 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.3432626650000001, 2024-04-20T01:47:50.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004764936, 2024-04-20T01:47:50.321 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.321 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.321 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.321 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957689+0000", 2024-04-20T01:47:50.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.321 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957689+0000", 2024-04-20T01:47:50.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957689+0000", 2024-04-20T01:47:50.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:47:50.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957705+0000", 2024-04-20T01:47:50.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5418999999999999e-05 2024-04-20T01:47:50.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957716+0000", 2024-04-20T01:47:50.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0815999999999999e-05 2024-04-20T01:47:50.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957727+0000", 2024-04-20T01:47:50.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.132e-05 2024-04-20T01:47:50.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957787+0000", 2024-04-20T01:47:50.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9734000000000001e-05 2024-04-20T01:47:50.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957799+0000", 2024-04-20T01:47:50.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.27e-05 2024-04-20T01:47:50.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.962434+0000", 2024-04-20T01:47:50.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046348989999999996 2024-04-20T01:47:50.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.962454+0000", 2024-04-20T01:47:50.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0196000000000001e-05 2024-04-20T01:47:50.326 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.326 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.326 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.327 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.cs2 42/30 ECSubWrite(tid=29, reqid=client.4643.0:328, at_version=42'24, trim_to=0'0, roll_forward_to=41'19))", 2024-04-20T01:47:50.327 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.957897+0000", 2024-04-20T01:47:50.327 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.3430550290000003, 2024-04-20T01:47:50.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045531, 2024-04-20T01:47:50.327 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.327 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.327 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957897+0000", 2024-04-20T01:47:50.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957897+0000", 2024-04-20T01:47:50.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957896+0000", 2024-04-20T01:47:50.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:50.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957916+0000", 2024-04-20T01:47:50.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9996e-05 2024-04-20T01:47:50.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957928+0000", 2024-04-20T01:47:50.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1773000000000001e-05 2024-04-20T01:47:50.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957952+0000", 2024-04-20T01:47:50.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3706e-05 2024-04-20T01:47:50.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957995+0000", 2024-04-20T01:47:50.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3075999999999999e-05 2024-04-20T01:47:50.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.958004+0000", 2024-04-20T01:47:50.330 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7280000000000001e-06 2024-04-20T01:47:50.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.962426+0000", 2024-04-20T01:47:50.330 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004422863 2024-04-20T01:47:50.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.962450+0000", 2024-04-20T01:47:50.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3740999999999999e-05 2024-04-20T01:47:50.331 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.331 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.331 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.331 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.6s4 42/30 ECSubWrite(tid=9, reqid=client.4643.0:331, at_version=0'0, trim_to=0'0, roll_forward_to=42'4))", 2024-04-20T01:47:50.331 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.963142+0000", 2024-04-20T01:47:50.331 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.3378102910000003, 2024-04-20T01:47:50.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025837099999999999, 2024-04-20T01:47:50.331 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.331 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.331 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.963142+0000", 2024-04-20T01:47:50.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.963142+0000", 2024-04-20T01:47:50.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.963141+0000", 2024-04-20T01:47:50.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:50.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.333 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.333 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.963160+0000", 2024-04-20T01:47:50.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8580000000000002e-05 2024-04-20T01:47:50.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.333 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.333 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.963170+0000", 2024-04-20T01:47:50.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0747999999999999e-05 2024-04-20T01:47:50.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.333 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.333 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.963178+0000", 2024-04-20T01:47:50.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9680000000000006e-06 2024-04-20T01:47:50.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.963213+0000", 2024-04-20T01:47:50.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4646e-05 2024-04-20T01:47:50.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.963219+0000", 2024-04-20T01:47:50.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0009999999999998e-06 2024-04-20T01:47:50.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.965707+0000", 2024-04-20T01:47:50.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024875470000000001 2024-04-20T01:47:50.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.965726+0000", 2024-04-20T01:47:50.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8957000000000002e-05 2024-04-20T01:47:50.335 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.335 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.335 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.335 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 47/30 ECSubWrite(tid=247, reqid=client.4643.0:389, at_version=47'39, trim_to=0'0, roll_forward_to=47'35))", 2024-04-20T01:47:50.335 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:48.746866+0000", 2024-04-20T01:47:50.335 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.5540864480000001, 2024-04-20T01:47:50.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027125830000000002, 2024-04-20T01:47:50.336 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.336 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.336 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746866+0000", 2024-04-20T01:47:50.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746866+0000", 2024-04-20T01:47:50.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746865+0000", 2024-04-20T01:47:50.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:50.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747124+0000", 2024-04-20T01:47:50.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000258972 2024-04-20T01:47:50.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747138+0000", 2024-04-20T01:47:50.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3799e-05 2024-04-20T01:47:50.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.338 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747148+0000", 2024-04-20T01:47:50.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0009e-05 2024-04-20T01:47:50.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.338 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747195+0000", 2024-04-20T01:47:50.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7522999999999999e-05 2024-04-20T01:47:50.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.338 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747208+0000", 2024-04-20T01:47:50.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2446000000000001e-05 2024-04-20T01:47:50.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.749553+0000", 2024-04-20T01:47:50.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023451000000000001 2024-04-20T01:47:50.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.749578+0000", 2024-04-20T01:47:50.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5473000000000002e-05 2024-04-20T01:47:50.339 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.339 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.339 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.340 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 47/30 ECSubWrite(tid=248, reqid=client.4643.0:390, at_version=47'40, trim_to=0'0, roll_forward_to=47'35))", 2024-04-20T01:47:50.340 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:48.747744+0000", 2024-04-20T01:47:50.340 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.5532081199999999, 2024-04-20T01:47:50.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045665469999999998, 2024-04-20T01:47:50.340 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.340 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.340 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747744+0000", 2024-04-20T01:47:50.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747744+0000", 2024-04-20T01:47:50.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747743+0000", 2024-04-20T01:47:50.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:50.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747771+0000", 2024-04-20T01:47:50.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8138000000000001e-05 2024-04-20T01:47:50.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747786+0000", 2024-04-20T01:47:50.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4598e-05 2024-04-20T01:47:50.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747796+0000", 2024-04-20T01:47:50.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0582e-05 2024-04-20T01:47:50.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747870+0000", 2024-04-20T01:47:50.343 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3578000000000006e-05 2024-04-20T01:47:50.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.343 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.343 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747882+0000", 2024-04-20T01:47:50.343 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1755e-05 2024-04-20T01:47:50.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.343 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.343 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.752288+0000", 2024-04-20T01:47:50.343 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044065190000000002 2024-04-20T01:47:50.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.343 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.343 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.752311+0000", 2024-04-20T01:47:50.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2331e-05 2024-04-20T01:47:50.344 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.344 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.344 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.344 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 48/30 ECSubWrite(tid=252, reqid=osd.13.0:252, at_version=48'42, trim_to=0'0, roll_forward_to=47'40))", 2024-04-20T01:47:50.344 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.710283+0000", 2024-04-20T01:47:50.344 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.59066919200000001, 2024-04-20T01:47:50.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004154647, 2024-04-20T01:47:50.344 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.344 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.344 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710283+0000", 2024-04-20T01:47:50.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710283+0000", 2024-04-20T01:47:50.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710283+0000", 2024-04-20T01:47:50.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:50.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710309+0000", 2024-04-20T01:47:50.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6055999999999999e-05 2024-04-20T01:47:50.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710321+0000", 2024-04-20T01:47:50.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2259000000000001e-05 2024-04-20T01:47:50.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710331+0000", 2024-04-20T01:47:50.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0013000000000001e-05 2024-04-20T01:47:50.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710408+0000", 2024-04-20T01:47:50.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7187999999999993e-05 2024-04-20T01:47:50.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710422+0000", 2024-04-20T01:47:50.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3949e-05 2024-04-20T01:47:50.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.714415+0000", 2024-04-20T01:47:50.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039929329999999997 2024-04-20T01:47:50.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.348 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.348 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.714438+0000", 2024-04-20T01:47:50.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2558e-05 2024-04-20T01:47:50.348 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.348 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.348 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.348 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 48/30 ECSubWrite(tid=253, reqid=osd.13.0:253, at_version=48'44, trim_to=0'0, roll_forward_to=47'40))", 2024-04-20T01:47:50.348 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.711468+0000", 2024-04-20T01:47:50.348 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.58948451099999999, 2024-04-20T01:47:50.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056178349999999998, 2024-04-20T01:47:50.349 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.349 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.349 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711468+0000", 2024-04-20T01:47:50.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711468+0000", 2024-04-20T01:47:50.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711467+0000", 2024-04-20T01:47:50.350 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:50.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.350 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711491+0000", 2024-04-20T01:47:50.350 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4023e-05 2024-04-20T01:47:50.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.350 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711503+0000", 2024-04-20T01:47:50.350 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2065000000000001e-05 2024-04-20T01:47:50.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.350 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.351 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711513+0000", 2024-04-20T01:47:50.351 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8509999999999994e-06 2024-04-20T01:47:50.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.351 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.351 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.351 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711561+0000", 2024-04-20T01:47:50.351 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8208999999999999e-05 2024-04-20T01:47:50.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.351 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.351 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.351 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711573+0000", 2024-04-20T01:47:50.351 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1955e-05 2024-04-20T01:47:50.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.351 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.351 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.352 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717067+0000", 2024-04-20T01:47:50.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054938909999999999 2024-04-20T01:47:50.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.352 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.352 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717085+0000", 2024-04-20T01:47:50.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8660000000000001e-05 2024-04-20T01:47:50.352 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.352 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.352 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.352 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.cs2 48/30 ECSubWrite(tid=41, reqid=osd.14.0:41, at_version=48'35, trim_to=0'0, roll_forward_to=47'33))", 2024-04-20T01:47:50.352 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.716056+0000", 2024-04-20T01:47:50.352 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.58489611500000005, 2024-04-20T01:47:50.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040926080000000002, 2024-04-20T01:47:50.353 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.353 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.353 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716056+0000", 2024-04-20T01:47:50.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716056+0000", 2024-04-20T01:47:50.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716056+0000", 2024-04-20T01:47:50.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:47:50.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716076+0000", 2024-04-20T01:47:50.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9726000000000001e-05 2024-04-20T01:47:50.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716086+0000", 2024-04-20T01:47:50.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0481e-05 2024-04-20T01:47:50.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716095+0000", 2024-04-20T01:47:50.355 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1069999999999994e-06 2024-04-20T01:47:50.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716164+0000", 2024-04-20T01:47:50.355 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9157000000000002e-05 2024-04-20T01:47:50.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716174+0000", 2024-04-20T01:47:50.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.465e-06 2024-04-20T01:47:50.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.356 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.356 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.720126+0000", 2024-04-20T01:47:50.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039519159999999998 2024-04-20T01:47:50.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.356 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.356 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.720149+0000", 2024-04-20T01:47:50.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2974000000000001e-05 2024-04-20T01:47:50.356 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.356 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.356 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.357 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.357 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.1s4 48/30 ECSubWrite(tid=119, reqid=osd.4.0:119, at_version=48'14, trim_to=0'0, roll_forward_to=47'12))", 2024-04-20T01:47:50.357 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.717339+0000", 2024-04-20T01:47:50.357 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.58361355299999995, 2024-04-20T01:47:50.357 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0079470240000000004, 2024-04-20T01:47:50.357 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.357 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.357 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.357 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.357 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.357 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717339+0000", 2024-04-20T01:47:50.357 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.358 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.358 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717339+0000", 2024-04-20T01:47:50.358 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.358 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.358 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717338+0000", 2024-04-20T01:47:50.358 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:50.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.358 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.358 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717357+0000", 2024-04-20T01:47:50.358 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8575000000000001e-05 2024-04-20T01:47:50.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717368+0000", 2024-04-20T01:47:50.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.145e-05 2024-04-20T01:47:50.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717377+0000", 2024-04-20T01:47:50.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4929999999999998e-06 2024-04-20T01:47:50.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717424+0000", 2024-04-20T01:47:50.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7240000000000002e-05 2024-04-20T01:47:50.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717435+0000", 2024-04-20T01:47:50.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0533000000000001e-05 2024-04-20T01:47:50.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.725244+0000", 2024-04-20T01:47:50.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0078092129999999997 2024-04-20T01:47:50.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.361 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.725286+0000", 2024-04-20T01:47:50.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1792999999999998e-05 2024-04-20T01:47:50.361 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.361 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.361 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.361 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.361 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 48/30 ECSubWrite(tid=255, reqid=osd.13.0:253, at_version=0'0, trim_to=0'0, roll_forward_to=48'44))", 2024-04-20T01:47:50.361 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.721176+0000", 2024-04-20T01:47:50.361 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.57977662500000005, 2024-04-20T01:47:50.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041402469999999997, 2024-04-20T01:47:50.361 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.361 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.361 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.721176+0000", 2024-04-20T01:47:50.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.721176+0000", 2024-04-20T01:47:50.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.721175+0000", 2024-04-20T01:47:50.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:50.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.721188+0000", 2024-04-20T01:47:50.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2887999999999999e-05 2024-04-20T01:47:50.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.721197+0000", 2024-04-20T01:47:50.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7249999999999992e-06 2024-04-20T01:47:50.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.721204+0000", 2024-04-20T01:47:50.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0909999999999998e-06 2024-04-20T01:47:50.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.721241+0000", 2024-04-20T01:47:50.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7264000000000001e-05 2024-04-20T01:47:50.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.721247+0000", 2024-04-20T01:47:50.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9039999999999997e-06 2024-04-20T01:47:50.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.365 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.725301+0000", 2024-04-20T01:47:50.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040542779999999997 2024-04-20T01:47:50.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.365 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.365 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.725316+0000", 2024-04-20T01:47:50.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4746000000000001e-05 2024-04-20T01:47:50.365 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.365 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.365 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.365 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 48/30 ECSubWrite(tid=256, reqid=client.4643.0:403, at_version=48'46, trim_to=0'0, roll_forward_to=48'44))", 2024-04-20T01:47:50.365 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.789490+0000", 2024-04-20T01:47:50.365 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.51146214899999998, 2024-04-20T01:47:50.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005092793, 2024-04-20T01:47:50.366 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.366 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.366 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.789490+0000", 2024-04-20T01:47:50.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.789490+0000", 2024-04-20T01:47:50.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.789489+0000", 2024-04-20T01:47:50.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:50.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.789916+0000", 2024-04-20T01:47:50.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00042631400000000001 2024-04-20T01:47:50.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.789930+0000", 2024-04-20T01:47:50.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4201e-05 2024-04-20T01:47:50.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.789938+0000", 2024-04-20T01:47:50.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2560000000000002e-06 2024-04-20T01:47:50.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.790005+0000", 2024-04-20T01:47:50.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.648e-05 2024-04-20T01:47:50.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.790029+0000", 2024-04-20T01:47:50.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.444e-05 2024-04-20T01:47:50.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.794559+0000", 2024-04-20T01:47:50.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045300560000000002 2024-04-20T01:47:50.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.794583+0000", 2024-04-20T01:47:50.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3666e-05 2024-04-20T01:47:50.369 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.369 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.369 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.370 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 48/30 ECSubWrite(tid=257, reqid=client.4643.0:404, at_version=48'47, trim_to=0'0, roll_forward_to=48'44))", 2024-04-20T01:47:50.370 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.791782+0000", 2024-04-20T01:47:50.370 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.50916987700000005, 2024-04-20T01:47:50.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00373625, 2024-04-20T01:47:50.370 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.370 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.370 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.370 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.791782+0000", 2024-04-20T01:47:50.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.791782+0000", 2024-04-20T01:47:50.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.791781+0000", 2024-04-20T01:47:50.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:50.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.792218+0000", 2024-04-20T01:47:50.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043652600000000001 2024-04-20T01:47:50.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.792230+0000", 2024-04-20T01:47:50.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1603999999999999e-05 2024-04-20T01:47:50.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.792236+0000", 2024-04-20T01:47:50.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6209999999999998e-06 2024-04-20T01:47:50.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.792278+0000", 2024-04-20T01:47:50.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2156000000000001e-05 2024-04-20T01:47:50.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.792289+0000", 2024-04-20T01:47:50.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0552e-05 2024-04-20T01:47:50.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.795502+0000", 2024-04-20T01:47:50.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003213526 2024-04-20T01:47:50.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.795519+0000", 2024-04-20T01:47:50.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6042e-05 2024-04-20T01:47:50.373 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.373 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.373 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.374 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 48/30 ECSubWrite(tid=258, reqid=client.4643.0:405, at_version=48'48, trim_to=0'0, roll_forward_to=48'44))", 2024-04-20T01:47:50.374 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.792778+0000", 2024-04-20T01:47:50.374 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.50817445500000002, 2024-04-20T01:47:50.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027568620000000001, 2024-04-20T01:47:50.374 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.374 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.374 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.792778+0000", 2024-04-20T01:47:50.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.792778+0000", 2024-04-20T01:47:50.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.792777+0000", 2024-04-20T01:47:50.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:50.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.792890+0000", 2024-04-20T01:47:50.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112587 2024-04-20T01:47:50.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.792902+0000", 2024-04-20T01:47:50.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2778e-05 2024-04-20T01:47:50.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.792910+0000", 2024-04-20T01:47:50.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8690000000000004e-06 2024-04-20T01:47:50.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.793959+0000", 2024-04-20T01:47:50.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001048743 2024-04-20T01:47:50.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.793969+0000", 2024-04-20T01:47:50.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6889999999999999e-06 2024-04-20T01:47:50.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.795523+0000", 2024-04-20T01:47:50.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001553844 2024-04-20T01:47:50.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.795535+0000", 2024-04-20T01:47:50.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1925e-05 2024-04-20T01:47:50.377 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.377 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.377 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.377 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.378 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.378 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:50.379 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-20T01:47:50.384 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:50.399+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:50.485 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:50.500+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:50.507 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:50.507 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:47:50.507 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:47:50.507 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:50.517 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-20T01:47:50.557 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:47:50.559 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T01:47:50.572 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2024-04-20T01:47:49.706398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":38732320,"num_objects":65,"num_object_clones":18,"num_object_copies":445,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":4,"num_read":360,"num_read_kb":30398,"num_write":592,"num_write_kb":46372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":463,"ondisk_log_size":463,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":501520,"kb_used_data":69092,"kb_used_omap":24,"kb_used_meta":432103,"kb_avail":1509447920,"statfs":{"total":1546188226560,"available":1545674670080,"internally_reserved":0,"allocated":70750208,"data_stored":76816481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25507,"internal_metadata":442473565},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8830976,"num_objects":-7,"num_object_clones":-2,"num_object_copies":-49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7,"num_whiteouts":-3,"num_read":47,"num_read_kb":12161,"num_write":188,"num_write_kb":13296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.891675"},"pg_stats":[{"pgid":"3.f","version":"41'12","reported_seq":68,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.807655+0000","last_change":"2024-04-20T01:47:42.818507+0000","last_active":"2024-04-20T01:47:44.807655+0000","last_peered":"2024-04-20T01:47:44.807655+0000","last_clean":"2024-04-20T01:47:44.807655+0000","last_became_active":"2024-04-20T01:47:32.867228+0000","last_became_peered":"2024-04-20T01:47:32.867228+0000","last_unstale":"2024-04-20T01:47:44.807655+0000","last_undegraded":"2024-04-20T01:47:44.807655+0000","last_fullsized":"2024-04-20T01:47:44.807655+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.328818+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015122383,"stat_sum":{"num_bytes":1228800,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":5142,"num_write":47,"num_write_kb":1488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,15,13,8,11,0],"acting":[2,3,15,13,8,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"3.e","version":"42'21","reported_seq":76,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.821067+0000","last_change":"2024-04-20T01:47:42.822202+0000","last_active":"2024-04-20T01:47:45.821067+0000","last_peered":"2024-04-20T01:47:45.821067+0000","last_clean":"2024-04-20T01:47:45.821067+0000","last_became_active":"2024-04-20T01:47:32.867409+0000","last_became_peered":"2024-04-20T01:47:32.867409+0000","last_unstale":"2024-04-20T01:47:45.821067+0000","last_undegraded":"2024-04-20T01:47:45.821067+0000","last_fullsized":"2024-04-20T01:47:45.821067+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:42.581217+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012353486,"stat_sum":{"num_bytes":1605632,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":24,"num_read_kb":1735,"num_write":13,"num_write_kb":3280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,1,10,8,3,12],"acting":[11,9,1,10,8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"3.d","version":"41'12","reported_seq":55,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.816124+0000","last_change":"2024-04-20T01:47:42.815568+0000","last_active":"2024-04-20T01:47:44.816124+0000","last_peered":"2024-04-20T01:47:44.816124+0000","last_clean":"2024-04-20T01:47:44.816124+0000","last_became_active":"2024-04-20T01:47:32.867954+0000","last_became_peered":"2024-04-20T01:47:32.867954+0000","last_unstale":"2024-04-20T01:47:44.816124+0000","last_undegraded":"2024-04-20T01:47:44.816124+0000","last_fullsized":"2024-04-20T01:47:44.816124+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.083554+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011117459999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,11,15,7,8,4],"acting":[6,2,11,15,7,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"3.c","version":"42'24","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.816615+0000","last_change":"2024-04-20T01:47:42.812324+0000","last_active":"2024-04-20T01:47:44.816615+0000","last_peered":"2024-04-20T01:47:44.816615+0000","last_clean":"2024-04-20T01:47:44.816615+0000","last_became_active":"2024-04-20T01:47:32.867926+0000","last_became_peered":"2024-04-20T01:47:32.867926+0000","last_unstale":"2024-04-20T01:47:44.816615+0000","last_undegraded":"2024-04-20T01:47:44.816615+0000","last_fullsized":"2024-04-20T01:47:44.816615+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:54.374528+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012044350000000001,"stat_sum":{"num_bytes":4292608,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":16,"num_write_kb":4192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,8,6,3,7,12,5],"acting":[14,8,6,3,7,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"3.b","version":"44'43","reported_seq":105,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.868627+0000","last_change":"2024-04-20T01:47:42.837347+0000","last_active":"2024-04-20T01:47:45.868627+0000","last_peered":"2024-04-20T01:47:45.868627+0000","last_clean":"2024-04-20T01:47:45.868627+0000","last_became_active":"2024-04-20T01:47:32.867309+0000","last_became_peered":"2024-04-20T01:47:32.867309+0000","last_unstale":"2024-04-20T01:47:45.868627+0000","last_undegraded":"2024-04-20T01:47:45.868627+0000","last_fullsized":"2024-04-20T01:47:45.868627+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:01.227735+0000","scrub_duration":0,"objects_trimmed":5,"snaptrim_duration":0.024423878999999999,"stat_sum":{"num_bytes":6799360,"num_objects":8,"num_object_clones":2,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":35,"num_read_kb":5390,"num_write":87,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,11,10,9],"acting":[4,0,1,12,11,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"3.a","version":"44'10","reported_seq":56,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.839665+0000","last_change":"2024-04-20T01:47:42.819658+0000","last_active":"2024-04-20T01:47:45.839665+0000","last_peered":"2024-04-20T01:47:45.839665+0000","last_clean":"2024-04-20T01:47:45.839665+0000","last_became_active":"2024-04-20T01:47:32.868002+0000","last_became_peered":"2024-04-20T01:47:32.868002+0000","last_unstale":"2024-04-20T01:47:45.839665+0000","last_undegraded":"2024-04-20T01:47:45.839665+0000","last_fullsized":"2024-04-20T01:47:45.839665+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:35.716198+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.012917052,"stat_sum":{"num_bytes":1736704,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":8,"num_read_kb":1971,"num_write":6,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,1,14,3,15,2],"acting":[6,10,1,14,3,15,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"3.9","version":"42'6","reported_seq":52,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.815818+0000","last_change":"2024-04-20T01:47:42.807106+0000","last_active":"2024-04-20T01:47:44.815818+0000","last_peered":"2024-04-20T01:47:44.815818+0000","last_clean":"2024-04-20T01:47:44.815818+0000","last_became_active":"2024-04-20T01:47:32.866706+0000","last_became_peered":"2024-04-20T01:47:32.866706+0000","last_unstale":"2024-04-20T01:47:44.815818+0000","last_undegraded":"2024-04-20T01:47:44.815818+0000","last_fullsized":"2024-04-20T01:47:44.815818+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:42.776206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056749800000000005,"stat_sum":{"num_bytes":507904,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":5,"num_write_kb":496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0,11,14,8,3],"acting":[7,1,0,11,14,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"3.8","version":"42'20","reported_seq":71,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.816129+0000","last_change":"2024-04-20T01:47:42.804832+0000","last_active":"2024-04-20T01:47:44.816129+0000","last_peered":"2024-04-20T01:47:44.816129+0000","last_clean":"2024-04-20T01:47:44.816129+0000","last_became_active":"2024-04-20T01:47:32.718069+0000","last_became_peered":"2024-04-20T01:47:32.718069+0000","last_unstale":"2024-04-20T01:47:44.816129+0000","last_undegraded":"2024-04-20T01:47:44.816129+0000","last_fullsized":"2024-04-20T01:47:44.816129+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:58.489885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053838200000000001,"stat_sum":{"num_bytes":3997696,"num_objects":7,"num_object_clones":3,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":535,"num_write":17,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,11,3,14,2,7],"acting":[5,0,11,3,14,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.806979+0000","last_change":"2024-04-20T01:47:20.071988+0000","last_active":"2024-04-20T01:47:44.806979+0000","last_peered":"2024-04-20T01:47:44.806979+0000","last_clean":"2024-04-20T01:47:44.806979+0000","last_became_active":"2024-04-20T01:47:20.071615+0000","last_became_peered":"2024-04-20T01:47:20.071615+0000","last_unstale":"2024-04-20T01:47:44.806979+0000","last_undegraded":"2024-04-20T01:47:44.806979+0000","last_fullsized":"2024-04-20T01:47:44.806979+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:40.106777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"41'5","reported_seq":50,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.816765+0000","last_change":"2024-04-20T01:47:42.809817+0000","last_active":"2024-04-20T01:47:44.816765+0000","last_peered":"2024-04-20T01:47:44.816765+0000","last_clean":"2024-04-20T01:47:44.816765+0000","last_became_active":"2024-04-20T01:47:32.867180+0000","last_became_peered":"2024-04-20T01:47:32.867180+0000","last_unstale":"2024-04-20T01:47:44.816765+0000","last_undegraded":"2024-04-20T01:47:44.816765+0000","last_fullsized":"2024-04-20T01:47:44.816765+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:43.358783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003779077,"stat_sum":{"num_bytes":360448,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":5,"num_write_kb":416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,4,14,12,0,11],"acting":[10,8,4,14,12,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.807531+0000","last_change":"2024-04-20T01:47:20.075613+0000","last_active":"2024-04-20T01:47:44.807531+0000","last_peered":"2024-04-20T01:47:44.807531+0000","last_clean":"2024-04-20T01:47:44.807531+0000","last_became_active":"2024-04-20T01:47:20.075314+0000","last_became_peered":"2024-04-20T01:47:20.075314+0000","last_unstale":"2024-04-20T01:47:44.807531+0000","last_undegraded":"2024-04-20T01:47:44.807531+0000","last_fullsized":"2024-04-20T01:47:44.807531+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:35.956173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,12],"acting":[2,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"44'18","reported_seq":66,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.845417+0000","last_change":"2024-04-20T01:47:42.809609+0000","last_active":"2024-04-20T01:47:45.845417+0000","last_peered":"2024-04-20T01:47:45.845417+0000","last_clean":"2024-04-20T01:47:45.845417+0000","last_became_active":"2024-04-20T01:47:32.719136+0000","last_became_peered":"2024-04-20T01:47:32.719136+0000","last_unstale":"2024-04-20T01:47:45.845417+0000","last_undegraded":"2024-04-20T01:47:45.845417+0000","last_fullsized":"2024-04-20T01:47:45.845417+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:46.940575+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0089564480000000005,"stat_sum":{"num_bytes":1540096,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":3,"num_write":30,"num_write_kb":2128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,4,14,11,5,10],"acting":[13,2,4,14,11,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.829483+0000","last_change":"2024-04-20T01:47:20.039221+0000","last_active":"2024-04-20T01:47:45.829483+0000","last_peered":"2024-04-20T01:47:45.829483+0000","last_clean":"2024-04-20T01:47:45.829483+0000","last_became_active":"2024-04-20T01:47:20.038972+0000","last_became_peered":"2024-04-20T01:47:20.038972+0000","last_unstale":"2024-04-20T01:47:45.829483+0000","last_undegraded":"2024-04-20T01:47:45.829483+0000","last_fullsized":"2024-04-20T01:47:45.829483+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:24.167832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"44'17","reported_seq":67,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.828308+0000","last_change":"2024-04-20T01:47:42.819924+0000","last_active":"2024-04-20T01:47:45.828308+0000","last_peered":"2024-04-20T01:47:45.828308+0000","last_clean":"2024-04-20T01:47:45.828308+0000","last_became_active":"2024-04-20T01:47:32.868365+0000","last_became_peered":"2024-04-20T01:47:32.868365+0000","last_unstale":"2024-04-20T01:47:45.828308+0000","last_undegraded":"2024-04-20T01:47:45.828308+0000","last_fullsized":"2024-04-20T01:47:45.828308+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:41.198793+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.015482437999999999,"stat_sum":{"num_bytes":1343488,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":14,"num_read_kb":1685,"num_write":29,"num_write_kb":1312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,1,5,15,4],"acting":[3,7,6,1,5,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.816422+0000","last_change":"2024-04-20T01:47:20.039014+0000","last_active":"2024-04-20T01:47:44.816422+0000","last_peered":"2024-04-20T01:47:44.816422+0000","last_clean":"2024-04-20T01:47:44.816422+0000","last_became_active":"2024-04-20T01:47:20.038764+0000","last_became_peered":"2024-04-20T01:47:20.038764+0000","last_unstale":"2024-04-20T01:47:44.816422+0000","last_undegraded":"2024-04-20T01:47:44.816422+0000","last_fullsized":"2024-04-20T01:47:44.816422+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:26.112165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"25'157","reported_seq":225,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.829609+0000","last_change":"2024-04-20T01:47:17.303029+0000","last_active":"2024-04-20T01:47:45.829609+0000","last_peered":"2024-04-20T01:47:45.829609+0000","last_clean":"2024-04-20T01:47:45.829609+0000","last_became_active":"2024-04-20T01:47:17.302105+0000","last_became_peered":"2024-04-20T01:47:17.302105+0000","last_unstale":"2024-04-20T01:47:45.829609+0000","last_undegraded":"2024-04-20T01:47:45.829609+0000","last_fullsized":"2024-04-20T01:47:45.829609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_clean_scrub_stamp":"2024-04-20T01:47:15.986754+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:30.481506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,15],"acting":[13,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.2","version":"44'31","reported_seq":92,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.844195+0000","last_change":"2024-04-20T01:47:42.813164+0000","last_active":"2024-04-20T01:47:45.844195+0000","last_peered":"2024-04-20T01:47:45.844195+0000","last_clean":"2024-04-20T01:47:45.844195+0000","last_became_active":"2024-04-20T01:47:32.866509+0000","last_became_peered":"2024-04-20T01:47:32.866509+0000","last_unstale":"2024-04-20T01:47:45.844195+0000","last_undegraded":"2024-04-20T01:47:45.844195+0000","last_fullsized":"2024-04-20T01:47:45.844195+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.395078+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012506468999999999,"stat_sum":{"num_bytes":4505600,"num_objects":9,"num_object_clones":3,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":32,"num_read_kb":8603,"num_write":40,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,6,10,14,1,7],"acting":[13,3,6,10,14,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.807472+0000","last_change":"2024-04-20T01:47:20.069279+0000","last_active":"2024-04-20T01:47:44.807472+0000","last_peered":"2024-04-20T01:47:44.807472+0000","last_clean":"2024-04-20T01:47:44.807472+0000","last_became_active":"2024-04-20T01:47:20.069005+0000","last_became_peered":"2024-04-20T01:47:20.069005+0000","last_unstale":"2024-04-20T01:47:44.807472+0000","last_undegraded":"2024-04-20T01:47:44.807472+0000","last_fullsized":"2024-04-20T01:47:44.807472+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:22.648743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"41'10","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.807023+0000","last_change":"2024-04-20T01:47:42.820323+0000","last_active":"2024-04-20T01:47:44.807023+0000","last_peered":"2024-04-20T01:47:44.807023+0000","last_clean":"2024-04-20T01:47:44.807023+0000","last_became_active":"2024-04-20T01:47:32.722189+0000","last_became_peered":"2024-04-20T01:47:32.722189+0000","last_unstale":"2024-04-20T01:47:44.807023+0000","last_undegraded":"2024-04-20T01:47:44.807023+0000","last_fullsized":"2024-04-20T01:47:44.807023+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:43.443403+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.007645771,"stat_sum":{"num_bytes":344064,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":1011,"num_write":5,"num_write_kb":768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,14,9,5,6,3,13],"acting":[4,14,9,5,6,3,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.816006+0000","last_change":"2024-04-20T01:47:20.038409+0000","last_active":"2024-04-20T01:47:44.816006+0000","last_peered":"2024-04-20T01:47:44.816006+0000","last_clean":"2024-04-20T01:47:44.816006+0000","last_became_active":"2024-04-20T01:47:20.038107+0000","last_became_peered":"2024-04-20T01:47:20.038107+0000","last_unstale":"2024-04-20T01:47:44.816006+0000","last_undegraded":"2024-04-20T01:47:44.816006+0000","last_fullsized":"2024-04-20T01:47:44.816006+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:44.125577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"44'61","reported_seq":113,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.892029+0000","last_change":"2024-04-20T01:47:42.832597+0000","last_active":"2024-04-20T01:47:45.892029+0000","last_peered":"2024-04-20T01:47:45.892029+0000","last_clean":"2024-04-20T01:47:45.892029+0000","last_became_active":"2024-04-20T01:47:32.867693+0000","last_became_peered":"2024-04-20T01:47:32.867693+0000","last_unstale":"2024-04-20T01:47:45.892029+0000","last_undegraded":"2024-04-20T01:47:45.892029+0000","last_fullsized":"2024-04-20T01:47:45.892029+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:31.824121+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.032078015000000001,"stat_sum":{"num_bytes":7225344,"num_objects":9,"num_object_clones":3,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":28,"num_read_kb":3805,"num_write":52,"num_write_kb":7840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,15,4,1,0,3,13],"acting":[8,15,4,1,0,3,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"3.7","version":"41'12","reported_seq":57,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.806971+0000","last_change":"2024-04-20T01:47:42.826203+0000","last_active":"2024-04-20T01:47:44.806971+0000","last_peered":"2024-04-20T01:47:44.806971+0000","last_clean":"2024-04-20T01:47:44.806971+0000","last_became_active":"2024-04-20T01:47:32.722002+0000","last_became_peered":"2024-04-20T01:47:32.722002+0000","last_unstale":"2024-04-20T01:47:44.806971+0000","last_undegraded":"2024-04-20T01:47:44.806971+0000","last_fullsized":"2024-04-20T01:47:44.806971+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:43.990113+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.013542391000000001,"stat_sum":{"num_bytes":2523136,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":28,"num_write_kb":2464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,0,8,5,14],"acting":[4,11,9,0,8,5,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.819716+0000","last_change":"2024-04-20T01:47:20.110600+0000","last_active":"2024-04-20T01:47:44.819716+0000","last_peered":"2024-04-20T01:47:44.819716+0000","last_clean":"2024-04-20T01:47:44.819716+0000","last_became_active":"2024-04-20T01:47:20.110260+0000","last_became_peered":"2024-04-20T01:47:20.110260+0000","last_unstale":"2024-04-20T01:47:44.819716+0000","last_undegraded":"2024-04-20T01:47:44.819716+0000","last_fullsized":"2024-04-20T01:47:44.819716+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:47.718290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.816246+0000","last_change":"2024-04-20T01:47:20.036634+0000","last_active":"2024-04-20T01:47:44.816246+0000","last_peered":"2024-04-20T01:47:44.816246+0000","last_clean":"2024-04-20T01:47:44.816246+0000","last_became_active":"2024-04-20T01:47:20.036328+0000","last_became_peered":"2024-04-20T01:47:20.036328+0000","last_unstale":"2024-04-20T01:47:44.816246+0000","last_undegraded":"2024-04-20T01:47:44.816246+0000","last_fullsized":"2024-04-20T01:47:44.816246+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.385250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,13],"acting":[6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"42'4","reported_seq":51,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-20T01:47:44.819220+0000","last_change":"2024-04-20T01:47:42.820437+0000","last_active":"2024-04-20T01:47:44.819220+0000","last_peered":"2024-04-20T01:47:44.819220+0000","last_clean":"2024-04-20T01:47:44.819220+0000","last_became_active":"2024-04-20T01:47:32.866834+0000","last_became_peered":"2024-04-20T01:47:32.866834+0000","last_unstale":"2024-04-20T01:47:44.819220+0000","last_undegraded":"2024-04-20T01:47:44.819220+0000","last_fullsized":"2024-04-20T01:47:44.819220+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:54.424527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001992705,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":7,"num_read_kb":386,"num_write":3,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,3,6,2,9],"acting":[0,13,1,3,6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":38141952,"num_objects":63,"num_object_clones":18,"num_object_copies":441,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":4,"num_read":224,"num_read_kb":30280,"num_write":389,"num_write_kb":42912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":66748416,"data_stored":83148800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":112,"acting":112,"num_store_stats":16},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280521,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":4724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":4837376,"data_stored":5903706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]}]},{"osd":15,"up_from":21,"seq":90194313225,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30244,"kb_used_data":3220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341596,"statfs":{"total":96636764160,"available":96605794304,"internally_reserved":0,"allocated":3297280,"data_stored":3344371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]}]},{"osd":14,"up_from":21,"seq":90194313226,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":5368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":5496832,"data_stored":5548889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,4,5,6,7,8,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]}]},{"osd":12,"up_from":21,"seq":90194313225,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30760,"kb_used_data":3740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341080,"statfs":{"total":96636764160,"available":96605265920,"internally_reserved":0,"allocated":3829760,"data_stored":4515221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]}]},{"osd":11,"up_from":21,"seq":90194313224,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31780,"kb_used_data":4752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340060,"statfs":{"total":96636764160,"available":96604221440,"internally_reserved":0,"allocated":4866048,"data_stored":5340596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,8,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":10,"up_from":21,"seq":90194313226,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":4260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":4362240,"data_stored":4935092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,4,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.915}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3049999999999997}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]}]},{"osd":9,"up_from":21,"seq":90194313226,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30032,"kb_used_data":3004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341808,"statfs":{"total":96636764160,"available":96606011392,"internally_reserved":0,"allocated":3076096,"data_stored":3513780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,4,6,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]},{"osd":8,"up_from":21,"seq":90194313225,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31260,"kb_used_data":4232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340580,"statfs":{"total":96636764160,"available":96604753920,"internally_reserved":0,"allocated":4333568,"data_stored":4525492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,6,7,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9630000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]}]},{"osd":7,"up_from":21,"seq":90194313226,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31140,"kb_used_data":4112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340700,"statfs":{"total":96636764160,"available":96604876800,"internally_reserved":0,"allocated":4210688,"data_stored":4259252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,3,4,6,8,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":6,"up_from":21,"seq":90194313224,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30580,"kb_used_data":3552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341260,"statfs":{"total":96636764160,"available":96605450240,"internally_reserved":0,"allocated":3637248,"data_stored":3648948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]}]},{"osd":1,"up_from":21,"seq":90194313225,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":5408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":5537792,"data_stored":6426036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]}]},{"osd":0,"up_from":21,"seq":90194313226,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":5140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":5263360,"data_stored":6056810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]}]},{"osd":2,"up_from":21,"seq":90194313224,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29672,"kb_used_data":2644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342168,"statfs":{"total":96636764160,"available":96606380032,"internally_reserved":0,"allocated":2707456,"data_stored":2473396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]}]},{"osd":3,"up_from":21,"seq":90194313226,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33500,"kb_used_data":6472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338340,"statfs":{"total":96636764160,"available":96602460160,"internally_reserved":0,"allocated":6627328,"data_stored":6991284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,4,5,6,7,8,9,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]}]},{"osd":4,"up_from":21,"seq":90194313226,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31528,"kb_used_data":4500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":4608000,"data_stored":5361076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,5,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]}]},{"osd":5,"up_from":21,"seq":90194313225,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30992,"kb_used_data":3964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340848,"statfs":{"total":96636764160,"available":96605028352,"internally_reserved":0,"allocated":4059136,"data_stored":3972532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5693440,"data_stored":7639040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5963776,"data_stored":8007680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2158592,"data_stored":2310144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6729728,"data_stored":8142848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5033984,"data_stored":7057408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3510272,"data_stored":3809280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3088384,"data_stored":3371008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3661824,"data_stored":3981312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4435968,"data_stored":5677056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":2850816,"data_stored":3665920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4136960,"data_stored":5201920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4640768,"data_stored":5607424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":3264512,"data_stored":4227072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":3743744,"data_stored":5087232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":4952064,"data_stored":5386240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":2883584,"data_stored":3977216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T01:47:50.575 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-20T01:47:50.575 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-20T01:47:50.575 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-04-20T01:47:50.575 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-20T01:47:50.585 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:50.600+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:50.646 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:50.646 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:47:50.646 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 30, 2024-04-20T01:47:50.646 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-20T01:47:50.646 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:50.656 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-20T01:47:50.686 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:47:50.700+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:47:50.695 INFO:tasks.rados.rados.0.smithi165.stdout:168: done (3 left) 2024-04-20T01:47:50.695 INFO:tasks.rados.rados.0.smithi165.stdout:169: done (2 left) 2024-04-20T01:47:50.695 INFO:tasks.rados.rados.0.smithi165.stdout:170: done (1 left) 2024-04-20T01:47:50.695 INFO:tasks.rados.rados.0.smithi165.stdout:173: done (0 left) 2024-04-20T01:47:50.695 INFO:tasks.rados.rados.0.smithi165.stdout:174: snap_remove snap 8 2024-04-20T01:47:50.697 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is unset 2024-04-20T01:47:50.786 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-20T01:47:50.788 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:50.802+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:50.828 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:50.828 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-20T01:47:50.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-20T01:47:50.828 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-20T01:47:50.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.828 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:10 3.5s0 3:a2e9ac6b:::smithi16579500-4:head [append 0~65536,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-20T01:47:50.829 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.752800+0000", 2024-04-20T01:47:50.829 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.088665894999998, 2024-04-20T01:47:50.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011440333, 2024-04-20T01:47:50.829 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.829 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:50.829 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:47:50.829 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:47:50.829 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:50.829 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 10 2024-04-20T01:47:50.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.829 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752800+0000", 2024-04-20T01:47:50.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752800+0000", 2024-04-20T01:47:50.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752799+0000", 2024-04-20T01:47:50.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:50.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752945+0000", 2024-04-20T01:47:50.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000145761 2024-04-20T01:47:50.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.831 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.831 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752955+0000", 2024-04-20T01:47:50.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0229e-05 2024-04-20T01:47:50.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.831 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.831 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752970+0000", 2024-04-20T01:47:50.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5242000000000001e-05 2024-04-20T01:47:50.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.831 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.832 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753046+0000", 2024-04-20T01:47:50.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6218999999999996e-05 2024-04-20T01:47:50.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:47:50.832 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753172+0000", 2024-04-20T01:47:50.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012566899999999999 2024-04-20T01:47:50.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.832 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753648+0000", 2024-04-20T01:47:50.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00047565500000000001 2024-04-20T01:47:50.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.832 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.756671+0000", 2024-04-20T01:47:50.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030231849999999998 2024-04-20T01:47:50.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.833 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:47:50.833 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764191+0000", 2024-04-20T01:47:50.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007520143 2024-04-20T01:47:50.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.833 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.833 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764240+0000", 2024-04-20T01:47:50.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9359e-05 2024-04-20T01:47:50.833 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.833 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.833 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.834 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:11 3.5s0 3:a2e9ac6b:::smithi16579500-4:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-20T01:47:50.834 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.752987+0000", 2024-04-20T01:47:50.834 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.088479259, 2024-04-20T01:47:50.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011858695000000001, 2024-04-20T01:47:50.834 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.834 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:50.834 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:47:50.834 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:47:50.834 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:50.834 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 11 2024-04-20T01:47:50.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.834 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752987+0000", 2024-04-20T01:47:50.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752987+0000", 2024-04-20T01:47:50.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752986+0000", 2024-04-20T01:47:50.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:50.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.836 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.836 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752995+0000", 2024-04-20T01:47:50.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7099999999999996e-06 2024-04-20T01:47:50.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.836 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.836 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753000+0000", 2024-04-20T01:47:50.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4900000000000002e-06 2024-04-20T01:47:50.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.836 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.836 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753003+0000", 2024-04-20T01:47:50.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4620000000000001e-06 2024-04-20T01:47:50.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.837 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.837 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753974+0000", 2024-04-20T01:47:50.837 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000971388 2024-04-20T01:47:50.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.837 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:47:50.837 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754054+0000", 2024-04-20T01:47:50.837 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9159000000000006e-05 2024-04-20T01:47:50.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.837 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.837 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754358+0000", 2024-04-20T01:47:50.837 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00030434800000000002 2024-04-20T01:47:50.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.759666+0000", 2024-04-20T01:47:50.838 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053076169999999997 2024-04-20T01:47:50.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:47:50.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764756+0000", 2024-04-20T01:47:50.838 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050904139999999997 2024-04-20T01:47:50.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764845+0000", 2024-04-20T01:47:50.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9488999999999996e-05 2024-04-20T01:47:50.839 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.839 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.839 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.839 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:12 3.5s0 3:a2e9ac6b:::smithi16579500-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-20T01:47:50.839 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.753012+0000", 2024-04-20T01:47:50.839 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.088454242000001, 2024-04-20T01:47:50.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014201745, 2024-04-20T01:47:50.839 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.839 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:47:50.839 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:47:50.839 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:47:50.840 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:50.840 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 12 2024-04-20T01:47:50.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.840 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.840 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.840 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753012+0000", 2024-04-20T01:47:50.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.840 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.840 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753012+0000", 2024-04-20T01:47:50.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753011+0000", 2024-04-20T01:47:50.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:50.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753016+0000", 2024-04-20T01:47:50.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.989e-06 2024-04-20T01:47:50.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753026+0000", 2024-04-20T01:47:50.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2690000000000005e-06 2024-04-20T01:47:50.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.842 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.842 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753033+0000", 2024-04-20T01:47:50.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3010000000000001e-06 2024-04-20T01:47:50.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.842 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.842 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754553+0000", 2024-04-20T01:47:50.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001519947 2024-04-20T01:47:50.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.842 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for rw locks", 2024-04-20T01:47:50.842 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754638+0000", 2024-04-20T01:47:50.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5421999999999996e-05 2024-04-20T01:47:50.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.765222+0000", 2024-04-20T01:47:50.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010583979 2024-04-20T01:47:50.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:47:50.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.765288+0000", 2024-04-20T01:47:50.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5296999999999995e-05 2024-04-20T01:47:50.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.767213+0000", 2024-04-20T01:47:50.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019259520000000001 2024-04-20T01:47:50.844 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.844 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.844 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.844 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:64 3.5s0 3:ae9f92b8:::smithi16579500-22:head [append 0~360448,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:50.844 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.781054+0000", 2024-04-20T01:47:50.844 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.060411773999999, 2024-04-20T01:47:50.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01202491, 2024-04-20T01:47:50.844 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.844 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:50.844 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:47:50.844 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:47:50.844 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:50.844 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 64 2024-04-20T01:47:50.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.845 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.845 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781054+0000", 2024-04-20T01:47:50.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.845 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781054+0000", 2024-04-20T01:47:50.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781053+0000", 2024-04-20T01:47:50.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:50.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.846 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781384+0000", 2024-04-20T01:47:50.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00033060899999999999 2024-04-20T01:47:50.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.846 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781394+0000", 2024-04-20T01:47:50.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0016e-05 2024-04-20T01:47:50.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.847 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781399+0000", 2024-04-20T01:47:50.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7930000000000003e-06 2024-04-20T01:47:50.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.847 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781446+0000", 2024-04-20T01:47:50.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6768999999999999e-05 2024-04-20T01:47:50.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:47:50.848 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781598+0000", 2024-04-20T01:47:50.848 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015158500000000001 2024-04-20T01:47:50.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.848 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.848 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.782377+0000", 2024-04-20T01:47:50.848 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000778975 2024-04-20T01:47:50.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.848 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.848 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784105+0000", 2024-04-20T01:47:50.848 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017285340000000001 2024-04-20T01:47:50.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.848 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:47:50.849 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793040+0000", 2024-04-20T01:47:50.849 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0089346440000000003 2024-04-20T01:47:50.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.849 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.849 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.849 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793079+0000", 2024-04-20T01:47:50.849 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9239000000000002e-05 2024-04-20T01:47:50.849 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.849 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.849 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.849 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.849 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:65 3.5s0 3:ae9f92b8:::smithi16579500-22:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:50.849 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.781409+0000", 2024-04-20T01:47:50.850 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.060057174000001, 2024-04-20T01:47:50.850 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011886404, 2024-04-20T01:47:50.850 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.850 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:50.850 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:47:50.850 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:47:50.850 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:50.850 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 65 2024-04-20T01:47:50.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.850 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.850 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.850 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.851 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781409+0000", 2024-04-20T01:47:50.851 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.851 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.851 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781409+0000", 2024-04-20T01:47:50.851 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.851 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.851 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781408+0000", 2024-04-20T01:47:50.851 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:50.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.852 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781411+0000", 2024-04-20T01:47:50.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6309999999999999e-06 2024-04-20T01:47:50.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.852 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.852 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781416+0000", 2024-04-20T01:47:50.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6550000000000003e-06 2024-04-20T01:47:50.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.852 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.852 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781423+0000", 2024-04-20T01:47:50.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4540000000000001e-06 2024-04-20T01:47:50.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.852 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.853 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.853 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.782838+0000", 2024-04-20T01:47:50.853 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001414741 2024-04-20T01:47:50.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.853 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.853 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:47:50.853 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.782921+0000", 2024-04-20T01:47:50.853 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3018999999999999e-05 2024-04-20T01:47:50.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.853 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.853 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.853 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.783284+0000", 2024-04-20T01:47:50.853 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00036349 2024-04-20T01:47:50.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.854 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.854 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784149+0000", 2024-04-20T01:47:50.854 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00086487700000000001 2024-04-20T01:47:50.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.854 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:47:50.854 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793195+0000", 2024-04-20T01:47:50.854 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0090454590000000005 2024-04-20T01:47:50.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.854 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.854 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793295+0000", 2024-04-20T01:47:50.854 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010049 2024-04-20T01:47:50.854 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.855 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.855 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.855 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:66 3.5s0 3:ae9f92b8:::smithi16579500-22:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:50.855 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.781436+0000", 2024-04-20T01:47:50.855 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.060029743000001, 2024-04-20T01:47:50.855 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014237204999999999, 2024-04-20T01:47:50.855 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.855 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:47:50.855 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:47:50.855 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:47:50.855 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:50.855 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 66 2024-04-20T01:47:50.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.856 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781436+0000", 2024-04-20T01:47:50.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781436+0000", 2024-04-20T01:47:50.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781436+0000", 2024-04-20T01:47:50.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:50.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781441+0000", 2024-04-20T01:47:50.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3199999999999999e-06 2024-04-20T01:47:50.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781446+0000", 2024-04-20T01:47:50.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.544e-06 2024-04-20T01:47:50.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781451+0000", 2024-04-20T01:47:50.858 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5570000000000003e-06 2024-04-20T01:47:50.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.858 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.858 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.783430+0000", 2024-04-20T01:47:50.858 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019785699999999998 2024-04-20T01:47:50.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.858 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for rw locks", 2024-04-20T01:47:50.858 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.783482+0000", 2024-04-20T01:47:50.858 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1780000000000002e-05 2024-04-20T01:47:50.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.858 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.858 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793826+0000", 2024-04-20T01:47:50.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010344132000000001 2024-04-20T01:47:50.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.859 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:47:50.859 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793907+0000", 2024-04-20T01:47:50.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1173000000000004e-05 2024-04-20T01:47:50.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.859 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.859 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.795673+0000", 2024-04-20T01:47:50.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001766442 2024-04-20T01:47:50.859 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.859 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.859 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.860 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.bs5 33/30 ECSubWrite(tid=8, reqid=client.4643.0:59, at_version=0'0, trim_to=0'0, roll_forward_to=33'4))", 2024-04-20T01:47:50.860 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.792962+0000", 2024-04-20T01:47:50.860 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.048504393999998, 2024-04-20T01:47:50.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043862119999999996, 2024-04-20T01:47:50.860 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.860 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.860 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.860 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.860 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.792962+0000", 2024-04-20T01:47:50.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.792962+0000", 2024-04-20T01:47:50.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.792961+0000", 2024-04-20T01:47:50.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:47:50.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.792972+0000", 2024-04-20T01:47:50.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1121000000000001e-05 2024-04-20T01:47:50.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.792979+0000", 2024-04-20T01:47:50.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.72e-06 2024-04-20T01:47:50.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.792986+0000", 2024-04-20T01:47:50.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0720000000000002e-06 2024-04-20T01:47:50.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.863 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793021+0000", 2024-04-20T01:47:50.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5027000000000002e-05 2024-04-20T01:47:50.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.863 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793028+0000", 2024-04-20T01:47:50.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6440000000000003e-06 2024-04-20T01:47:50.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.863 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797311+0000", 2024-04-20T01:47:50.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042826979999999997 2024-04-20T01:47:50.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.797348+0000", 2024-04-20T01:47:50.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7141000000000002e-05 2024-04-20T01:47:50.864 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.864 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.864 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.864 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s6 33/30 ECSubWrite(tid=177, reqid=client.4643.0:83, at_version=33'6, trim_to=0'0, roll_forward_to=33'4))", 2024-04-20T01:47:50.864 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.800448+0000", 2024-04-20T01:47:50.864 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.041017802999999, 2024-04-20T01:47:50.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005436504, 2024-04-20T01:47:50.864 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.864 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.865 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.800448+0000", 2024-04-20T01:47:50.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.800448+0000", 2024-04-20T01:47:50.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.800448+0000", 2024-04-20T01:47:50.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:50.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.800463+0000", 2024-04-20T01:47:50.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5854000000000001e-05 2024-04-20T01:47:50.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.800476+0000", 2024-04-20T01:47:50.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2157999999999999e-05 2024-04-20T01:47:50.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.800485+0000", 2024-04-20T01:47:50.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9020000000000005e-06 2024-04-20T01:47:50.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.800538+0000", 2024-04-20T01:47:50.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3647000000000003e-05 2024-04-20T01:47:50.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.800545+0000", 2024-04-20T01:47:50.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8580000000000002e-06 2024-04-20T01:47:50.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.805859+0000", 2024-04-20T01:47:50.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053143360000000002 2024-04-20T01:47:50.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.868 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.868 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.805885+0000", 2024-04-20T01:47:50.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5301999999999999e-05 2024-04-20T01:47:50.868 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.868 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.868 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.868 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 33/30 ECSubWrite(tid=178, reqid=client.4643.0:85, at_version=33'7, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:50.868 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.801466+0000", 2024-04-20T01:47:50.868 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.040000114000001, 2024-04-20T01:47:50.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043518330000000003, 2024-04-20T01:47:50.869 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.869 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.869 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.801466+0000", 2024-04-20T01:47:50.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.801466+0000", 2024-04-20T01:47:50.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.801465+0000", 2024-04-20T01:47:50.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:50.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.801628+0000", 2024-04-20T01:47:50.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016292699999999999 2024-04-20T01:47:50.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.801640+0000", 2024-04-20T01:47:50.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1953999999999999e-05 2024-04-20T01:47:50.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.801649+0000", 2024-04-20T01:47:50.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3739999999999998e-06 2024-04-20T01:47:50.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.801692+0000", 2024-04-20T01:47:50.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3093000000000002e-05 2024-04-20T01:47:50.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.801702+0000", 2024-04-20T01:47:50.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0139000000000001e-05 2024-04-20T01:47:50.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.872 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.872 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.805797+0000", 2024-04-20T01:47:50.872 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040953220000000002 2024-04-20T01:47:50.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.872 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.872 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.872 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.805818+0000", 2024-04-20T01:47:50.872 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0547000000000002e-05 2024-04-20T01:47:50.872 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.872 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.872 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.872 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.872 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 42/30 ECSubWrite(tid=229, reqid=client.4643.0:302, at_version=42'29, trim_to=0'0, roll_forward_to=42'27))", 2024-04-20T01:47:50.872 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.829864+0000", 2024-04-20T01:47:50.873 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 7.0116016830000003, 2024-04-20T01:47:50.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044347550000000003, 2024-04-20T01:47:50.873 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.873 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.873 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.873 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.873 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.829864+0000", 2024-04-20T01:47:50.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.873 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.873 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.829864+0000", 2024-04-20T01:47:50.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.829863+0000", 2024-04-20T01:47:50.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:50.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830522+0000", 2024-04-20T01:47:50.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00065908700000000004 2024-04-20T01:47:50.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830537+0000", 2024-04-20T01:47:50.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5088e-05 2024-04-20T01:47:50.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830547+0000", 2024-04-20T01:47:50.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9389999999999992e-06 2024-04-20T01:47:50.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830598+0000", 2024-04-20T01:47:50.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0268000000000001e-05 2024-04-20T01:47:50.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830608+0000", 2024-04-20T01:47:50.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0163000000000001e-05 2024-04-20T01:47:50.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.834276+0000", 2024-04-20T01:47:50.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003668494 2024-04-20T01:47:50.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.834299+0000", 2024-04-20T01:47:50.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2694e-05 2024-04-20T01:47:50.876 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.876 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.876 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.877 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.bs5 47/30 ECSubWrite(tid=114, reqid=client.4643.0:392, at_version=47'49, trim_to=0'0, roll_forward_to=47'47))", 2024-04-20T01:47:50.877 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:48.746780+0000", 2024-04-20T01:47:50.877 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 2.094686184, 2024-04-20T01:47:50.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054560069999999997, 2024-04-20T01:47:50.877 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.877 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.877 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.877 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.877 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746780+0000", 2024-04-20T01:47:50.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.877 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.877 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746780+0000", 2024-04-20T01:47:50.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746779+0000", 2024-04-20T01:47:50.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:50.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746900+0000", 2024-04-20T01:47:50.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121138 2024-04-20T01:47:50.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746917+0000", 2024-04-20T01:47:50.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6852999999999999e-05 2024-04-20T01:47:50.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746927+0000", 2024-04-20T01:47:50.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0237999999999999e-05 2024-04-20T01:47:50.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746982+0000", 2024-04-20T01:47:50.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5290000000000001e-05 2024-04-20T01:47:50.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746993+0000", 2024-04-20T01:47:50.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0781000000000001e-05 2024-04-20T01:47:50.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.752210+0000", 2024-04-20T01:47:50.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052170059999999997 2024-04-20T01:47:50.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.752236+0000", 2024-04-20T01:47:50.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5672000000000001e-05 2024-04-20T01:47:50.880 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.880 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.880 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.881 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 47/30 ECSubWrite(tid=247, reqid=client.4643.0:389, at_version=47'39, trim_to=0'0, roll_forward_to=47'35))", 2024-04-20T01:47:50.881 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:48.746964+0000", 2024-04-20T01:47:50.881 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 2.0945015499999999, 2024-04-20T01:47:50.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056576650000000001, 2024-04-20T01:47:50.881 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.881 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.881 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746964+0000", 2024-04-20T01:47:50.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746964+0000", 2024-04-20T01:47:50.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746964+0000", 2024-04-20T01:47:50.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:50.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747077+0000", 2024-04-20T01:47:50.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113581 2024-04-20T01:47:50.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.883 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747089+0000", 2024-04-20T01:47:50.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1795e-05 2024-04-20T01:47:50.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.883 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747098+0000", 2024-04-20T01:47:50.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1630000000000002e-06 2024-04-20T01:47:50.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.883 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747143+0000", 2024-04-20T01:47:50.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5286e-05 2024-04-20T01:47:50.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.884 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.884 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747154+0000", 2024-04-20T01:47:50.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0488e-05 2024-04-20T01:47:50.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.884 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.884 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.752599+0000", 2024-04-20T01:47:50.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054446670000000003 2024-04-20T01:47:50.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.884 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.884 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.752622+0000", 2024-04-20T01:47:50.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3521999999999999e-05 2024-04-20T01:47:50.884 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.884 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.885 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.885 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 47/30 ECSubWrite(tid=248, reqid=client.4643.0:390, at_version=47'40, trim_to=0'0, roll_forward_to=47'35))", 2024-04-20T01:47:50.885 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:48.747742+0000", 2024-04-20T01:47:50.885 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 2.0937240670000001, 2024-04-20T01:47:50.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049122280000000003, 2024-04-20T01:47:50.885 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.885 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.885 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747742+0000", 2024-04-20T01:47:50.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.886 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.886 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747742+0000", 2024-04-20T01:47:50.886 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.886 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.886 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747741+0000", 2024-04-20T01:47:50.886 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:50.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.886 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.886 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747771+0000", 2024-04-20T01:47:50.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0212000000000001e-05 2024-04-20T01:47:50.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747785+0000", 2024-04-20T01:47:50.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3305000000000001e-05 2024-04-20T01:47:50.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747794+0000", 2024-04-20T01:47:50.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8780000000000004e-06 2024-04-20T01:47:50.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747843+0000", 2024-04-20T01:47:50.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9455e-05 2024-04-20T01:47:50.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.888 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.888 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747854+0000", 2024-04-20T01:47:50.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0723999999999999e-05 2024-04-20T01:47:50.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.888 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.888 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.752643+0000", 2024-04-20T01:47:50.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047892969999999997 2024-04-20T01:47:50.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.888 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.888 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.752654+0000", 2024-04-20T01:47:50.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1133e-05 2024-04-20T01:47:50.889 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.889 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.889 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.889 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s6 48/30 ECSubWrite(tid=251, reqid=osd.13.0:251, at_version=48'22, trim_to=0'0, roll_forward_to=47'20))", 2024-04-20T01:47:50.889 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.710374+0000", 2024-04-20T01:47:50.889 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.1310918400000001, 2024-04-20T01:47:50.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069805379999999997, 2024-04-20T01:47:50.889 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.889 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.889 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710374+0000", 2024-04-20T01:47:50.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710374+0000", 2024-04-20T01:47:50.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710374+0000", 2024-04-20T01:47:50.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:50.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.891 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.891 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.891 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710395+0000", 2024-04-20T01:47:50.891 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1396e-05 2024-04-20T01:47:50.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.891 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.891 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.891 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710417+0000", 2024-04-20T01:47:50.891 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1857000000000001e-05 2024-04-20T01:47:50.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.891 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.891 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.891 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710423+0000", 2024-04-20T01:47:50.891 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1469999999999998e-06 2024-04-20T01:47:50.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.892 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.892 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.892 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710498+0000", 2024-04-20T01:47:50.892 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5072000000000002e-05 2024-04-20T01:47:50.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.892 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.892 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.892 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710509+0000", 2024-04-20T01:47:50.892 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0685e-05 2024-04-20T01:47:50.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.892 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.892 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.892 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717332+0000", 2024-04-20T01:47:50.892 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006823048 2024-04-20T01:47:50.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.893 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.893 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.893 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717355+0000", 2024-04-20T01:47:50.893 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2731e-05 2024-04-20T01:47:50.893 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.893 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.893 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.893 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.893 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 48/30 ECSubWrite(tid=253, reqid=osd.13.0:253, at_version=48'44, trim_to=0'0, roll_forward_to=47'40))", 2024-04-20T01:47:50.893 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.711134+0000", 2024-04-20T01:47:50.893 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.1303318629999999, 2024-04-20T01:47:50.893 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006623304, 2024-04-20T01:47:50.893 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.894 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.894 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.894 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.894 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.894 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711134+0000", 2024-04-20T01:47:50.894 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.894 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.894 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.894 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711134+0000", 2024-04-20T01:47:50.894 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.894 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.894 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.895 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711134+0000", 2024-04-20T01:47:50.895 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:50.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.895 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.895 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.895 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711155+0000", 2024-04-20T01:47:50.895 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1673000000000001e-05 2024-04-20T01:47:50.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.895 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.895 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.895 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711166+0000", 2024-04-20T01:47:50.895 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0558e-05 2024-04-20T01:47:50.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.895 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.896 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.896 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711174+0000", 2024-04-20T01:47:50.896 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3359999999999995e-06 2024-04-20T01:47:50.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.896 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.896 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.896 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711257+0000", 2024-04-20T01:47:50.896 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3312999999999993e-05 2024-04-20T01:47:50.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.896 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.896 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.896 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711273+0000", 2024-04-20T01:47:50.896 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5701000000000001e-05 2024-04-20T01:47:50.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.896 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.897 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.897 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717738+0000", 2024-04-20T01:47:50.897 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064653339999999997 2024-04-20T01:47:50.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.897 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.897 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.897 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717757+0000", 2024-04-20T01:47:50.897 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8972999999999999e-05 2024-04-20T01:47:50.897 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.897 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.897 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.897 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.897 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s6 48/30 ECSubWrite(tid=254, reqid=osd.13.0:251, at_version=0'0, trim_to=0'0, roll_forward_to=48'22))", 2024-04-20T01:47:50.897 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.718148+0000", 2024-04-20T01:47:50.898 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.123317608, 2024-04-20T01:47:50.898 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047546410000000004, 2024-04-20T01:47:50.898 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.898 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.898 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.898 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.898 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.898 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718148+0000", 2024-04-20T01:47:50.898 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.898 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.898 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.898 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718148+0000", 2024-04-20T01:47:50.898 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.899 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.899 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.899 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718148+0000", 2024-04-20T01:47:50.899 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:50.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.899 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.899 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.899 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718158+0000", 2024-04-20T01:47:50.899 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0142e-05 2024-04-20T01:47:50.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.899 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.899 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.899 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718169+0000", 2024-04-20T01:47:50.899 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.063e-05 2024-04-20T01:47:50.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.900 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.900 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.900 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718177+0000", 2024-04-20T01:47:50.900 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1580000000000005e-06 2024-04-20T01:47:50.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.900 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.900 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.900 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718231+0000", 2024-04-20T01:47:50.900 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4571999999999999e-05 2024-04-20T01:47:50.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.900 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.900 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.900 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718241+0000", 2024-04-20T01:47:50.901 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4490000000000005e-06 2024-04-20T01:47:50.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.901 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.901 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.901 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.722884+0000", 2024-04-20T01:47:50.901 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046432390000000004 2024-04-20T01:47:50.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.901 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.901 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.901 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.722903+0000", 2024-04-20T01:47:50.901 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9051000000000001e-05 2024-04-20T01:47:50.901 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.901 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.901 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.902 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.902 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.bs5 48/30 ECSubWrite(tid=121, reqid=osd.4.0:121, at_version=48'53, trim_to=0'0, roll_forward_to=47'49))", 2024-04-20T01:47:50.902 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.718213+0000", 2024-04-20T01:47:50.902 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.123252937, 2024-04-20T01:47:50.902 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047005479999999997, 2024-04-20T01:47:50.902 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.902 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.902 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.902 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.902 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.902 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718213+0000", 2024-04-20T01:47:50.903 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.903 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.903 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.903 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718213+0000", 2024-04-20T01:47:50.903 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.903 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.903 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.903 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718213+0000", 2024-04-20T01:47:50.903 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:50.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.903 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.903 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.904 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718231+0000", 2024-04-20T01:47:50.904 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8839999999999999e-05 2024-04-20T01:47:50.904 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.904 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.904 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.904 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718254+0000", 2024-04-20T01:47:50.904 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2259000000000002e-05 2024-04-20T01:47:50.904 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.904 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.904 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.904 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718261+0000", 2024-04-20T01:47:50.904 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3640000000000001e-06 2024-04-20T01:47:50.904 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.904 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.905 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.905 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718326+0000", 2024-04-20T01:47:50.905 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4694000000000001e-05 2024-04-20T01:47:50.905 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.905 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.905 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.905 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718335+0000", 2024-04-20T01:47:50.905 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4900000000000006e-06 2024-04-20T01:47:50.905 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.905 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.905 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.905 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.722905+0000", 2024-04-20T01:47:50.905 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045696850000000004 2024-04-20T01:47:50.905 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.905 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.906 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.906 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.722914+0000", 2024-04-20T01:47:50.906 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7490000000000004e-06 2024-04-20T01:47:50.906 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.906 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.906 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.906 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.906 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:412 3.5s0 3:ae9f92b8:::smithi16579500-22:head [read 0~360448,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-04-20T01:47:50.906 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.788698+0000", 2024-04-20T01:47:50.906 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.052767735, 2024-04-20T01:47:50.906 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041018490000000003, 2024-04-20T01:47:50.906 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.906 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:47:50.907 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:47:50.907 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:47:50.907 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:50.907 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 412 2024-04-20T01:47:50.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.907 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.907 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.907 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.907 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.788698+0000", 2024-04-20T01:47:50.907 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.907 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.907 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.907 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.788698+0000", 2024-04-20T01:47:50.907 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.908 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.908 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.908 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.788698+0000", 2024-04-20T01:47:50.908 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:50.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.908 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.908 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.908 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.788702+0000", 2024-04-20T01:47:50.908 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.614e-06 2024-04-20T01:47:50.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.908 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.908 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.908 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.788705+0000", 2024-04-20T01:47:50.909 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.591e-06 2024-04-20T01:47:50.909 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.909 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.909 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.909 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.788708+0000", 2024-04-20T01:47:50.909 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9469999999999999e-06 2024-04-20T01:47:50.909 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.909 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.909 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.909 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.790272+0000", 2024-04-20T01:47:50.909 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001564326 2024-04-20T01:47:50.909 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.909 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.909 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:47:50.910 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.790372+0000", 2024-04-20T01:47:50.910 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9748999999999997e-05 2024-04-20T01:47:50.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.910 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.910 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.910 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.792800+0000", 2024-04-20T01:47:50.910 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024279150000000001 2024-04-20T01:47:50.910 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.910 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.910 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.910 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.910 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:413 3.5s0 3:ae9f92b8:::smithi16579500-22:head [read 0~360448,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-04-20T01:47:50.910 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.788725+0000", 2024-04-20T01:47:50.911 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.0527413939999999, 2024-04-20T01:47:50.911 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047934550000000003, 2024-04-20T01:47:50.911 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.911 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:47:50.911 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:47:50.911 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:47:50.911 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:50.911 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 413 2024-04-20T01:47:50.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.911 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.911 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.911 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.911 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.788725+0000", 2024-04-20T01:47:50.911 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.912 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.912 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.912 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.788725+0000", 2024-04-20T01:47:50.912 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.912 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.912 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.912 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.912 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.788724+0000", 2024-04-20T01:47:50.912 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:47:50.912 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.912 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.912 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.912 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.788726+0000", 2024-04-20T01:47:50.912 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.902e-06 2024-04-20T01:47:50.912 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.913 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.913 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.913 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.788730+0000", 2024-04-20T01:47:50.913 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6820000000000001e-06 2024-04-20T01:47:50.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.913 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.913 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.913 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.788733+0000", 2024-04-20T01:47:50.913 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.022e-06 2024-04-20T01:47:50.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.913 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.913 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.913 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.791867+0000", 2024-04-20T01:47:50.913 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003133645 2024-04-20T01:47:50.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.914 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.914 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:47:50.914 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.791947+0000", 2024-04-20T01:47:50.914 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0133e-05 2024-04-20T01:47:50.914 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.914 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.914 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.914 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.793518+0000", 2024-04-20T01:47:50.914 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001571306 2024-04-20T01:47:50.914 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.914 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.914 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.914 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.914 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.915 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 48/30 ECSubWrite(tid=256, reqid=client.4643.0:403, at_version=48'46, trim_to=0'0, roll_forward_to=48'44))", 2024-04-20T01:47:50.915 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.789581+0000", 2024-04-20T01:47:50.915 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.051884491, 2024-04-20T01:47:50.915 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048595499999999998, 2024-04-20T01:47:50.915 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:50.915 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:50.915 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:50.915 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.915 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:50.915 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.789581+0000", 2024-04-20T01:47:50.915 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.915 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.915 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:50.915 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.789581+0000", 2024-04-20T01:47:50.916 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:50.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.916 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.916 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:50.916 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.789581+0000", 2024-04-20T01:47:50.916 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:50.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.916 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.916 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:50.916 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.790322+0000", 2024-04-20T01:47:50.916 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00074123400000000001 2024-04-20T01:47:50.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.916 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.916 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:50.917 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.790331+0000", 2024-04-20T01:47:50.917 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8799999999999997e-06 2024-04-20T01:47:50.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.917 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.917 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:50.917 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.790339+0000", 2024-04-20T01:47:50.917 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2689999999999999e-06 2024-04-20T01:47:50.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.917 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.917 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:50.917 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.790385+0000", 2024-04-20T01:47:50.917 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5813000000000003e-05 2024-04-20T01:47:50.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.917 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.917 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:50.918 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.790396+0000", 2024-04-20T01:47:50.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0719e-05 2024-04-20T01:47:50.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.918 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.918 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:50.918 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.794417+0000", 2024-04-20T01:47:50.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040212370000000004 2024-04-20T01:47:50.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:50.918 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:50.918 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:50.918 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.794441+0000", 2024-04-20T01:47:50.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4153999999999999e-05 2024-04-20T01:47:50.918 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.918 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.918 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.919 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:50.919 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:50.919 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:50.920 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-20T01:47:50.921 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:50.935+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:50.950 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:47:50.971 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:47:50.991 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:47:50.993 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:47:50.993 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:50.993 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 49, 2024-04-20T01:47:50.993 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "9c2f64c1-aa35-4337-9f5c-081b9e3a2938", 2024-04-20T01:47:50.993 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2024-04-20T01:47:02.934241+0000", 2024-04-20T01:47:50.993 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-04-20T01:47:50.704874+0000", 2024-04-20T01:47:50.993 INFO:teuthology.orchestra.run.smithi044.stdout: "last_up_change": "2024-04-20T01:47:15.986754+0000", 2024-04-20T01:47:50.994 INFO:teuthology.orchestra.run.smithi044.stdout: "last_in_change": "2024-04-20T01:47:07.894719+0000", 2024-04-20T01:47:50.994 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-20T01:47:50.994 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_num": 5799936, 2024-04-20T01:47:50.994 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_set": [ 2024-04-20T01:47:50.994 INFO:teuthology.orchestra.run.smithi044.stdout: "pglog_hardlimit", 2024-04-20T01:47:50.994 INFO:teuthology.orchestra.run.smithi044.stdout: "purged_snapdirs", 2024-04-20T01:47:50.994 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_deletes", 2024-04-20T01:47:50.994 INFO:teuthology.orchestra.run.smithi044.stdout: "sortbitwise" 2024-04-20T01:47:50.994 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:47:50.994 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_version": 5, 2024-04-20T01:47:50.994 INFO:teuthology.orchestra.run.smithi044.stdout: "full_ratio": 0.94999998807907104, 2024-04-20T01:47:50.994 INFO:teuthology.orchestra.run.smithi044.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-20T01:47:50.994 INFO:teuthology.orchestra.run.smithi044.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-20T01:47:50.995 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_snapshot": "", 2024-04-20T01:47:50.995 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_max": 3, 2024-04-20T01:47:50.995 INFO:teuthology.orchestra.run.smithi044.stdout: "max_osd": 16, 2024-04-20T01:47:50.995 INFO:teuthology.orchestra.run.smithi044.stdout: "require_min_compat_client": "luminous", 2024-04-20T01:47:50.995 INFO:teuthology.orchestra.run.smithi044.stdout: "min_compat_client": "jewel", 2024-04-20T01:47:50.995 INFO:teuthology.orchestra.run.smithi044.stdout: "require_osd_release": "reef", 2024-04-20T01:47:50.995 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_crimson": false, 2024-04-20T01:47:50.995 INFO:teuthology.orchestra.run.smithi044.stdout: "pools": [ 2024-04-20T01:47:50.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:50.995 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 1, 2024-04-20T01:47:50.995 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": ".mgr", 2024-04-20T01:47:50.995 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-04-20T01:47:15.195181+0000", 2024-04-20T01:47:50.995 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T01:47:50.995 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool", 2024-04-20T01:47:50.996 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 1, 2024-04-20T01:47:50.996 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 2, 2024-04-20T01:47:50.996 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 1, 2024-04-20T01:47:50.996 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 0, 2024-04-20T01:47:50.996 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-04-20T01:47:50.996 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-04-20T01:47:50.996 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-04-20T01:47:50.996 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-20T01:47:50.996 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-04-20T01:47:50.996 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-04-20T01:47:50.996 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 1, 2024-04-20T01:47:50.996 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 1, 2024-04-20T01:47:50.996 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 1, 2024-04-20T01:47:50.996 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 1, 2024-04-20T01:47:50.996 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 1, 2024-04-20T01:47:50.997 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-04-20T01:47:50.997 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-04-20T01:47:50.997 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-04-20T01:47:50.997 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-04-20T01:47:50.997 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-04-20T01:47:50.997 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-04-20T01:47:50.997 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-04-20T01:47:50.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:50.997 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "24", 2024-04-20T01:47:50.997 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-04-20T01:47:50.997 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-20T01:47:50.997 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-04-20T01:47:50.997 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-04-20T01:47:50.997 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-04-20T01:47:50.998 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 0, 2024-04-20T01:47:50.998 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 0, 2024-04-20T01:47:50.998 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-04-20T01:47:50.998 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-04-20T01:47:50.998 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-04-20T01:47:50.998 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-04-20T01:47:50.998 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-04-20T01:47:50.998 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-04-20T01:47:50.998 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-04-20T01:47:50.998 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-04-20T01:47:50.998 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-04-20T01:47:50.998 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-04-20T01:47:50.998 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-04-20T01:47:50.998 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-20T01:47:50.999 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-20T01:47:50.999 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-20T01:47:50.999 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-04-20T01:47:50.999 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-04-20T01:47:50.999 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "", 2024-04-20T01:47:50.999 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-04-20T01:47:50.999 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-04-20T01:47:50.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:50.999 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-04-20T01:47:50.999 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-04-20T01:47:50.999 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-04-20T01:47:50.999 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-04-20T01:47:50.999 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-04-20T01:47:50.999 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-04-20T01:47:51.000 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-04-20T01:47:51.000 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-04-20T01:47:51.000 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 0, 2024-04-20T01:47:51.000 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-04-20T01:47:51.000 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-04-20T01:47:51.000 INFO:teuthology.orchestra.run.smithi044.stdout: "options": { 2024-04-20T01:47:51.000 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_max": 32, 2024-04-20T01:47:51.000 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_min": 1 2024-04-20T01:47:51.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.000 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-04-20T01:47:51.000 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": {} 2024-04-20T01:47:51.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.000 INFO:teuthology.orchestra.run.smithi044.stdout: "read_balance": { 2024-04-20T01:47:51.000 INFO:teuthology.orchestra.run.smithi044.stdout: "score_acting": 15.380000114440918, 2024-04-20T01:47:51.000 INFO:teuthology.orchestra.run.smithi044.stdout: "score_stable": 15.380000114440918, 2024-04-20T01:47:51.001 INFO:teuthology.orchestra.run.smithi044.stdout: "optimal_score": 0.12999999523162842, 2024-04-20T01:47:51.001 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_acting": 2, 2024-04-20T01:47:51.001 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_stable": 2, 2024-04-20T01:47:51.001 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity_weighted": 1, 2024-04-20T01:47:51.001 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity": 1, 2024-04-20T01:47:51.001 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity_weighted": 1 2024-04-20T01:47:51.001 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.001 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 2, 2024-04-20T01:47:51.001 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": "rbd", 2024-04-20T01:47:51.001 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-04-20T01:47:18.448004+0000", 2024-04-20T01:47:51.001 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T01:47:51.001 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool", 2024-04-20T01:47:51.002 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 1, 2024-04-20T01:47:51.002 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 2, 2024-04-20T01:47:51.002 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 1, 2024-04-20T01:47:51.002 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 0, 2024-04-20T01:47:51.002 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-04-20T01:47:51.002 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-04-20T01:47:51.002 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-04-20T01:47:51.002 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-20T01:47:51.002 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-04-20T01:47:51.002 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-04-20T01:47:51.002 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 8, 2024-04-20T01:47:51.002 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 8, 2024-04-20T01:47:51.002 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 8, 2024-04-20T01:47:51.003 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 8, 2024-04-20T01:47:51.003 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 8, 2024-04-20T01:47:51.003 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-04-20T01:47:51.003 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-04-20T01:47:51.003 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-04-20T01:47:51.003 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-04-20T01:47:51.003 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-04-20T01:47:51.003 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-04-20T01:47:51.003 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-04-20T01:47:51.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.003 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "27", 2024-04-20T01:47:51.003 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-04-20T01:47:51.003 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-20T01:47:51.003 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-04-20T01:47:51.004 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-04-20T01:47:51.004 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-04-20T01:47:51.004 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 0, 2024-04-20T01:47:51.004 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 0, 2024-04-20T01:47:51.004 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-04-20T01:47:51.004 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-04-20T01:47:51.004 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-04-20T01:47:51.004 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-04-20T01:47:51.004 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-04-20T01:47:51.004 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-04-20T01:47:51.004 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-04-20T01:47:51.004 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-04-20T01:47:51.004 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-04-20T01:47:51.004 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-04-20T01:47:51.004 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-04-20T01:47:51.005 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-20T01:47:51.005 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-20T01:47:51.005 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-20T01:47:51.005 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-04-20T01:47:51.005 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-04-20T01:47:51.005 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "", 2024-04-20T01:47:51.005 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-04-20T01:47:51.005 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-04-20T01:47:51.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.005 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-04-20T01:47:51.005 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-04-20T01:47:51.005 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-04-20T01:47:51.005 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-04-20T01:47:51.006 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-04-20T01:47:51.006 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-04-20T01:47:51.006 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-04-20T01:47:51.006 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-04-20T01:47:51.006 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 0, 2024-04-20T01:47:51.006 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-04-20T01:47:51.006 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-04-20T01:47:51.006 INFO:teuthology.orchestra.run.smithi044.stdout: "options": {}, 2024-04-20T01:47:51.006 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-04-20T01:47:51.006 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd": {} 2024-04-20T01:47:51.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.006 INFO:teuthology.orchestra.run.smithi044.stdout: "read_balance": { 2024-04-20T01:47:51.006 INFO:teuthology.orchestra.run.smithi044.stdout: "score_acting": 3.9900000095367432, 2024-04-20T01:47:51.006 INFO:teuthology.orchestra.run.smithi044.stdout: "score_stable": 3.9900000095367432, 2024-04-20T01:47:51.006 INFO:teuthology.orchestra.run.smithi044.stdout: "optimal_score": 0.68999999761581421, 2024-04-20T01:47:51.007 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_acting": 2.75, 2024-04-20T01:47:51.007 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_stable": 2.75, 2024-04-20T01:47:51.007 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity_weighted": 1, 2024-04-20T01:47:51.007 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity": 1, 2024-04-20T01:47:51.007 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity_weighted": 1 2024-04-20T01:47:51.007 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.007 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 3, 2024-04-20T01:47:51.007 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": "unique_pool_0", 2024-04-20T01:47:51.007 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-04-20T01:47:30.734962+0000", 2024-04-20T01:47:51.007 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 8193, 2024-04-20T01:47:51.007 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-04-20T01:47:51.007 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 3, 2024-04-20T01:47:51.008 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 7, 2024-04-20T01:47:51.008 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 5, 2024-04-20T01:47:51.008 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 1, 2024-04-20T01:47:51.008 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-04-20T01:47:51.008 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-04-20T01:47:51.008 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-04-20T01:47:51.008 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-20T01:47:51.008 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-04-20T01:47:51.008 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-04-20T01:47:51.008 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 16, 2024-04-20T01:47:51.008 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 16, 2024-04-20T01:47:51.008 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 16, 2024-04-20T01:47:51.008 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 16, 2024-04-20T01:47:51.008 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 16, 2024-04-20T01:47:51.008 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-04-20T01:47:51.009 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-04-20T01:47:51.009 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-04-20T01:47:51.009 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-04-20T01:47:51.009 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-04-20T01:47:51.009 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-04-20T01:47:51.009 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-04-20T01:47:51.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.009 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "49", 2024-04-20T01:47:51.009 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-04-20T01:47:51.009 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-20T01:47:51.009 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-04-20T01:47:51.009 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-04-20T01:47:51.009 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-04-20T01:47:51.009 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 17, 2024-04-20T01:47:51.009 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 49, 2024-04-20T01:47:51.010 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-04-20T01:47:51.010 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-04-20T01:47:51.010 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-04-20T01:47:51.010 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-04-20T01:47:51.010 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-04-20T01:47:51.010 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-04-20T01:47:51.010 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-04-20T01:47:51.010 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-04-20T01:47:51.010 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-04-20T01:47:51.010 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-04-20T01:47:51.010 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-04-20T01:47:51.010 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-20T01:47:51.010 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-20T01:47:51.011 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-20T01:47:51.011 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-04-20T01:47:51.011 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-04-20T01:47:51.011 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "shecprofile", 2024-04-20T01:47:51.011 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-04-20T01:47:51.011 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-04-20T01:47:51.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.011 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-04-20T01:47:51.011 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-04-20T01:47:51.011 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-04-20T01:47:51.011 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-04-20T01:47:51.011 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-04-20T01:47:51.011 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-04-20T01:47:51.011 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-04-20T01:47:51.012 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-04-20T01:47:51.012 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 16384, 2024-04-20T01:47:51.012 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-04-20T01:47:51.012 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-04-20T01:47:51.012 INFO:teuthology.orchestra.run.smithi044.stdout: "options": {}, 2024-04-20T01:47:51.012 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-04-20T01:47:51.012 INFO:teuthology.orchestra.run.smithi044.stdout: "rados": {} 2024-04-20T01:47:51.012 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.012 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.012 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:47:51.012 INFO:teuthology.orchestra.run.smithi044.stdout: "osds": [ 2024-04-20T01:47:51.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.012 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 0, 2024-04-20T01:47:51.012 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "3903b88f-ff2b-4252-9480-92b701b1e59b", 2024-04-20T01:47:51.012 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:51.012 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:51.013 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:51.013 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:51.013 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:51.013 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:51.013 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:51.013 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:51.013 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:51.013 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:51.013 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:51.013 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.013 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.013 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6824", 2024-04-20T01:47:51.013 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:47:51.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.014 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.014 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6825", 2024-04-20T01:47:51.014 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:47:51.014 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.014 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.014 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:51.014 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.014 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.014 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6826", 2024-04-20T01:47:51.015 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:47:51.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.015 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.015 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.015 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6827", 2024-04-20T01:47:51.015 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:47:51.015 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.015 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.015 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:51.015 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.015 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.015 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.015 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6830", 2024-04-20T01:47:51.015 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:47:51.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.016 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.016 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6831", 2024-04-20T01:47:51.016 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:47:51.016 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.016 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.016 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:51.016 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.016 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.016 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6828", 2024-04-20T01:47:51.016 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:47:51.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.017 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.017 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6829", 2024-04-20T01:47:51.017 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:47:51.017 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.017 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.017 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6825/43518", 2024-04-20T01:47:51.017 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6827/43518", 2024-04-20T01:47:51.017 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6831/43518", 2024-04-20T01:47:51.017 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6829/43518", 2024-04-20T01:47:51.017 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:51.017 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:51.018 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:51.018 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.018 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 1, 2024-04-20T01:47:51.018 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "c5ff0175-b4f0-40a5-86c7-7a0a351bf070", 2024-04-20T01:47:51.018 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:51.018 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:51.018 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:51.018 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:51.018 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:51.018 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:51.018 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:51.019 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 0, 2024-04-20T01:47:51.019 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:51.019 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:51.019 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:51.019 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.019 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.019 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6816", 2024-04-20T01:47:51.019 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:47:51.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.019 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.019 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6817", 2024-04-20T01:47:51.019 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:47:51.020 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.020 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.020 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:51.020 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.020 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.020 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6818", 2024-04-20T01:47:51.020 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:47:51.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.020 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.020 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6819", 2024-04-20T01:47:51.020 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:47:51.020 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.021 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.021 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:51.021 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.021 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.021 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6822", 2024-04-20T01:47:51.021 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:47:51.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.021 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.022 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6823", 2024-04-20T01:47:51.022 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:47:51.022 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.022 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.022 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:51.022 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.022 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.022 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6820", 2024-04-20T01:47:51.022 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:47:51.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.023 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.023 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6821", 2024-04-20T01:47:51.023 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:47:51.023 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.023 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.023 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.121:6817/74361", 2024-04-20T01:47:51.023 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.121:6819/74361", 2024-04-20T01:47:51.023 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.121:6823/74361", 2024-04-20T01:47:51.023 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.121:6821/74361", 2024-04-20T01:47:51.023 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:51.023 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:51.023 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:51.024 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.024 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 2, 2024-04-20T01:47:51.024 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "7af325c1-36b5-420c-a2a9-44c00a002977", 2024-04-20T01:47:51.024 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:51.024 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:51.024 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:51.024 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:51.024 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:51.024 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:51.024 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:51.024 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:51.025 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:51.025 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:51.025 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:51.025 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.025 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.025 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6809", 2024-04-20T01:47:51.025 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:47:51.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.025 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.025 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6811", 2024-04-20T01:47:51.025 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:47:51.025 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.025 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.026 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:51.026 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.026 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.026 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6814", 2024-04-20T01:47:51.026 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:47:51.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.026 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.026 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6816", 2024-04-20T01:47:51.026 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:47:51.026 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.026 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6822", 2024-04-20T01:47:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:47:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6823", 2024-04-20T01:47:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:47:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6819", 2024-04-20T01:47:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:47:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6821", 2024-04-20T01:47:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:47:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.123:6811/43108", 2024-04-20T01:47:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.123:6816/43108", 2024-04-20T01:47:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.123:6823/43108", 2024-04-20T01:47:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.123:6821/43108", 2024-04-20T01:47:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 3, 2024-04-20T01:47:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "99334f43-83ea-4c3f-b91d-ae72c8361b8c", 2024-04-20T01:47:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:51.029 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:51.030 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:51.030 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:51.030 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:51.030 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:51.030 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:51.030 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:51.030 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:51.030 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:51.030 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.030 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.030 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6824", 2024-04-20T01:47:51.030 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:47:51.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.031 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.031 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6825", 2024-04-20T01:47:51.031 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:47:51.031 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.031 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.031 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:51.031 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.031 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.031 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6826", 2024-04-20T01:47:51.031 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:47:51.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.032 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.032 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6827", 2024-04-20T01:47:51.032 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:47:51.032 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.032 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.032 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:51.032 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.032 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.032 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6830", 2024-04-20T01:47:51.032 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:47:51.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.033 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.033 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6831", 2024-04-20T01:47:51.033 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:47:51.033 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.033 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.033 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:51.033 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.033 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.033 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6828", 2024-04-20T01:47:51.033 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:47:51.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.034 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.034 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6829", 2024-04-20T01:47:51.034 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:47:51.034 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.034 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.034 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.165:6825/74545", 2024-04-20T01:47:51.034 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.165:6827/74545", 2024-04-20T01:47:51.034 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.165:6831/74545", 2024-04-20T01:47:51.034 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.165:6829/74545", 2024-04-20T01:47:51.034 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:51.034 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:51.034 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:51.035 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.035 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 4, 2024-04-20T01:47:51.035 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "8764d799-af67-4e6c-889f-9fdd91201cc2", 2024-04-20T01:47:51.035 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:51.035 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:51.035 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:51.035 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:51.035 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:51.035 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:51.035 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:51.035 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:51.035 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:51.035 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:51.036 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:51.036 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.036 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.036 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6809", 2024-04-20T01:47:51.036 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:47:51.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.036 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.036 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6811", 2024-04-20T01:47:51.036 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:47:51.036 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.036 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.036 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:51.037 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.037 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.037 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6813", 2024-04-20T01:47:51.037 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:47:51.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.037 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.037 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6815", 2024-04-20T01:47:51.037 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:47:51.037 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.037 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.037 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:51.038 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.038 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.038 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6821", 2024-04-20T01:47:51.038 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:47:51.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.038 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.038 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6823", 2024-04-20T01:47:51.038 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:47:51.038 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.038 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.038 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:51.038 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.039 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.039 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6817", 2024-04-20T01:47:51.039 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:47:51.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.039 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.039 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6819", 2024-04-20T01:47:51.039 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:47:51.039 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.039 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.039 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6811/43517", 2024-04-20T01:47:51.039 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6815/43517", 2024-04-20T01:47:51.040 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6823/43517", 2024-04-20T01:47:51.040 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6819/43517", 2024-04-20T01:47:51.040 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:51.040 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:51.040 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:51.040 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.040 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 5, 2024-04-20T01:47:51.040 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "2556b2fa-4dcc-479c-a379-6f9fe96e9c77", 2024-04-20T01:47:51.040 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:51.040 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:51.040 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:51.040 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:51.040 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:51.041 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:51.041 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:51.041 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:51.041 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:51.041 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:51.041 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:51.041 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.041 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.041 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6800", 2024-04-20T01:47:51.041 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:47:51.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.041 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.041 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6801", 2024-04-20T01:47:51.042 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:47:51.042 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.042 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.042 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:51.042 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.042 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.042 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6802", 2024-04-20T01:47:51.042 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:47:51.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.042 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.042 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6803", 2024-04-20T01:47:51.043 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:47:51.043 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.043 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.043 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:51.043 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.043 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.043 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6806", 2024-04-20T01:47:51.043 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:47:51.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.043 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.043 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6807", 2024-04-20T01:47:51.043 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:47:51.044 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.044 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.044 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:51.044 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.044 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.044 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6804", 2024-04-20T01:47:51.044 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:47:51.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.044 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.044 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6805", 2024-04-20T01:47:51.044 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:47:51.045 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.045 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.045 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.121:6801/74354", 2024-04-20T01:47:51.045 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.121:6803/74354", 2024-04-20T01:47:51.045 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.121:6807/74354", 2024-04-20T01:47:51.045 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.121:6805/74354", 2024-04-20T01:47:51.045 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:51.045 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:51.045 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:51.045 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.045 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 6, 2024-04-20T01:47:51.045 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "baeaae3a-fff4-4ccc-a08a-ff32db4531ba", 2024-04-20T01:47:51.046 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:51.046 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:51.046 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:51.046 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:51.046 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:51.046 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:51.046 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:51.046 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:51.046 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:51.046 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:51.046 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:51.046 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.047 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.047 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6800", 2024-04-20T01:47:51.047 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:47:51.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.047 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.047 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6801", 2024-04-20T01:47:51.047 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:47:51.047 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.047 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.047 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:51.047 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.047 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.048 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6802", 2024-04-20T01:47:51.048 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:47:51.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.048 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.048 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6803", 2024-04-20T01:47:51.048 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:47:51.048 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.048 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.048 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:51.048 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.048 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.048 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6806", 2024-04-20T01:47:51.049 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:47:51.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.049 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.049 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6807", 2024-04-20T01:47:51.049 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:47:51.049 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.049 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.049 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:51.049 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.049 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.049 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6804", 2024-04-20T01:47:51.050 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:47:51.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.050 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.050 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6805", 2024-04-20T01:47:51.050 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:47:51.050 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.050 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.050 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.123:6801/43107", 2024-04-20T01:47:51.050 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.123:6803/43107", 2024-04-20T01:47:51.050 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.123:6807/43107", 2024-04-20T01:47:51.050 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.123:6805/43107", 2024-04-20T01:47:51.050 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:51.051 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:51.051 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:51.051 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.051 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 7, 2024-04-20T01:47:51.051 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "0019f005-3e62-4d9d-b334-f1d347a484ce", 2024-04-20T01:47:51.051 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:51.051 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:51.051 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:51.051 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:51.051 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:51.051 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:51.051 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:51.051 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:51.052 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:51.052 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:51.052 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:51.052 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.052 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.052 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6800", 2024-04-20T01:47:51.052 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:47:51.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.052 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.052 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6801", 2024-04-20T01:47:51.052 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:47:51.052 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.053 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.053 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:51.053 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.053 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.053 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6802", 2024-04-20T01:47:51.053 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:47:51.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.053 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.053 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6803", 2024-04-20T01:47:51.053 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:47:51.053 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.053 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.054 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:51.054 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.054 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.054 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6806", 2024-04-20T01:47:51.054 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:47:51.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.054 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.054 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6807", 2024-04-20T01:47:51.054 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:47:51.054 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.054 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.055 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:51.055 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.055 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.055 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6804", 2024-04-20T01:47:51.055 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:47:51.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.055 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.055 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6805", 2024-04-20T01:47:51.055 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:47:51.055 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.055 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.056 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.165:6801/74543", 2024-04-20T01:47:51.056 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.165:6803/74543", 2024-04-20T01:47:51.056 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.165:6807/74543", 2024-04-20T01:47:51.056 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.165:6805/74543", 2024-04-20T01:47:51.056 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:51.056 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:51.056 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:51.056 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.056 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.056 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 8, 2024-04-20T01:47:51.056 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "6f5200a3-8980-4241-a6fc-ef604fa612b2", 2024-04-20T01:47:51.056 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:51.056 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:51.057 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:51.057 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:51.057 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:51.057 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:51.057 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:51.057 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:51.057 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:51.057 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:51.057 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:51.057 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.057 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.057 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6808", 2024-04-20T01:47:51.057 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:47:51.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.058 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.058 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6810", 2024-04-20T01:47:51.058 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:47:51.058 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.058 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.058 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.058 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:51.058 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.058 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.058 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.058 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6812", 2024-04-20T01:47:51.058 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:47:51.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.059 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.059 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6814", 2024-04-20T01:47:51.059 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:47:51.059 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.059 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.059 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:51.059 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.059 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.059 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.059 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6820", 2024-04-20T01:47:51.059 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:47:51.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.060 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.060 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6822", 2024-04-20T01:47:51.060 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:47:51.060 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.060 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.060 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:51.060 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.060 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.060 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.060 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6816", 2024-04-20T01:47:51.060 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:47:51.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.061 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.061 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.061 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6818", 2024-04-20T01:47:51.061 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:47:51.061 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.061 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.061 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.061 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6810/43515", 2024-04-20T01:47:51.061 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6814/43515", 2024-04-20T01:47:51.061 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6822/43515", 2024-04-20T01:47:51.061 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6818/43515", 2024-04-20T01:47:51.061 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:51.061 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:51.061 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:51.062 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.062 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.062 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 9, 2024-04-20T01:47:51.062 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "29cf9aaa-3a99-4605-92f4-64f1f6a1728d", 2024-04-20T01:47:51.062 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:51.062 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:51.062 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:51.062 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:51.062 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:51.062 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:51.062 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:51.062 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 0, 2024-04-20T01:47:51.062 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:51.063 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:51.063 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:51.063 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.063 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.063 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6824", 2024-04-20T01:47:51.063 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:47:51.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.063 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.063 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.063 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6825", 2024-04-20T01:47:51.063 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:47:51.063 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.063 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.064 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:51.064 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.064 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.064 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6826", 2024-04-20T01:47:51.064 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:47:51.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.064 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.064 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.064 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6827", 2024-04-20T01:47:51.064 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:47:51.064 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.064 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.064 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.065 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:51.065 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.065 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.065 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6830", 2024-04-20T01:47:51.065 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:47:51.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.065 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.065 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.065 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6831", 2024-04-20T01:47:51.065 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:47:51.065 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.065 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.066 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:51.066 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.066 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.066 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6828", 2024-04-20T01:47:51.066 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:47:51.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.066 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.066 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.066 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6829", 2024-04-20T01:47:51.066 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:47:51.066 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.066 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.066 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.121:6825/74379", 2024-04-20T01:47:51.067 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.121:6827/74379", 2024-04-20T01:47:51.067 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.121:6831/74379", 2024-04-20T01:47:51.067 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.121:6829/74379", 2024-04-20T01:47:51.067 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:51.067 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:51.067 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:51.067 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.067 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.067 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 10, 2024-04-20T01:47:51.067 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "705906c4-1514-4754-ac40-45226a837180", 2024-04-20T01:47:51.067 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:51.067 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:51.067 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:51.068 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:51.068 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:51.068 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:51.068 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:51.068 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:51.068 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:51.068 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:51.068 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:51.068 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.068 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.068 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6808", 2024-04-20T01:47:51.068 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:47:51.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.068 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.069 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.069 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6810", 2024-04-20T01:47:51.069 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:47:51.069 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.069 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.069 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:51.069 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.069 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.069 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6812", 2024-04-20T01:47:51.069 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:47:51.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.069 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.070 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.070 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6813", 2024-04-20T01:47:51.070 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:47:51.070 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.070 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.070 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:51.070 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.070 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.070 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.070 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6818", 2024-04-20T01:47:51.070 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:47:51.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.070 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.074 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.074 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6820", 2024-04-20T01:47:51.074 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:47:51.074 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.074 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.074 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:51.074 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.074 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.074 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.074 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6815", 2024-04-20T01:47:51.074 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:47:51.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.075 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.075 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.075 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6817", 2024-04-20T01:47:51.075 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:47:51.075 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.075 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.075 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.075 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.123:6810/43109", 2024-04-20T01:47:51.075 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.123:6813/43109", 2024-04-20T01:47:51.075 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.123:6820/43109", 2024-04-20T01:47:51.075 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.123:6817/43109", 2024-04-20T01:47:51.075 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:51.075 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:51.075 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:51.076 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.076 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.076 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 11, 2024-04-20T01:47:51.076 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "b45f2645-72d1-4d72-89e9-2212f5aa074c", 2024-04-20T01:47:51.076 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:51.076 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:51.076 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:51.076 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:51.076 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:51.076 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:51.076 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:51.076 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:51.076 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:51.076 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:51.077 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:51.077 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.077 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.077 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6808", 2024-04-20T01:47:51.077 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:47:51.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.077 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.077 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6809", 2024-04-20T01:47:51.077 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:47:51.077 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.077 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.078 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:51.078 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.078 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.078 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6810", 2024-04-20T01:47:51.078 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:47:51.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.078 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.078 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.078 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6811", 2024-04-20T01:47:51.078 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:47:51.078 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.078 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.078 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.078 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:51.079 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.079 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.079 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6814", 2024-04-20T01:47:51.079 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:47:51.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.079 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.079 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.079 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6815", 2024-04-20T01:47:51.079 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:47:51.079 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.079 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.079 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:51.080 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.080 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.080 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6812", 2024-04-20T01:47:51.080 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:47:51.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.080 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.080 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.080 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6813", 2024-04-20T01:47:51.080 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:47:51.080 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.080 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.080 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.165:6809/74544", 2024-04-20T01:47:51.081 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.165:6811/74544", 2024-04-20T01:47:51.081 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.165:6815/74544", 2024-04-20T01:47:51.081 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.165:6813/74544", 2024-04-20T01:47:51.081 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:51.081 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:51.081 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:51.081 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.081 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.081 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.081 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 12, 2024-04-20T01:47:51.081 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "3ea410b9-a368-467b-a179-2e3aba562d63", 2024-04-20T01:47:51.081 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:51.081 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:51.081 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:51.081 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:51.082 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:51.082 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:51.082 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:51.082 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 0, 2024-04-20T01:47:51.082 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:51.082 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:51.082 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:51.082 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.082 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.082 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.082 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6800", 2024-04-20T01:47:51.082 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:47:51.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.082 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.083 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.083 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6801", 2024-04-20T01:47:51.083 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:47:51.083 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.083 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.083 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:51.083 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.083 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.083 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.083 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6802", 2024-04-20T01:47:51.083 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:47:51.083 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.083 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.083 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.084 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6803", 2024-04-20T01:47:51.084 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:47:51.084 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.084 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.084 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:51.084 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.084 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.084 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6806", 2024-04-20T01:47:51.084 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:47:51.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.084 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.084 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.085 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6807", 2024-04-20T01:47:51.085 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:47:51.085 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.085 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.085 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:51.085 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.085 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.085 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6804", 2024-04-20T01:47:51.085 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:47:51.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.085 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.085 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.086 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6805", 2024-04-20T01:47:51.086 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:47:51.086 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.086 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.086 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6801/43516", 2024-04-20T01:47:51.086 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6803/43516", 2024-04-20T01:47:51.086 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6807/43516", 2024-04-20T01:47:51.086 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6805/43516", 2024-04-20T01:47:51.086 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:51.086 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:51.086 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:51.086 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.086 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.087 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 13, 2024-04-20T01:47:51.087 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "15a5154b-e7cb-4b57-8468-dcb048d93f63", 2024-04-20T01:47:51.087 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:51.087 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:51.087 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:51.087 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:51.087 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:51.087 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:51.087 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 22, 2024-04-20T01:47:51.087 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:51.087 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:51.087 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:51.087 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:51.087 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.087 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.088 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.088 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6808", 2024-04-20T01:47:51.088 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:47:51.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.088 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.088 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6809", 2024-04-20T01:47:51.088 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:47:51.088 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.088 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.088 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.088 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:51.088 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.088 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.089 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.089 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6810", 2024-04-20T01:47:51.089 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:47:51.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.089 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.089 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6811", 2024-04-20T01:47:51.089 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:47:51.089 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.089 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.089 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:51.089 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.090 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.090 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6814", 2024-04-20T01:47:51.090 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:47:51.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.090 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.090 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6815", 2024-04-20T01:47:51.090 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:47:51.090 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.090 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.090 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:51.090 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.090 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.091 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.091 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6812", 2024-04-20T01:47:51.091 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:47:51.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.091 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.091 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6813", 2024-04-20T01:47:51.091 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:47:51.091 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.091 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.091 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.091 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.121:6809/74355", 2024-04-20T01:47:51.091 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.121:6811/74355", 2024-04-20T01:47:51.091 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.121:6815/74355", 2024-04-20T01:47:51.091 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.121:6813/74355", 2024-04-20T01:47:51.092 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:51.092 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:51.092 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:51.092 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.092 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.092 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 14, 2024-04-20T01:47:51.092 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "23b53da2-f848-4e76-9632-4d9effd5b430", 2024-04-20T01:47:51.092 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:51.092 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:51.092 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:51.092 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:51.092 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:51.092 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:51.092 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:51.093 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:51.093 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:51.093 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:51.093 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:51.093 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.093 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.093 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6824", 2024-04-20T01:47:51.093 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:47:51.093 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.093 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.093 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.093 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6825", 2024-04-20T01:47:51.093 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:47:51.094 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.094 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.094 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:51.094 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.094 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.094 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6826", 2024-04-20T01:47:51.094 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:47:51.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.094 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.094 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.094 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6827", 2024-04-20T01:47:51.094 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:47:51.094 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.095 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.095 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:51.095 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.095 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.095 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6830", 2024-04-20T01:47:51.095 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:47:51.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.095 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.095 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.095 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6831", 2024-04-20T01:47:51.095 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:47:51.095 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.095 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.096 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:51.096 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.096 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.096 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6828", 2024-04-20T01:47:51.096 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:47:51.096 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.096 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.096 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.096 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6829", 2024-04-20T01:47:51.096 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:47:51.096 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.096 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.097 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.123:6825/43110", 2024-04-20T01:47:51.097 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.123:6827/43110", 2024-04-20T01:47:51.097 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.123:6831/43110", 2024-04-20T01:47:51.097 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.123:6829/43110", 2024-04-20T01:47:51.097 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:51.097 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:51.097 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:51.097 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.097 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.097 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 15, 2024-04-20T01:47:51.097 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "546efc0f-82a3-47d2-aeec-2d849eb1faa1", 2024-04-20T01:47:51.097 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:51.097 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:51.098 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:51.098 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:51.098 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:51.098 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:51.098 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:51.098 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 0, 2024-04-20T01:47:51.098 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:51.098 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:51.098 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:51.098 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.098 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.098 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.098 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6816", 2024-04-20T01:47:51.098 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:47:51.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.099 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.099 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.099 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6817", 2024-04-20T01:47:51.099 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:47:51.099 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.099 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.099 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:51.099 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.099 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.099 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.099 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6818", 2024-04-20T01:47:51.099 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:47:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6819", 2024-04-20T01:47:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:47:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6822", 2024-04-20T01:47:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:47:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.100 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6823", 2024-04-20T01:47:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:47:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6820", 2024-04-20T01:47:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:47:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:51.101 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6821", 2024-04-20T01:47:51.102 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:47:51.102 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.102 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.102 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.165:6817/74562", 2024-04-20T01:47:51.102 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.165:6819/74562", 2024-04-20T01:47:51.102 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.165:6823/74562", 2024-04-20T01:47:51.102 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.165:6821/74562", 2024-04-20T01:47:51.102 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:51.102 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:51.102 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:51.102 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.102 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.102 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:47:51.103 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_xinfo": [ 2024-04-20T01:47:51.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.103 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 0, 2024-04-20T01:47:51.103 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:51.103 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:51.103 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:51.103 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:51.103 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:51.103 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.124928+0000", 2024-04-20T01:47:51.103 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:51.103 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.103 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.103 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 1, 2024-04-20T01:47:51.103 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:51.104 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:51.104 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:51.104 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:51.104 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:51.104 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.093711+0000", 2024-04-20T01:47:51.104 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:51.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.104 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.104 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 2, 2024-04-20T01:47:51.104 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:51.104 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:51.104 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:51.104 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:51.104 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:51.105 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.008326+0000", 2024-04-20T01:47:51.105 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:51.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.105 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.105 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 3, 2024-04-20T01:47:51.105 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:51.105 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:51.105 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:51.105 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:51.105 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:51.105 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.085397+0000", 2024-04-20T01:47:51.105 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:51.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.105 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.105 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 4, 2024-04-20T01:47:51.106 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:51.106 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:51.106 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:51.106 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:51.106 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:51.106 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.117767+0000", 2024-04-20T01:47:51.106 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:51.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.106 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 5, 2024-04-20T01:47:51.106 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:51.106 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:51.106 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:51.106 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:51.107 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:51.107 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:12.999445+0000", 2024-04-20T01:47:51.107 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:51.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.107 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.107 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 6, 2024-04-20T01:47:51.107 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:51.107 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:51.107 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:51.107 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:51.107 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:51.107 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.032367+0000", 2024-04-20T01:47:51.107 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:51.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.108 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 7, 2024-04-20T01:47:51.108 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:51.108 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:51.108 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:51.108 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:51.108 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:51.108 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.058776+0000", 2024-04-20T01:47:51.108 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:51.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.108 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.108 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 8, 2024-04-20T01:47:51.108 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:51.108 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:51.108 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:51.109 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:51.109 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:51.109 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.113013+0000", 2024-04-20T01:47:51.109 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:51.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.109 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.109 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 9, 2024-04-20T01:47:51.109 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:51.109 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:51.109 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:51.109 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:51.109 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:51.109 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.110714+0000", 2024-04-20T01:47:51.109 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:51.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.110 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 10, 2024-04-20T01:47:51.110 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:51.110 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:51.110 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:51.110 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:51.110 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:51.110 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.092533+0000", 2024-04-20T01:47:51.110 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:51.110 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.110 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.110 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 11, 2024-04-20T01:47:51.111 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:51.111 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:51.111 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:51.111 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:51.111 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:51.111 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.083135+0000", 2024-04-20T01:47:51.111 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:51.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.111 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 12, 2024-04-20T01:47:51.111 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:51.111 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:51.111 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:51.111 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:51.111 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:51.111 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.094847+0000", 2024-04-20T01:47:51.112 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:51.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.112 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 13, 2024-04-20T01:47:51.112 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:51.112 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:51.112 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:51.112 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:51.112 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:51.112 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.059856+0000", 2024-04-20T01:47:51.112 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:51.112 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.112 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.112 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 14, 2024-04-20T01:47:51.112 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:51.113 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:51.113 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:51.113 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:51.113 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:51.113 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.128995+0000", 2024-04-20T01:47:51.113 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:51.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.113 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 15, 2024-04-20T01:47:51.113 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:51.113 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:51.113 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:51.113 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:51.113 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:51.114 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.061883+0000", 2024-04-20T01:47:51.114 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:51.114 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.114 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:47:51.114 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap": [], 2024-04-20T01:47:51.114 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap_items": [], 2024-04-20T01:47:51.114 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap_primaries": [], 2024-04-20T01:47:51.114 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_temp": [], 2024-04-20T01:47:51.114 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_temp": [], 2024-04-20T01:47:51.114 INFO:teuthology.orchestra.run.smithi044.stdout: "blocklist": {}, 2024-04-20T01:47:51.114 INFO:teuthology.orchestra.run.smithi044.stdout: "range_blocklist": {}, 2024-04-20T01:47:51.114 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profiles": { 2024-04-20T01:47:51.114 INFO:teuthology.orchestra.run.smithi044.stdout: "default": { 2024-04-20T01:47:51.114 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-failure-domain": "osd", 2024-04-20T01:47:51.114 INFO:teuthology.orchestra.run.smithi044.stdout: "k": "2", 2024-04-20T01:47:51.115 INFO:teuthology.orchestra.run.smithi044.stdout: "m": "1", 2024-04-20T01:47:51.115 INFO:teuthology.orchestra.run.smithi044.stdout: "plugin": "jerasure", 2024-04-20T01:47:51.115 INFO:teuthology.orchestra.run.smithi044.stdout: "technique": "reed_sol_van" 2024-04-20T01:47:51.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.115 INFO:teuthology.orchestra.run.smithi044.stdout: "shecprofile": { 2024-04-20T01:47:51.115 INFO:teuthology.orchestra.run.smithi044.stdout: "c": "2", 2024-04-20T01:47:51.115 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-device-class": "", 2024-04-20T01:47:51.115 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-failure-domain": "osd", 2024-04-20T01:47:51.115 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-root": "default", 2024-04-20T01:47:51.115 INFO:teuthology.orchestra.run.smithi044.stdout: "k": "4", 2024-04-20T01:47:51.115 INFO:teuthology.orchestra.run.smithi044.stdout: "m": "3", 2024-04-20T01:47:51.115 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "shecprofile", 2024-04-20T01:47:51.115 INFO:teuthology.orchestra.run.smithi044.stdout: "plugin": "shec", 2024-04-20T01:47:51.115 INFO:teuthology.orchestra.run.smithi044.stdout: "technique": "multiple" 2024-04-20T01:47:51.116 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.116 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps_queue": [ 2024-04-20T01:47:51.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.116 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 3, 2024-04-20T01:47:51.116 INFO:teuthology.orchestra.run.smithi044.stdout: "snaps": [ 2024-04-20T01:47:51.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.116 INFO:teuthology.orchestra.run.smithi044.stdout: "begin": 2, 2024-04-20T01:47:51.116 INFO:teuthology.orchestra.run.smithi044.stdout: "length": 1 2024-04-20T01:47:51.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.116 INFO:teuthology.orchestra.run.smithi044.stdout: "begin": 7, 2024-04-20T01:47:51.116 INFO:teuthology.orchestra.run.smithi044.stdout: "length": 1 2024-04-20T01:47:51.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.117 INFO:teuthology.orchestra.run.smithi044.stdout: "begin": 10, 2024-04-20T01:47:51.117 INFO:teuthology.orchestra.run.smithi044.stdout: "length": 1 2024-04-20T01:47:51.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:51.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:51.117 INFO:teuthology.orchestra.run.smithi044.stdout: "begin": 12, 2024-04-20T01:47:51.117 INFO:teuthology.orchestra.run.smithi044.stdout: "length": 2 2024-04-20T01:47:51.117 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.117 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:51.117 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.117 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:47:51.117 INFO:teuthology.orchestra.run.smithi044.stdout: "new_removed_snaps": [], 2024-04-20T01:47:51.117 INFO:teuthology.orchestra.run.smithi044.stdout: "new_purged_snaps": [], 2024-04-20T01:47:51.117 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_node_flags": {}, 2024-04-20T01:47:51.117 INFO:teuthology.orchestra.run.smithi044.stdout: "device_class_flags": {}, 2024-04-20T01:47:51.118 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": { 2024-04-20T01:47:51.118 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode_enabled": false, 2024-04-20T01:47:51.118 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_bucket_count": 0, 2024-04-20T01:47:51.118 INFO:teuthology.orchestra.run.smithi044.stdout: "degraded_stretch_mode": 0, 2024-04-20T01:47:51.118 INFO:teuthology.orchestra.run.smithi044.stdout: "recovering_stretch_mode": 0, 2024-04-20T01:47:51.118 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode_bucket": 0 2024-04-20T01:47:51.118 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:51.118 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:51.120 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:51.120 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:47:51.120 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:47:51.121 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:51.122 INFO:tasks.thrashosds.thrasher:j is {'epoch': 49, 'fsid': '9c2f64c1-aa35-4337-9f5c-081b9e3a2938', 'created': '2024-04-20T01:47:02.934241+0000', 'modified': '2024-04-20T01:47:50.704874+0000', 'last_up_change': '2024-04-20T01:47:15.986754+0000', 'last_in_change': '2024-04-20T01:47:07.894719+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T01:47:15.195181+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 15.380000114440918, 'score_stable': 15.380000114440918, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-04-20T01:47:18.448004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 3.990000009536743, 'score_stable': 3.990000009536743, 'optimal_score': 0.6899999976158142, 'raw_score_acting': 2.75, 'raw_score_stable': 2.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-04-20T01:47:30.734962+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 7, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 49, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'shecprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '3903b88f-ff2b-4252-9480-92b701b1e59b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6824', 'nonce': 43518}, {'type': 'v1', 'addr': '172.21.15.44:6825', 'nonce': 43518}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6826', 'nonce': 43518}, {'type': 'v1', 'addr': '172.21.15.44:6827', 'nonce': 43518}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6830', 'nonce': 43518}, {'type': 'v1', 'addr': '172.21.15.44:6831', 'nonce': 43518}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6828', 'nonce': 43518}, {'type': 'v1', 'addr': '172.21.15.44:6829', 'nonce': 43518}]}, 'public_addr': '172.21.15.44:6825/43518', 'cluster_addr': '172.21.15.44:6827/43518', 'heartbeat_back_addr': '172.21.15.44:6831/43518', 'heartbeat_front_addr': '172.21.15.44:6829/43518', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c5ff0175-b4f0-40a5-86c7-7a0a351bf070', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6816', 'nonce': 74361}, {'type': 'v1', 'addr': '172.21.15.121:6817', 'nonce': 74361}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6818', 'nonce': 74361}, {'type': 'v1', 'addr': '172.21.15.121:6819', 'nonce': 74361}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6822', 'nonce': 74361}, {'type': 'v1', 'addr': '172.21.15.121:6823', 'nonce': 74361}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6820', 'nonce': 74361}, {'type': 'v1', 'addr': '172.21.15.121:6821', 'nonce': 74361}]}, 'public_addr': '172.21.15.121:6817/74361', 'cluster_addr': '172.21.15.121:6819/74361', 'heartbeat_back_addr': '172.21.15.121:6823/74361', 'heartbeat_front_addr': '172.21.15.121:6821/74361', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '7af325c1-36b5-420c-a2a9-44c00a002977', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6809', 'nonce': 43108}, {'type': 'v1', 'addr': '172.21.15.123:6811', 'nonce': 43108}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6814', 'nonce': 43108}, {'type': 'v1', 'addr': '172.21.15.123:6816', 'nonce': 43108}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6822', 'nonce': 43108}, {'type': 'v1', 'addr': '172.21.15.123:6823', 'nonce': 43108}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6819', 'nonce': 43108}, {'type': 'v1', 'addr': '172.21.15.123:6821', 'nonce': 43108}]}, 'public_addr': '172.21.15.123:6811/43108', 'cluster_addr': '172.21.15.123:6816/43108', 'heartbeat_back_addr': '172.21.15.123:6823/43108', 'heartbeat_front_addr': '172.21.15.123:6821/43108', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '99334f43-83ea-4c3f-b91d-ae72c8361b8c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6824', 'nonce': 74545}, {'type': 'v1', 'addr': '172.21.15.165:6825', 'nonce': 74545}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6826', 'nonce': 74545}, {'type': 'v1', 'addr': '172.21.15.165:6827', 'nonce': 74545}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6830', 'nonce': 74545}, {'type': 'v1', 'addr': '172.21.15.165:6831', 'nonce': 74545}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6828', 'nonce': 74545}, {'type': 'v1', 'addr': '172.21.15.165:6829', 'nonce': 74545}]}, 'public_addr': '172.21.15.165:6825/74545', 'cluster_addr': '172.21.15.165:6827/74545', 'heartbeat_back_addr': '172.21.15.165:6831/74545', 'heartbeat_front_addr': '172.21.15.165:6829/74545', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8764d799-af67-4e6c-889f-9fdd91201cc2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6809', 'nonce': 43517}, {'type': 'v1', 'addr': '172.21.15.44:6811', 'nonce': 43517}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6813', 'nonce': 43517}, {'type': 'v1', 'addr': '172.21.15.44:6815', 'nonce': 43517}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6821', 'nonce': 43517}, {'type': 'v1', 'addr': '172.21.15.44:6823', 'nonce': 43517}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6817', 'nonce': 43517}, {'type': 'v1', 'addr': '172.21.15.44:6819', 'nonce': 43517}]}, 'public_addr': '172.21.15.44:6811/43517', 'cluster_addr': '172.21.15.44:6815/43517', 'heartbeat_back_addr': '172.21.15.44:6823/43517', 'heartbeat_front_addr': '172.21.15.44:6819/43517', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '2556b2fa-4dcc-479c-a379-6f9fe96e9c77', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6800', 'nonce': 74354}, {'type': 'v1', 'addr': '172.21.15.121:6801', 'nonce': 74354}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6802', 'nonce': 74354}, {'type': 'v1', 'addr': '172.21.15.121:6803', 'nonce': 74354}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6806', 'nonce': 74354}, {'type': 'v1', 'addr': '172.21.15.121:6807', 'nonce': 74354}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6804', 'nonce': 74354}, {'type': 'v1', 'addr': '172.21.15.121:6805', 'nonce': 74354}]}, 'public_addr': '172.21.15.121:6801/74354', 'cluster_addr': '172.21.15.121:6803/74354', 'heartbeat_back_addr': '172.21.15.121:6807/74354', 'heartbeat_front_addr': '172.21.15.121:6805/74354', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'baeaae3a-fff4-4ccc-a08a-ff32db4531ba', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6800', 'nonce': 43107}, {'type': 'v1', 'addr': '172.21.15.123:6801', 'nonce': 43107}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6802', 'nonce': 43107}, {'type': 'v1', 'addr': '172.21.15.123:6803', 'nonce': 43107}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6806', 'nonce': 43107}, {'type': 'v1', 'addr': '172.21.15.123:6807', 'nonce': 43107}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6804', 'nonce': 43107}, {'type': 'v1', 'addr': '172.21.15.123:6805', 'nonce': 43107}]}, 'public_addr': '172.21.15.123:6801/43107', 'cluster_addr': '172.21.15.123:6803/43107', 'heartbeat_back_addr': '172.21.15.123:6807/43107', 'heartbeat_front_addr': '172.21.15.123:6805/43107', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '0019f005-3e62-4d9d-b334-f1d347a484ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6800', 'nonce': 74543}, {'type': 'v1', 'addr': '172.21.15.165:6801', 'nonce': 74543}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6802', 'nonce': 74543}, {'type': 'v1', 'addr': '172.21.15.165:6803', 'nonce': 74543}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6806', 'nonce': 74543}, {'type': 'v1', 'addr': '172.21.15.165:6807', 'nonce': 74543}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6804', 'nonce': 74543}, {'type': 'v1', 'addr': '172.21.15.165:6805', 'nonce': 74543}]}, 'public_addr': '172.21.15.165:6801/74543', 'cluster_addr': '172.21.15.165:6803/74543', 'heartbeat_back_addr': '172.21.15.165:6807/74543', 'heartbeat_front_addr': '172.21.15.165:6805/74543', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '6f5200a3-8980-4241-a6fc-ef604fa612b2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6808', 'nonce': 43515}, {'type': 'v1', 'addr': '172.21.15.44:6810', 'nonce': 43515}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6812', 'nonce': 43515}, {'type': 'v1', 'addr': '172.21.15.44:6814', 'nonce': 43515}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6820', 'nonce': 43515}, {'type': 'v1', 'addr': '172.21.15.44:6822', 'nonce': 43515}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6816', 'nonce': 43515}, {'type': 'v1', 'addr': '172.21.15.44:6818', 'nonce': 43515}]}, 'public_addr': '172.21.15.44:6810/43515', 'cluster_addr': '172.21.15.44:6814/43515', 'heartbeat_back_addr': '172.21.15.44:6822/43515', 'heartbeat_front_addr': '172.21.15.44:6818/43515', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '29cf9aaa-3a99-4605-92f4-64f1f6a1728d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6824', 'nonce': 74379}, {'type': 'v1', 'addr': '172.21.15.121:6825', 'nonce': 74379}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6826', 'nonce': 74379}, {'type': 'v1', 'addr': '172.21.15.121:6827', 'nonce': 74379}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6830', 'nonce': 74379}, {'type': 'v1', 'addr': '172.21.15.121:6831', 'nonce': 74379}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6828', 'nonce': 74379}, {'type': 'v1', 'addr': '172.21.15.121:6829', 'nonce': 74379}]}, 'public_addr': '172.21.15.121:6825/74379', 'cluster_addr': '172.21.15.121:6827/74379', 'heartbeat_back_addr': '172.21.15.121:6831/74379', 'heartbeat_front_addr': '172.21.15.121:6829/74379', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '705906c4-1514-4754-ac40-45226a837180', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6808', 'nonce': 43109}, {'type': 'v1', 'addr': '172.21.15.123:6810', 'nonce': 43109}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6812', 'nonce': 43109}, {'type': 'v1', 'addr': '172.21.15.123:6813', 'nonce': 43109}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6818', 'nonce': 43109}, {'type': 'v1', 'addr': '172.21.15.123:6820', 'nonce': 43109}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6815', 'nonce': 43109}, {'type': 'v1', 'addr': '172.21.15.123:6817', 'nonce': 43109}]}, 'public_addr': '172.21.15.123:6810/43109', 'cluster_addr': '172.21.15.123:6813/43109', 'heartbeat_back_addr': '172.21.15.123:6820/43109', 'heartbeat_front_addr': '172.21.15.123:6817/43109', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': 'b45f2645-72d1-4d72-89e9-2212f5aa074c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6808', 'nonce': 74544}, {'type': 'v1', 'addr': '172.21.15.165:6809', 'nonce': 74544}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6810', 'nonce': 74544}, {'type': 'v1', 'addr': '172.21.15.165:6811', 'nonce': 74544}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6814', 'nonce': 74544}, {'type': 'v1', 'addr': '172.21.15.165:6815', 'nonce': 74544}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6812', 'nonce': 74544}, {'type': 'v1', 'addr': '172.21.15.165:6813', 'nonce': 74544}]}, 'public_addr': '172.21.15.165:6809/74544', 'cluster_addr': '172.21.15.165:6811/74544', 'heartbeat_back_addr': '172.21.15.165:6815/74544', 'heartbeat_front_addr': '172.21.15.165:6813/74544', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': '3ea410b9-a368-467b-a179-2e3aba562d63', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6800', 'nonce': 43516}, {'type': 'v1', 'addr': '172.21.15.44:6801', 'nonce': 43516}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6802', 'nonce': 43516}, {'type': 'v1', 'addr': '172.21.15.44:6803', 'nonce': 43516}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6806', 'nonce': 43516}, {'type': 'v1', 'addr': '172.21.15.44:6807', 'nonce': 43516}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6804', 'nonce': 43516}, {'type': 'v1', 'addr': '172.21.15.44:6805', 'nonce': 43516}]}, 'public_addr': '172.21.15.44:6801/43516', 'cluster_addr': '172.21.15.44:6803/43516', 'heartbeat_back_addr': '172.21.15.44:6807/43516', 'heartbeat_front_addr': '172.21.15.44:6805/43516', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': '15a5154b-e7cb-4b57-8468-dcb048d93f63', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6808', 'nonce': 74355}, {'type': 'v1', 'addr': '172.21.15.121:6809', 'nonce': 74355}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6810', 'nonce': 74355}, {'type': 'v1', 'addr': '172.21.15.121:6811', 'nonce': 74355}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6814', 'nonce': 74355}, {'type': 'v1', 'addr': '172.21.15.121:6815', 'nonce': 74355}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6812', 'nonce': 74355}, {'type': 'v1', 'addr': '172.21.15.121:6813', 'nonce': 74355}]}, 'public_addr': '172.21.15.121:6809/74355', 'cluster_addr': '172.21.15.121:6811/74355', 'heartbeat_back_addr': '172.21.15.121:6815/74355', 'heartbeat_front_addr': '172.21.15.121:6813/74355', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '23b53da2-f848-4e76-9632-4d9effd5b430', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6824', 'nonce': 43110}, {'type': 'v1', 'addr': '172.21.15.123:6825', 'nonce': 43110}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6826', 'nonce': 43110}, {'type': 'v1', 'addr': '172.21.15.123:6827', 'nonce': 43110}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6830', 'nonce': 43110}, {'type': 'v1', 'addr': '172.21.15.123:6831', 'nonce': 43110}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6828', 'nonce': 43110}, {'type': 'v1', 'addr': '172.21.15.123:6829', 'nonce': 43110}]}, 'public_addr': '172.21.15.123:6825/43110', 'cluster_addr': '172.21.15.123:6827/43110', 'heartbeat_back_addr': '172.21.15.123:6831/43110', 'heartbeat_front_addr': '172.21.15.123:6829/43110', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': '546efc0f-82a3-47d2-aeec-2d849eb1faa1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6816', 'nonce': 74562}, {'type': 'v1', 'addr': '172.21.15.165:6817', 'nonce': 74562}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6818', 'nonce': 74562}, {'type': 'v1', 'addr': '172.21.15.165:6819', 'nonce': 74562}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6822', 'nonce': 74562}, {'type': 'v1', 'addr': '172.21.15.165:6823', 'nonce': 74562}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6820', 'nonce': 74562}, {'type': 'v1', 'addr': '172.21.15.165:6821', 'nonce': 74562}]}, 'public_addr': '172.21.15.165:6817/74562', 'cluster_addr': '172.21.15.165:6819/74562', 'heartbeat_back_addr': '172.21.15.165:6823/74562', 'heartbeat_front_addr': '172.21.15.165:6821/74562', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.124928+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.093711+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.008326+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.085397+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.117767+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:12.999445+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.032367+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.058776+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.113013+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.110714+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.092533+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.083135+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.094847+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.059856+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.128995+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.061883+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'shecprofile': {'c': '2', 'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'k': '4', 'm': '3', 'name': 'shecprofile', 'plugin': 'shec', 'technique': 'multiple'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}, {'begin': 7, 'length': 1}, {'begin': 10, 'length': 1}, {'begin': 12, 'length': 2}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-20T01:47:51.122 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T01:47:51.123 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-20T01:47:51.123 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:51.134+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:51.124 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:47:51.124 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:47:51.124 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:47:51.124 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:47:51.124 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:47:51.124 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:47:51.138 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:47:51.159 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:47:51.181 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:47:51.201 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:47:51.220 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:51.235+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:51.222 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:47:51.242 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:47:51.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:47:51.258+0000 7f20aa180700 0 -- 172.21.15.44:0/3752528180 >> [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] conn(0x7f20a421f540 msgr2=0x7f20a4221930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-20T01:47:51.288 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:51.288 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:51.298 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-20T01:47:51.321 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:51.336+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:51.422 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:51.436+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:47:51.425 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:51.425 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:51.435 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-20T01:47:51.484 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:47:51.486 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:47:51.486 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:47:51.486 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:47:51.486 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:47:51.486 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:47:51.486 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:47:51.486 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:47:51.486 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:47:51.486 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:47:51.486 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:47:51.487 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:47:51.487 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:47:51.487 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:47:51.487 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:47:51.487 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:47:51.487 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:47:51.516 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:47:51.518 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T01:47:51.523 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:51.538+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:47:51.530 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2024-04-20T01:47:51.142904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":40141344,"num_objects":60,"num_object_clones":15,"num_object_copies":410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":3,"num_read":381,"num_read_kb":38338,"num_write":631,"num_write_kb":49796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":532,"ondisk_log_size":532,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":514200,"kb_used_data":81820,"kb_used_omap":24,"kb_used_meta":432103,"kb_avail":1509435240,"statfs":{"total":1546188226560,"available":1545661685760,"internally_reserved":0,"allocated":83783680,"data_stored":96092204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25507,"internal_metadata":442473565},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1409024,"num_objects":-5,"num_object_clones":-3,"num_object_copies":-35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":-1,"num_read":21,"num_read_kb":7940,"num_write":39,"num_write_kb":3424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000903"},"pg_stats":[{"pgid":"3.f","version":"41'12","reported_seq":86,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-20T01:47:49.715730+0000","last_change":"2024-04-20T01:47:49.715730+0000","last_active":"2024-04-20T01:47:49.715730+0000","last_peered":"2024-04-20T01:47:49.715730+0000","last_clean":"2024-04-20T01:47:49.715730+0000","last_became_active":"2024-04-20T01:47:32.867228+0000","last_became_peered":"2024-04-20T01:47:32.867228+0000","last_unstale":"2024-04-20T01:47:49.715730+0000","last_undegraded":"2024-04-20T01:47:49.715730+0000","last_fullsized":"2024-04-20T01:47:49.715730+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.328818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064634000000000002,"stat_sum":{"num_bytes":1228800,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":5142,"num_write":47,"num_write_kb":1488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,15,13,8,11,0],"acting":[2,3,15,13,8,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"1"},{"start":"a","length":"1"},{"start":"c","length":"2"}]},{"pgid":"3.e","version":"47'23","reported_seq":99,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-20T01:47:49.708827+0000","last_change":"2024-04-20T01:47:49.708827+0000","last_active":"2024-04-20T01:47:49.708827+0000","last_peered":"2024-04-20T01:47:49.708827+0000","last_clean":"2024-04-20T01:47:49.708827+0000","last_became_active":"2024-04-20T01:47:32.867409+0000","last_became_peered":"2024-04-20T01:47:32.867409+0000","last_unstale":"2024-04-20T01:47:49.708827+0000","last_undegraded":"2024-04-20T01:47:49.708827+0000","last_fullsized":"2024-04-20T01:47:49.708827+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:42.581217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048063400000000002,"stat_sum":{"num_bytes":802816,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":2312,"num_write":13,"num_write_kb":3280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,1,10,8,3,12],"acting":[11,9,1,10,8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"1"},{"start":"a","length":"1"},{"start":"c","length":"2"}]},{"pgid":"3.d","version":"41'12","reported_seq":72,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-20T01:47:49.715420+0000","last_change":"2024-04-20T01:47:49.715750+0000","last_active":"2024-04-20T01:47:49.715420+0000","last_peered":"2024-04-20T01:47:49.715420+0000","last_clean":"2024-04-20T01:47:49.715420+0000","last_became_active":"2024-04-20T01:47:32.867954+0000","last_became_peered":"2024-04-20T01:47:32.867954+0000","last_unstale":"2024-04-20T01:47:49.715420+0000","last_undegraded":"2024-04-20T01:47:49.715420+0000","last_fullsized":"2024-04-20T01:47:49.715420+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.083554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048058699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,11,15,7,8,4],"acting":[6,2,11,15,7,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"1"},{"start":"a","length":"1"},{"start":"c","length":"2"}]},{"pgid":"3.c","version":"48'36","reported_seq":97,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-20T01:47:49.726732+0000","last_change":"2024-04-20T01:47:49.726732+0000","last_active":"2024-04-20T01:47:49.726732+0000","last_peered":"2024-04-20T01:47:49.726732+0000","last_clean":"2024-04-20T01:47:49.726732+0000","last_became_active":"2024-04-20T01:47:32.867926+0000","last_became_peered":"2024-04-20T01:47:32.867926+0000","last_unstale":"2024-04-20T01:47:49.726732+0000","last_undegraded":"2024-04-20T01:47:49.726732+0000","last_fullsized":"2024-04-20T01:47:49.726732+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:54.374528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014313243999999999,"stat_sum":{"num_bytes":5652480,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":40,"num_write_kb":5520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,8,6,3,7,12,5],"acting":[14,8,6,3,7,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"1"},{"start":"a","length":"1"},{"start":"c","length":"2"}]},{"pgid":"3.b","version":"48'53","reported_seq":131,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-20T01:47:49.789795+0000","last_change":"2024-04-20T01:47:49.733776+0000","last_active":"2024-04-20T01:47:49.789795+0000","last_peered":"2024-04-20T01:47:49.789795+0000","last_clean":"2024-04-20T01:47:49.789795+0000","last_became_active":"2024-04-20T01:47:32.867309+0000","last_became_peered":"2024-04-20T01:47:32.867309+0000","last_unstale":"2024-04-20T01:47:49.789795+0000","last_undegraded":"2024-04-20T01:47:49.789795+0000","last_fullsized":"2024-04-20T01:47:49.789795+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:01.227735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019404819,"stat_sum":{"num_bytes":6799360,"num_objects":8,"num_object_clones":2,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":36,"num_read_kb":5390,"num_write":93,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,11,10,9],"acting":[4,0,1,12,11,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"1"},{"start":"a","length":"1"},{"start":"c","length":"2"}]},{"pgid":"3.a","version":"48'14","reported_seq":77,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-20T01:47:49.725920+0000","last_change":"2024-04-20T01:47:49.726725+0000","last_active":"2024-04-20T01:47:49.725920+0000","last_peered":"2024-04-20T01:47:49.725920+0000","last_clean":"2024-04-20T01:47:49.725920+0000","last_became_active":"2024-04-20T01:47:32.868002+0000","last_became_peered":"2024-04-20T01:47:32.868002+0000","last_unstale":"2024-04-20T01:47:49.725920+0000","last_undegraded":"2024-04-20T01:47:49.725920+0000","last_fullsized":"2024-04-20T01:47:49.725920+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:35.716198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011174864,"stat_sum":{"num_bytes":1736704,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":8,"num_read_kb":1971,"num_write":6,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,1,14,3,15,2],"acting":[6,10,1,14,3,15,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"1"},{"start":"a","length":"1"},{"start":"c","length":"2"}]},{"pgid":"3.9","version":"48'14","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-20T01:47:49.723596+0000","last_change":"2024-04-20T01:47:49.723596+0000","last_active":"2024-04-20T01:47:49.723596+0000","last_peered":"2024-04-20T01:47:49.723596+0000","last_clean":"2024-04-20T01:47:49.723596+0000","last_became_active":"2024-04-20T01:47:32.866706+0000","last_became_peered":"2024-04-20T01:47:32.866706+0000","last_unstale":"2024-04-20T01:47:49.723596+0000","last_undegraded":"2024-04-20T01:47:49.723596+0000","last_fullsized":"2024-04-20T01:47:49.723596+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:42.776206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0094887670000000004,"stat_sum":{"num_bytes":1097728,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":10,"num_read_kb":1251,"num_write":7,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0,11,14,8,3],"acting":[7,1,0,11,14,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"1"},{"start":"a","length":"1"},{"start":"c","length":"2"}]},{"pgid":"3.8","version":"48'33","reported_seq":102,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-20T01:47:49.725952+0000","last_change":"2024-04-20T01:47:49.726515+0000","last_active":"2024-04-20T01:47:49.725952+0000","last_peered":"2024-04-20T01:47:49.725952+0000","last_clean":"2024-04-20T01:47:49.725952+0000","last_became_active":"2024-04-20T01:47:32.718069+0000","last_became_peered":"2024-04-20T01:47:32.718069+0000","last_unstale":"2024-04-20T01:47:49.725952+0000","last_undegraded":"2024-04-20T01:47:49.725952+0000","last_fullsized":"2024-04-20T01:47:49.725952+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:58.489885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011437699000000001,"stat_sum":{"num_bytes":4751360,"num_objects":5,"num_object_clones":1,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":17,"num_read_kb":536,"num_write":21,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,11,3,14,2,7],"acting":[5,0,11,3,14,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"1"},{"start":"a","length":"1"},{"start":"c","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-20T01:47:49.716280+0000","last_change":"2024-04-20T01:47:20.071988+0000","last_active":"2024-04-20T01:47:49.716280+0000","last_peered":"2024-04-20T01:47:49.716280+0000","last_clean":"2024-04-20T01:47:49.716280+0000","last_became_active":"2024-04-20T01:47:20.071615+0000","last_became_peered":"2024-04-20T01:47:20.071615+0000","last_unstale":"2024-04-20T01:47:49.716280+0000","last_undegraded":"2024-04-20T01:47:49.716280+0000","last_fullsized":"2024-04-20T01:47:49.716280+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:40.106777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"41'5","reported_seq":70,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-20T01:47:49.792777+0000","last_change":"2024-04-20T01:47:49.718469+0000","last_active":"2024-04-20T01:47:49.792777+0000","last_peered":"2024-04-20T01:47:49.792777+0000","last_clean":"2024-04-20T01:47:49.792777+0000","last_became_active":"2024-04-20T01:47:32.867180+0000","last_became_peered":"2024-04-20T01:47:32.867180+0000","last_unstale":"2024-04-20T01:47:49.792777+0000","last_undegraded":"2024-04-20T01:47:49.792777+0000","last_fullsized":"2024-04-20T01:47:49.792777+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:43.358783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044367499999999998,"stat_sum":{"num_bytes":360448,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":1059,"num_write":5,"num_write_kb":416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,4,14,12,0,11],"acting":[10,8,4,14,12,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"1"},{"start":"a","length":"1"},{"start":"c","length":"2"}]},{"pgid":"2.2","version":"0'0","reported_seq":32,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-20T01:47:49.715004+0000","last_change":"2024-04-20T01:47:20.075613+0000","last_active":"2024-04-20T01:47:49.715004+0000","last_peered":"2024-04-20T01:47:49.715004+0000","last_clean":"2024-04-20T01:47:49.715004+0000","last_became_active":"2024-04-20T01:47:20.075314+0000","last_became_peered":"2024-04-20T01:47:20.075314+0000","last_unstale":"2024-04-20T01:47:49.715004+0000","last_undegraded":"2024-04-20T01:47:49.715004+0000","last_fullsized":"2024-04-20T01:47:49.715004+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:35.956173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,12],"acting":[2,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"44'18","reported_seq":66,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.845417+0000","last_change":"2024-04-20T01:47:42.809609+0000","last_active":"2024-04-20T01:47:45.845417+0000","last_peered":"2024-04-20T01:47:45.845417+0000","last_clean":"2024-04-20T01:47:45.845417+0000","last_became_active":"2024-04-20T01:47:32.719136+0000","last_became_peered":"2024-04-20T01:47:32.719136+0000","last_unstale":"2024-04-20T01:47:45.845417+0000","last_undegraded":"2024-04-20T01:47:45.845417+0000","last_fullsized":"2024-04-20T01:47:45.845417+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:46.940575+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0089564480000000005,"stat_sum":{"num_bytes":1540096,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":3,"num_write":30,"num_write_kb":2128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,4,14,11,5,10],"acting":[13,2,4,14,11,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.829483+0000","last_change":"2024-04-20T01:47:20.039221+0000","last_active":"2024-04-20T01:47:45.829483+0000","last_peered":"2024-04-20T01:47:45.829483+0000","last_clean":"2024-04-20T01:47:45.829483+0000","last_became_active":"2024-04-20T01:47:20.038972+0000","last_became_peered":"2024-04-20T01:47:20.038972+0000","last_unstale":"2024-04-20T01:47:45.829483+0000","last_undegraded":"2024-04-20T01:47:45.829483+0000","last_fullsized":"2024-04-20T01:47:45.829483+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:24.167832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"47'19","reported_seq":86,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-20T01:47:49.710877+0000","last_change":"2024-04-20T01:47:49.711257+0000","last_active":"2024-04-20T01:47:49.710877+0000","last_peered":"2024-04-20T01:47:49.710877+0000","last_clean":"2024-04-20T01:47:49.710877+0000","last_became_active":"2024-04-20T01:47:32.868365+0000","last_became_peered":"2024-04-20T01:47:32.868365+0000","last_unstale":"2024-04-20T01:47:49.710877+0000","last_undegraded":"2024-04-20T01:47:49.710877+0000","last_fullsized":"2024-04-20T01:47:49.710877+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:41.198793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052462099999999996,"stat_sum":{"num_bytes":851968,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":1685,"num_write":29,"num_write_kb":1312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,1,5,15,4],"acting":[3,7,6,1,5,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"1"},{"start":"a","length":"1"},{"start":"c","length":"2"}]},{"pgid":"2.3","version":"0'0","reported_seq":31,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-20T01:47:48.715498+0000","last_change":"2024-04-20T01:47:20.039014+0000","last_active":"2024-04-20T01:47:48.715498+0000","last_peered":"2024-04-20T01:47:48.715498+0000","last_clean":"2024-04-20T01:47:48.715498+0000","last_became_active":"2024-04-20T01:47:20.038764+0000","last_became_peered":"2024-04-20T01:47:20.038764+0000","last_unstale":"2024-04-20T01:47:48.715498+0000","last_undegraded":"2024-04-20T01:47:48.715498+0000","last_fullsized":"2024-04-20T01:47:48.715498+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:26.112165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"25'157","reported_seq":225,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.829609+0000","last_change":"2024-04-20T01:47:17.303029+0000","last_active":"2024-04-20T01:47:45.829609+0000","last_peered":"2024-04-20T01:47:45.829609+0000","last_clean":"2024-04-20T01:47:45.829609+0000","last_became_active":"2024-04-20T01:47:17.302105+0000","last_became_peered":"2024-04-20T01:47:17.302105+0000","last_unstale":"2024-04-20T01:47:45.829609+0000","last_undegraded":"2024-04-20T01:47:45.829609+0000","last_fullsized":"2024-04-20T01:47:45.829609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_clean_scrub_stamp":"2024-04-20T01:47:15.986754+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:30.481506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,15],"acting":[13,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.2","version":"44'31","reported_seq":92,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T01:47:45.844195+0000","last_change":"2024-04-20T01:47:42.813164+0000","last_active":"2024-04-20T01:47:45.844195+0000","last_peered":"2024-04-20T01:47:45.844195+0000","last_clean":"2024-04-20T01:47:45.844195+0000","last_became_active":"2024-04-20T01:47:32.866509+0000","last_became_peered":"2024-04-20T01:47:32.866509+0000","last_unstale":"2024-04-20T01:47:45.844195+0000","last_undegraded":"2024-04-20T01:47:45.844195+0000","last_fullsized":"2024-04-20T01:47:45.844195+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.395078+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012506468999999999,"stat_sum":{"num_bytes":4505600,"num_objects":9,"num_object_clones":3,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":32,"num_read_kb":8603,"num_write":40,"num_write_kb":4672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,6,10,14,1,7],"acting":[13,3,6,10,14,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"2"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-20T01:47:49.714895+0000","last_change":"2024-04-20T01:47:20.069279+0000","last_active":"2024-04-20T01:47:49.714895+0000","last_peered":"2024-04-20T01:47:49.714895+0000","last_clean":"2024-04-20T01:47:49.714895+0000","last_became_active":"2024-04-20T01:47:20.069005+0000","last_became_peered":"2024-04-20T01:47:20.069005+0000","last_unstale":"2024-04-20T01:47:49.714895+0000","last_undegraded":"2024-04-20T01:47:49.714895+0000","last_fullsized":"2024-04-20T01:47:49.714895+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:22.648743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"48'14","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-20T01:47:49.731240+0000","last_change":"2024-04-20T01:47:49.731240+0000","last_active":"2024-04-20T01:47:49.731240+0000","last_peered":"2024-04-20T01:47:49.731240+0000","last_clean":"2024-04-20T01:47:49.731240+0000","last_became_active":"2024-04-20T01:47:32.722189+0000","last_became_peered":"2024-04-20T01:47:32.722189+0000","last_unstale":"2024-04-20T01:47:49.731240+0000","last_undegraded":"2024-04-20T01:47:49.731240+0000","last_fullsized":"2024-04-20T01:47:49.731240+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:43.443403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017027532000000001,"stat_sum":{"num_bytes":475136,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":1011,"num_write":6,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,14,9,5,6,3,13],"acting":[4,14,9,5,6,3,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"1"},{"start":"a","length":"1"},{"start":"c","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-20T01:47:48.708303+0000","last_change":"2024-04-20T01:47:20.038409+0000","last_active":"2024-04-20T01:47:48.708303+0000","last_peered":"2024-04-20T01:47:48.708303+0000","last_clean":"2024-04-20T01:47:48.708303+0000","last_became_active":"2024-04-20T01:47:20.038107+0000","last_became_peered":"2024-04-20T01:47:20.038107+0000","last_unstale":"2024-04-20T01:47:48.708303+0000","last_undegraded":"2024-04-20T01:47:48.708303+0000","last_fullsized":"2024-04-20T01:47:48.708303+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:44.125577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"48'73","reported_seq":140,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-20T01:47:49.730510+0000","last_change":"2024-04-20T01:47:49.721901+0000","last_active":"2024-04-20T01:47:49.730510+0000","last_peered":"2024-04-20T01:47:49.730510+0000","last_clean":"2024-04-20T01:47:49.730510+0000","last_became_active":"2024-04-20T01:47:32.867693+0000","last_became_peered":"2024-04-20T01:47:32.867693+0000","last_unstale":"2024-04-20T01:47:49.730510+0000","last_undegraded":"2024-04-20T01:47:49.730510+0000","last_fullsized":"2024-04-20T01:47:49.730510+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:31.824121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014938359,"stat_sum":{"num_bytes":7225344,"num_objects":9,"num_object_clones":3,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":28,"num_read_kb":3805,"num_write":54,"num_write_kb":7840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,15,4,1,0,3,13],"acting":[8,15,4,1,0,3,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"1"},{"start":"a","length":"1"},{"start":"c","length":"2"}]},{"pgid":"3.7","version":"41'12","reported_seq":78,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-20T01:47:49.731798+0000","last_change":"2024-04-20T01:47:49.731798+0000","last_active":"2024-04-20T01:47:49.731798+0000","last_peered":"2024-04-20T01:47:49.731798+0000","last_clean":"2024-04-20T01:47:49.731798+0000","last_became_active":"2024-04-20T01:47:32.722002+0000","last_became_peered":"2024-04-20T01:47:32.722002+0000","last_unstale":"2024-04-20T01:47:49.731798+0000","last_undegraded":"2024-04-20T01:47:49.731798+0000","last_fullsized":"2024-04-20T01:47:49.731798+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:43.990113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017326798000000001,"stat_sum":{"num_bytes":2523136,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":5058,"num_write":28,"num_write_kb":2464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,0,8,5,14],"acting":[4,11,9,0,8,5,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"1"},{"start":"a","length":"1"},{"start":"c","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-20T01:47:48.716213+0000","last_change":"2024-04-20T01:47:20.110600+0000","last_active":"2024-04-20T01:47:48.716213+0000","last_peered":"2024-04-20T01:47:48.716213+0000","last_clean":"2024-04-20T01:47:48.716213+0000","last_became_active":"2024-04-20T01:47:20.110260+0000","last_became_peered":"2024-04-20T01:47:20.110260+0000","last_unstale":"2024-04-20T01:47:48.716213+0000","last_undegraded":"2024-04-20T01:47:48.716213+0000","last_fullsized":"2024-04-20T01:47:48.716213+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:47.718290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":31,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-20T01:47:48.714016+0000","last_change":"2024-04-20T01:47:20.036634+0000","last_active":"2024-04-20T01:47:48.714016+0000","last_peered":"2024-04-20T01:47:48.714016+0000","last_clean":"2024-04-20T01:47:48.714016+0000","last_became_active":"2024-04-20T01:47:20.036328+0000","last_became_peered":"2024-04-20T01:47:20.036328+0000","last_unstale":"2024-04-20T01:47:48.714016+0000","last_undegraded":"2024-04-20T01:47:48.714016+0000","last_fullsized":"2024-04-20T01:47:48.714016+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.385250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,13],"acting":[6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"47'6","reported_seq":70,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-20T01:47:49.713113+0000","last_change":"2024-04-20T01:47:49.717191+0000","last_active":"2024-04-20T01:47:49.713113+0000","last_peered":"2024-04-20T01:47:49.713113+0000","last_clean":"2024-04-20T01:47:49.713113+0000","last_became_active":"2024-04-20T01:47:32.866834+0000","last_became_peered":"2024-04-20T01:47:32.866834+0000","last_unstale":"2024-04-20T01:47:49.713113+0000","last_undegraded":"2024-04-20T01:47:49.713113+0000","last_fullsized":"2024-04-20T01:47:49.713113+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:54.424527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041951949999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":386,"num_write":3,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,3,6,2,9],"acting":[0,13,1,3,6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"},{"start":"7","length":"1"},{"start":"a","length":"1"},{"start":"c","length":"2"}]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":39550976,"num_objects":58,"num_object_clones":15,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":3,"num_read":245,"num_read_kb":38220,"num_write":428,"num_write_kb":46336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72704000,"data_stored":89808896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":375,"ondisk_log_size":375,"up":112,"acting":112,"num_store_stats":16},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280521,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":4724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":4837376,"data_stored":5903706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]}]},{"osd":15,"up_from":21,"seq":90194313226,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30856,"kb_used_data":3832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340984,"statfs":{"total":96636764160,"available":96605167616,"internally_reserved":0,"allocated":3923968,"data_stored":4438105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]}]},{"osd":14,"up_from":21,"seq":90194313227,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33732,"kb_used_data":6708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338108,"statfs":{"total":96636764160,"available":96602222592,"internally_reserved":0,"allocated":6868992,"data_stored":8111070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,4,5,6,7,8,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]}]},{"osd":12,"up_from":21,"seq":90194313226,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30964,"kb_used_data":3940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340876,"statfs":{"total":96636764160,"available":96605057024,"internally_reserved":0,"allocated":4034560,"data_stored":4257337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]}]},{"osd":11,"up_from":21,"seq":90194313225,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":5280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":5406720,"data_stored":5768761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,8,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]}]},{"osd":10,"up_from":21,"seq":90194313227,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":5028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":5148672,"data_stored":5633593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,4,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]}]},{"osd":9,"up_from":21,"seq":90194313227,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30228,"kb_used_data":3204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341612,"statfs":{"total":96636764160,"available":96605810688,"internally_reserved":0,"allocated":3280896,"data_stored":3085881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,4,6,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]}]},{"osd":8,"up_from":21,"seq":90194313226,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":5228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":5353472,"data_stored":6288953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,6,7,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9630000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]}]},{"osd":7,"up_from":21,"seq":90194313227,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":5300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":5427200,"data_stored":6551097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,3,4,6,8,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":6,"up_from":21,"seq":90194313225,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":4412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":4517888,"data_stored":5199417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]}]},{"osd":1,"up_from":21,"seq":90194313226,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33828,"kb_used_data":6804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338012,"statfs":{"total":96636764160,"available":96602124288,"internally_reserved":0,"allocated":6967296,"data_stored":8156729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]}]},{"osd":0,"up_from":21,"seq":90194313227,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33492,"kb_used_data":6468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338348,"statfs":{"total":96636764160,"available":96602468352,"internally_reserved":0,"allocated":6623232,"data_stored":7693295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]}]},{"osd":2,"up_from":21,"seq":90194313225,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29900,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341940,"statfs":{"total":96636764160,"available":96606146560,"internally_reserved":0,"allocated":2945024,"data_stored":3200569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":3,"up_from":21,"seq":90194313227,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35124,"kb_used_data":8100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336716,"statfs":{"total":96636764160,"available":96600797184,"internally_reserved":0,"allocated":8294400,"data_stored":10237497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,4,5,6,7,8,9,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]}]},{"osd":4,"up_from":21,"seq":90194313227,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":5444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":5574656,"data_stored":6313529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,5,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]}]},{"osd":5,"up_from":21,"seq":90194313226,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31496,"kb_used_data":4472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340344,"statfs":{"total":96636764160,"available":96604512256,"internally_reserved":0,"allocated":4579328,"data_stored":5252665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5996544,"data_stored":7364608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6336512,"data_stored":7827456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2314240,"data_stored":2871296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7663616,"data_stored":9908224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4943872,"data_stored":5984256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3948544,"data_stored":4923392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3887104,"data_stored":4870144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4796416,"data_stored":6221824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4722688,"data_stored":5959680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":2650112,"data_stored":2756608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4517888,"data_stored":5304320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4775936,"data_stored":5439488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":3403776,"data_stored":3928064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":3743744,"data_stored":5087232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":6242304,"data_stored":7782400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":2760704,"data_stored":3579904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T01:47:51.533 INFO:tasks.thrashosds.thrasher:Setting 3.4 to [2, 11, 5, 1, 0, 10, 15, 6, 13, 7, 4, 12, 9, 14] 2024-04-20T01:47:51.533 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.4', '2', '11', '5', '1', '0', '10', '15', '6', '13', '7', '4', '12', '9', '14'] 2024-04-20T01:47:51.533 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.4 2 11 5 1 0 10 15 6 13 7 4 12 9 14 2024-04-20T01:47:51.578 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:51.578 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:51.589 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-20T01:47:51.624 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:51.639+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:51.696 INFO:tasks.rados.rados.0.smithi165.stdout:174: done (0 left) 2024-04-20T01:47:51.696 INFO:tasks.rados.rados.0.smithi165.stdout:175: snap_create 2024-04-20T01:47:51.718 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:51.718 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:51.724 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:51.739+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:47:51.729 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-20T01:47:51.825 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:51.840+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:51.858 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:51.859 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:51.869 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-20T01:47:51.926 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:47:51.940+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:47:51.999 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:51.999 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:52.009 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-20T01:47:52.026 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:47:52.041+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:47:52.127 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:52.142+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:52.138 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:52.138 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:52.148 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-20T01:47:52.232 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:52.247+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:47:52.318 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:52.318 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:52.328 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-20T01:47:52.329 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:47:52.344+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:47:52.430 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:52.445+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:47:52.497 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:52.497 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:52.507 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-20T01:47:52.551 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:52.566+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:52.631 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:52.646+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:47:52.636 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:52.636 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:52.647 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-20T01:47:52.690 INFO:tasks.rados.rados.0.smithi165.stdout:175: done (0 left) 2024-04-20T01:47:52.690 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 17 current snap is 10 2024-04-20T01:47:52.690 INFO:tasks.rados.rados.0.smithi165.stdout:176: seq_num 85 ranges {2179072=671744,2850816=770048} 2024-04-20T01:47:52.698 INFO:teuthology.orchestra.run.smithi044.stderr:set 3.4 pg_upmap_items mapping to [2->11,5->1,0->10,15->6,13->7,4->12,9->14] 2024-04-20T01:47:52.706 INFO:tasks.rados.rados.0.smithi165.stdout:176: writing smithi16579500-17 from 2179072 to 2850816 tid 1 2024-04-20T01:47:52.710 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-20T01:47:52.711 INFO:tasks.rados.rados.0.smithi165.stdout:176: writing smithi16579500-17 from 2850816 to 3620864 tid 2 2024-04-20T01:47:52.711 INFO:tasks.rados.rados.0.smithi165.stdout:177: snap_create 2024-04-20T01:47:52.714 INFO:tasks.rados.rados.0.smithi165.stdout:176: finishing write tid 1 to smithi16579500-17 2024-04-20T01:47:52.717 INFO:tasks.rados.rados.0.smithi165.stdout:176: finishing write tid 2 to smithi16579500-17 2024-04-20T01:47:52.719 INFO:tasks.rados.rados.0.smithi165.stdout:176: finishing write tid 3 to smithi16579500-17 2024-04-20T01:47:52.720 INFO:tasks.rados.rados.0.smithi165.stdout:176: finishing write tid 4 to smithi16579500-17 2024-04-20T01:47:52.720 INFO:tasks.rados.rados.0.smithi165.stdout:176: oid 17 updating version 0 to 52 2024-04-20T01:47:52.720 INFO:tasks.rados.rados.0.smithi165.stdout:176: oid 17 updating version 52 to 53 2024-04-20T01:47:52.721 INFO:tasks.rados.rados.0.smithi165.stdout:176: oid 17 version 53 is already newer than 51 2024-04-20T01:47:52.721 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 17 v 53 (ObjNum 85 snap 10 seq_num 85) dirty exists 2024-04-20T01:47:52.721 INFO:tasks.rados.rados.0.smithi165.stdout:176: left oid 17 (ObjNum 85 snap 10 seq_num 85) 2024-04-20T01:47:52.756 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:47:52.770+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:47:52.820 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:52.821 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:52.831 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-20T01:47:52.872 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:52.887+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:47:52.934 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:52.948+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:53.001 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:53.001 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:53.011 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-20T01:47:53.054 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:47:53.069+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:47:53.136 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:53.151+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:53.138 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:53.138 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:53.149 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-20T01:47:53.237 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:53.252+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:47:53.318 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:53.318 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:53.328 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-20T01:47:53.338 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:53.352+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:53.439 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:53.454+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:53.481 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:53.481 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:53.493 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-20T01:47:53.503 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-20T01:47:53.540 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:53.555+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:47:53.641 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:53.656+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:53.643 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:53.643 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:53.655 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-20T01:47:53.694 INFO:tasks.rados.rados.0.smithi165.stdout:176: done (1 left) 2024-04-20T01:47:53.695 INFO:tasks.rados.rados.0.smithi165.stdout:177: done (0 left) 2024-04-20T01:47:53.695 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 18 current snap is 11 2024-04-20T01:47:53.695 INFO:tasks.rados.rados.0.smithi165.stdout:178: seq_num 86 ranges {0=688128,688128=540672,1228800=229376} 2024-04-20T01:47:53.698 INFO:tasks.rados.rados.0.smithi165.stdout:178: writing smithi16579500-18 from 0 to 688128 tid 1 2024-04-20T01:47:53.701 INFO:tasks.rados.rados.0.smithi165.stdout:178: writing smithi16579500-18 from 688128 to 1228800 tid 2 2024-04-20T01:47:53.703 INFO:tasks.rados.rados.0.smithi165.stdout:178: writing smithi16579500-18 from 1228800 to 1458176 tid 3 2024-04-20T01:47:53.703 INFO:tasks.rados.rados.0.smithi165.stdout:179: copy_from oid 25 from oid 47 current snap is 11 2024-04-20T01:47:53.703 INFO:tasks.rados.rados.0.smithi165.stdout:180: rollback oid 7 current snap is 11 2024-04-20T01:47:53.704 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 7 to 9 2024-04-20T01:47:53.704 INFO:tasks.rados.rados.0.smithi165.stdout:181: setattr oid 3 current snap is 11 2024-04-20T01:47:53.704 INFO:tasks.rados.rados.0.smithi165.stdout:180: finishing rollback tid 1 to smithi16579500-7 2024-04-20T01:47:53.704 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 7 v 0 (ObjNum 81 snap 0 seq_num 1) dirty dne 2024-04-20T01:47:53.704 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is set 2024-04-20T01:47:53.704 INFO:tasks.rados.rados.0.smithi165.stdout:180: done (3 left) 2024-04-20T01:47:53.705 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 26 current snap is 11 2024-04-20T01:47:53.705 INFO:tasks.rados.rados.0.smithi165.stdout:182: seq_num 87 ranges {1720320=704512,2424832=688128,3112960=81920} 2024-04-20T01:47:53.714 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:47:53.718 INFO:tasks.rados.rados.0.smithi165.stdout:182: writing smithi16579500-26 from 1720320 to 2424832 tid 1 2024-04-20T01:47:53.722 INFO:tasks.rados.rados.0.smithi165.stdout:182: writing smithi16579500-26 from 2424832 to 3112960 tid 2 2024-04-20T01:47:53.725 INFO:tasks.rados.rados.0.smithi165.stdout:182: writing smithi16579500-26 from 3112960 to 3194880 tid 3 2024-04-20T01:47:53.725 INFO:tasks.rados.rados.0.smithi165.stdout:183: rollback oid 32 current snap is 11 2024-04-20T01:47:53.725 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 32 to 10 2024-04-20T01:47:53.725 INFO:tasks.rados.rados.0.smithi165.stdout:178: finishing write tid 1 to smithi16579500-18 2024-04-20T01:47:53.725 INFO:tasks.rados.rados.0.smithi165.stdout:178: finishing write tid 2 to smithi16579500-18 2024-04-20T01:47:53.725 INFO:tasks.rados.rados.0.smithi165.stdout:178: finishing write tid 3 to smithi16579500-18 2024-04-20T01:47:53.725 INFO:tasks.rados.rados.0.smithi165.stdout:178: finishing write tid 4 to smithi16579500-18 2024-04-20T01:47:53.725 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 3 v 34 (ObjNum 289 snap 0 seq_num 1) dirty exists 2024-04-20T01:47:53.725 INFO:tasks.rados.rados.0.smithi165.stdout:178: finishing write tid 5 to smithi16579500-18 2024-04-20T01:47:53.726 INFO:tasks.rados.rados.0.smithi165.stdout:178: oid 18 updating version 0 to 16 2024-04-20T01:47:53.726 INFO:tasks.rados.rados.0.smithi165.stdout:178: oid 18 updating version 16 to 17 2024-04-20T01:47:53.726 INFO:tasks.rados.rados.0.smithi165.stdout:178: oid 18 version 17 is already newer than 15 2024-04-20T01:47:53.726 INFO:tasks.rados.rados.0.smithi165.stdout:178: oid 18 version 17 is already newer than 14 2024-04-20T01:47:53.726 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 18 v 17 (ObjNum 86 snap 11 seq_num 86) dirty exists 2024-04-20T01:47:53.726 INFO:tasks.rados.rados.0.smithi165.stdout:178: left oid 18 (ObjNum 86 snap 11 seq_num 86) 2024-04-20T01:47:53.726 INFO:tasks.rados.rados.0.smithi165.stdout:178: done (4 left) 2024-04-20T01:47:53.726 INFO:tasks.rados.rados.0.smithi165.stdout:181: done (3 left) 2024-04-20T01:47:53.726 INFO:tasks.rados.rados.0.smithi165.stdout:184: snap_create 2024-04-20T01:47:53.729 INFO:tasks.rados.rados.0.smithi165.stdout:179: finishing copy_from racing read to smithi16579500-25 2024-04-20T01:47:53.731 INFO:tasks.rados.rados.0.smithi165.stdout:183: finishing rollback tid 0 to smithi16579500-32 2024-04-20T01:47:53.732 INFO:tasks.rados.rados.0.smithi165.stdout:183: finishing rollback tid 1 to smithi16579500-32 2024-04-20T01:47:53.732 INFO:tasks.rados.rados.0.smithi165.stdout:183: finishing rollback tid 2 to smithi16579500-32 2024-04-20T01:47:53.732 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 32 v 21 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-04-20T01:47:53.733 INFO:tasks.rados.rados.0.smithi165.stdout:182: finishing write tid 1 to smithi16579500-26 2024-04-20T01:47:53.733 INFO:tasks.rados.rados.0.smithi165.stdout:182: finishing write tid 2 to smithi16579500-26 2024-04-20T01:47:53.733 INFO:tasks.rados.rados.0.smithi165.stdout:182: finishing write tid 3 to smithi16579500-26 2024-04-20T01:47:53.734 INFO:tasks.rados.rados.0.smithi165.stdout:182: finishing write tid 4 to smithi16579500-26 2024-04-20T01:47:53.735 INFO:tasks.rados.rados.0.smithi165.stdout:179: finishing copy_from to smithi16579500-25 2024-04-20T01:47:53.735 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 25 v 36 (ObjNum 80 snap 8 seq_num 80) dirty exists 2024-04-20T01:47:53.736 INFO:tasks.rados.rados.0.smithi165.stdout:182: finishing write tid 5 to smithi16579500-26 2024-04-20T01:47:53.736 INFO:tasks.rados.rados.0.smithi165.stdout:182: oid 26 updating version 0 to 56 2024-04-20T01:47:53.736 INFO:tasks.rados.rados.0.smithi165.stdout:182: oid 26 updating version 56 to 58 2024-04-20T01:47:53.736 INFO:tasks.rados.rados.0.smithi165.stdout:182: oid 26 version 58 is already newer than 57 2024-04-20T01:47:53.736 INFO:tasks.rados.rados.0.smithi165.stdout:182: oid 26 version 58 is already newer than 55 2024-04-20T01:47:53.736 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 26 v 58 (ObjNum 87 snap 11 seq_num 87) dirty exists 2024-04-20T01:47:53.736 INFO:tasks.rados.rados.0.smithi165.stdout:182: left oid 26 (ObjNum 87 snap 11 seq_num 87) 2024-04-20T01:47:53.742 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:53.757+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:47:53.745 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:47:53.773 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:47:53.797 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:47:53.830 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:47:53.839 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:53.839 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-04-20T01:47:53.839 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-04-20T01:47:53.839 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-04-20T01:47:53.839 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.839 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.7s3 36/30 ECSubWrite(tid=58, reqid=client.4643.0:211, at_version=36'8, trim_to=0'0, roll_forward_to=33'5))", 2024-04-20T01:47:53.840 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.782029+0000", 2024-04-20T01:47:53.840 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 16.070308581999999, 2024-04-20T01:47:53.840 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0082533060000000002, 2024-04-20T01:47:53.840 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:53.840 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:53.840 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:53.840 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.840 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:53.840 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782029+0000", 2024-04-20T01:47:53.840 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.840 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.840 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.840 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:53.841 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782029+0000", 2024-04-20T01:47:53.841 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.841 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.841 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:53.841 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782028+0000", 2024-04-20T01:47:53.841 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:53.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.841 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.841 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:53.841 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782164+0000", 2024-04-20T01:47:53.841 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013607300000000001 2024-04-20T01:47:53.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.841 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.842 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:53.842 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782181+0000", 2024-04-20T01:47:53.842 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6651999999999999e-05 2024-04-20T01:47:53.842 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.842 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.842 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:53.842 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782196+0000", 2024-04-20T01:47:53.842 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5370999999999999e-05 2024-04-20T01:47:53.842 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.842 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.842 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:53.842 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.783434+0000", 2024-04-20T01:47:53.842 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001237581 2024-04-20T01:47:53.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.843 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.843 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:53.843 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.783444+0000", 2024-04-20T01:47:53.843 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0686999999999999e-05 2024-04-20T01:47:53.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.843 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.843 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:53.843 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.790251+0000", 2024-04-20T01:47:53.843 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068071729999999997 2024-04-20T01:47:53.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.843 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.843 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:53.843 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.790283+0000", 2024-04-20T01:47:53.843 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1250000000000001e-05 2024-04-20T01:47:53.844 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.844 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.844 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.844 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.844 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.844 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs1 36/30 ECSubWrite(tid=59, reqid=client.4643.0:214, at_version=36'14, trim_to=0'0, roll_forward_to=33'12))", 2024-04-20T01:47:53.844 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.783999+0000", 2024-04-20T01:47:53.844 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 16.068339506000001, 2024-04-20T01:47:53.844 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011678028, 2024-04-20T01:47:53.844 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:53.845 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:53.845 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:53.845 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.845 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:53.845 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.783999+0000", 2024-04-20T01:47:53.845 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.845 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.845 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.845 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:53.845 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.783999+0000", 2024-04-20T01:47:53.845 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.845 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.845 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.846 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:53.846 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.783997+0000", 2024-04-20T01:47:53.846 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:53.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.846 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.846 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:53.846 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.784024+0000", 2024-04-20T01:47:53.846 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6801999999999999e-05 2024-04-20T01:47:53.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.846 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.846 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:53.846 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.784037+0000", 2024-04-20T01:47:53.847 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2361000000000001e-05 2024-04-20T01:47:53.847 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.847 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.847 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:53.847 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.784046+0000", 2024-04-20T01:47:53.847 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5009999999999993e-06 2024-04-20T01:47:53.847 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.847 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.847 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:53.847 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.785306+0000", 2024-04-20T01:47:53.847 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001259942 2024-04-20T01:47:53.847 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.847 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.848 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:53.848 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.785316+0000", 2024-04-20T01:47:53.848 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0309000000000001e-05 2024-04-20T01:47:53.848 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.848 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.848 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:53.848 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.795648+0000", 2024-04-20T01:47:53.848 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010331127000000001 2024-04-20T01:47:53.848 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.848 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.848 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:53.848 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.795677+0000", 2024-04-20T01:47:53.849 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9016000000000001e-05 2024-04-20T01:47:53.849 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.849 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.849 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.849 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.849 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.849 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s4 36/30 ECSubWrite(tid=40, reqid=client.4643.0:215, at_version=36'28, trim_to=0'0, roll_forward_to=36'26))", 2024-04-20T01:47:53.849 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.808801+0000", 2024-04-20T01:47:53.849 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 16.043536908, 2024-04-20T01:47:53.849 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0082210219999999997, 2024-04-20T01:47:53.849 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:53.849 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:53.850 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:53.850 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.850 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:53.850 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808801+0000", 2024-04-20T01:47:53.850 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.850 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.850 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.850 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:53.850 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808801+0000", 2024-04-20T01:47:53.850 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.850 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.851 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.851 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:53.851 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808800+0000", 2024-04-20T01:47:53.851 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:53.851 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.851 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.851 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:53.851 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808834+0000", 2024-04-20T01:47:53.851 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3283e-05 2024-04-20T01:47:53.851 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.851 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.851 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:53.851 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808847+0000", 2024-04-20T01:47:53.852 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3587999999999999e-05 2024-04-20T01:47:53.852 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.852 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.852 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:53.852 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808857+0000", 2024-04-20T01:47:53.852 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6069999999999997e-06 2024-04-20T01:47:53.852 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.852 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.852 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:53.853 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808906+0000", 2024-04-20T01:47:53.853 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9283000000000002e-05 2024-04-20T01:47:53.853 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.853 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.853 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:53.853 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808917+0000", 2024-04-20T01:47:53.853 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0465999999999999e-05 2024-04-20T01:47:53.853 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.853 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.853 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:53.853 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.816976+0000", 2024-04-20T01:47:53.854 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0080597199999999994 2024-04-20T01:47:53.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.854 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.854 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:53.854 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.817022+0000", 2024-04-20T01:47:53.854 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5802e-05 2024-04-20T01:47:53.854 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.854 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.854 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.854 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.855 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs1 37/30 ECSubWrite(tid=64, reqid=client.4643.0:225, at_version=37'16, trim_to=0'0, roll_forward_to=36'14))", 2024-04-20T01:47:53.855 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.788088+0000", 2024-04-20T01:47:53.855 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 15.064250486000001, 2024-04-20T01:47:53.855 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010015382, 2024-04-20T01:47:53.855 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:53.855 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:53.855 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:53.855 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.855 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:53.855 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788088+0000", 2024-04-20T01:47:53.855 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.856 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.856 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:53.856 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788088+0000", 2024-04-20T01:47:53.856 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.856 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.856 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.856 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:53.856 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788087+0000", 2024-04-20T01:47:53.856 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:53.856 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.856 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.857 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:53.857 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788230+0000", 2024-04-20T01:47:53.857 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014296 2024-04-20T01:47:53.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.857 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.857 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:53.857 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788255+0000", 2024-04-20T01:47:53.857 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4790000000000002e-05 2024-04-20T01:47:53.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.857 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.857 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:53.857 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788276+0000", 2024-04-20T01:47:53.857 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0806999999999999e-05 2024-04-20T01:47:53.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.858 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.858 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:53.858 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791925+0000", 2024-04-20T01:47:53.858 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003649949 2024-04-20T01:47:53.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.858 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.858 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:53.858 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791937+0000", 2024-04-20T01:47:53.858 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1544e-05 2024-04-20T01:47:53.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.858 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.858 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:53.858 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.798083+0000", 2024-04-20T01:47:53.859 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0061460079999999997 2024-04-20T01:47:53.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.859 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.859 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:53.859 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.798103+0000", 2024-04-20T01:47:53.859 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9907000000000001e-05 2024-04-20T01:47:53.859 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.859 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.859 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.859 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.859 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs1 37/30 ECSubWrite(tid=65, reqid=client.4643.0:226, at_version=37'17, trim_to=0'0, roll_forward_to=36'14))", 2024-04-20T01:47:53.859 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.790333+0000", 2024-04-20T01:47:53.859 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 15.062005154, 2024-04-20T01:47:53.860 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0077991670000000001, 2024-04-20T01:47:53.860 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:53.860 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:53.860 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:53.860 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.860 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:53.860 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.790333+0000", 2024-04-20T01:47:53.860 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.860 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.860 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.860 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:53.860 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.790333+0000", 2024-04-20T01:47:53.861 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.861 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:53.861 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.790331+0000", 2024-04-20T01:47:53.861 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:53.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.861 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:53.861 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.790388+0000", 2024-04-20T01:47:53.861 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6341999999999997e-05 2024-04-20T01:47:53.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.861 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:53.862 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.790411+0000", 2024-04-20T01:47:53.862 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3876999999999999e-05 2024-04-20T01:47:53.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.862 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:53.862 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.790426+0000", 2024-04-20T01:47:53.862 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4612e-05 2024-04-20T01:47:53.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.862 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:53.862 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.792906+0000", 2024-04-20T01:47:53.862 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0024795899999999998 2024-04-20T01:47:53.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.863 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:53.863 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.792916+0000", 2024-04-20T01:47:53.863 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0763e-05 2024-04-20T01:47:53.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.863 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.863 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:53.863 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.798116+0000", 2024-04-20T01:47:53.863 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051997470000000002 2024-04-20T01:47:53.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.863 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.863 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:53.863 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.798132+0000", 2024-04-20T01:47:53.863 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5869999999999999e-05 2024-04-20T01:47:53.863 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.864 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.864 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.864 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.864 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs1 42/30 ECSubWrite(tid=98, reqid=client.4643.0:321, at_version=42'36, trim_to=0'0, roll_forward_to=42'35))", 2024-04-20T01:47:53.864 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:43.943166+0000", 2024-04-20T01:47:53.864 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.9091715750000002, 2024-04-20T01:47:53.864 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.006430173, 2024-04-20T01:47:53.864 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:53.864 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:53.864 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:53.864 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.864 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:53.864 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943166+0000", 2024-04-20T01:47:53.865 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.865 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.865 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:53.865 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943166+0000", 2024-04-20T01:47:53.865 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.865 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.865 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:53.865 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943165+0000", 2024-04-20T01:47:53.865 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:53.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.865 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.865 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:53.865 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943256+0000", 2024-04-20T01:47:53.866 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0706999999999997e-05 2024-04-20T01:47:53.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.866 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:53.866 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943275+0000", 2024-04-20T01:47:53.866 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8702000000000002e-05 2024-04-20T01:47:53.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.866 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:53.866 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943285+0000", 2024-04-20T01:47:53.866 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0664e-05 2024-04-20T01:47:53.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.866 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:53.867 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943363+0000", 2024-04-20T01:47:53.867 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7194000000000007e-05 2024-04-20T01:47:53.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.867 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:53.867 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943374+0000", 2024-04-20T01:47:53.867 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1043000000000001e-05 2024-04-20T01:47:53.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.867 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:53.867 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.949567+0000", 2024-04-20T01:47:53.867 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0061931900000000003 2024-04-20T01:47:53.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.868 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:53.868 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.949597+0000", 2024-04-20T01:47:53.868 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9723e-05 2024-04-20T01:47:53.868 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.868 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.868 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.868 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.868 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.869 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s4 42/30 ECSubWrite(tid=66, reqid=client.4643.0:319, at_version=0'0, trim_to=0'0, roll_forward_to=42'52))", 2024-04-20T01:47:53.869 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:43.944741+0000", 2024-04-20T01:47:53.869 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.9075966449999999, 2024-04-20T01:47:53.869 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0064178619999999999, 2024-04-20T01:47:53.869 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:53.869 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:53.869 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:53.869 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.869 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:53.869 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.944741+0000", 2024-04-20T01:47:53.869 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.869 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.869 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.869 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:53.869 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.944741+0000", 2024-04-20T01:47:53.870 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.870 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.870 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:53.870 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.944740+0000", 2024-04-20T01:47:53.870 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:53.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.870 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.870 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:53.870 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.944755+0000", 2024-04-20T01:47:53.870 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5150000000000001e-05 2024-04-20T01:47:53.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.871 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.871 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:53.871 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.944771+0000", 2024-04-20T01:47:53.871 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5336e-05 2024-04-20T01:47:53.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.871 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.871 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:53.871 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.944781+0000", 2024-04-20T01:47:53.871 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0457e-05 2024-04-20T01:47:53.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.871 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.871 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:53.871 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.944860+0000", 2024-04-20T01:47:53.871 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8675000000000001e-05 2024-04-20T01:47:53.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.872 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.872 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:53.872 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.944870+0000", 2024-04-20T01:47:53.872 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0210000000000001e-05 2024-04-20T01:47:53.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.872 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.872 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:53.872 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.951128+0000", 2024-04-20T01:47:53.872 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0062579159999999997 2024-04-20T01:47:53.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.872 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.872 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:53.872 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.951159+0000", 2024-04-20T01:47:53.873 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1374000000000001e-05 2024-04-20T01:47:53.873 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.873 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.873 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.873 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.873 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs1 42/30 ECSubWrite(tid=99, reqid=client.4643.0:322, at_version=42'37, trim_to=0'0, roll_forward_to=42'35))", 2024-04-20T01:47:53.873 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:43.945520+0000", 2024-04-20T01:47:53.873 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 9.9068183540000003, 2024-04-20T01:47:53.873 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068373820000000004, 2024-04-20T01:47:53.873 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:53.873 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:53.873 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:53.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.874 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:53.874 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.945520+0000", 2024-04-20T01:47:53.874 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.874 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:53.874 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.945520+0000", 2024-04-20T01:47:53.874 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.874 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:53.874 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.945519+0000", 2024-04-20T01:47:53.875 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:53.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.875 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.875 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:53.875 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.945573+0000", 2024-04-20T01:47:53.875 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3891999999999998e-05 2024-04-20T01:47:53.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.875 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.875 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:53.875 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.945588+0000", 2024-04-20T01:47:53.875 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5712000000000001e-05 2024-04-20T01:47:53.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.875 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.875 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:53.876 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.945599+0000", 2024-04-20T01:47:53.876 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0334e-05 2024-04-20T01:47:53.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.876 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.876 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:53.876 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.945680+0000", 2024-04-20T01:47:53.876 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0872999999999996e-05 2024-04-20T01:47:53.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.876 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.876 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:53.876 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.945691+0000", 2024-04-20T01:47:53.876 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1265e-05 2024-04-20T01:47:53.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.876 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.877 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:53.877 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.952321+0000", 2024-04-20T01:47:53.877 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0066306510000000004 2024-04-20T01:47:53.877 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.877 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.877 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:53.877 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.952357+0000", 2024-04-20T01:47:53.877 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5568000000000002e-05 2024-04-20T01:47:53.877 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.877 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.877 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.877 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.877 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.877 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.8s1 47/30 ECSubWrite(tid=39, reqid=osd.5.0:39, at_version=47'27, trim_to=0'0, roll_forward_to=46'25))", 2024-04-20T01:47:53.878 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.718435+0000", 2024-04-20T01:47:53.878 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.1339034909999999, 2024-04-20T01:47:53.878 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0080245720000000006, 2024-04-20T01:47:53.878 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:53.878 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:53.878 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:53.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.878 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:53.878 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.718435+0000", 2024-04-20T01:47:53.878 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.878 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:53.878 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.718435+0000", 2024-04-20T01:47:53.878 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:53.879 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.718434+0000", 2024-04-20T01:47:53.879 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:53.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:53.879 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.718459+0000", 2024-04-20T01:47:53.879 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5162999999999999e-05 2024-04-20T01:47:53.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:53.880 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.718477+0000", 2024-04-20T01:47:53.880 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8593999999999998e-05 2024-04-20T01:47:53.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.880 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.880 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:53.880 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.718488+0000", 2024-04-20T01:47:53.880 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0862999999999999e-05 2024-04-20T01:47:53.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.880 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.880 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:53.880 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719011+0000", 2024-04-20T01:47:53.880 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052293800000000003 2024-04-20T01:47:53.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.880 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.880 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:53.881 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719022+0000", 2024-04-20T01:47:53.881 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0862999999999999e-05 2024-04-20T01:47:53.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.881 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.881 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:53.881 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.726426+0000", 2024-04-20T01:47:53.881 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0074044039999999998 2024-04-20T01:47:53.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.881 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.881 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:53.881 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.726459+0000", 2024-04-20T01:47:53.881 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2648000000000003e-05 2024-04-20T01:47:53.881 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.881 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.882 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.882 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.8s1 47/30 ECSubWrite(tid=40, reqid=osd.5.0:40, at_version=47'29, trim_to=0'0, roll_forward_to=46'25))", 2024-04-20T01:47:53.882 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.719174+0000", 2024-04-20T01:47:53.882 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.1331639459999998, 2024-04-20T01:47:53.882 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0103924, 2024-04-20T01:47:53.882 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:53.882 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:53.882 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:53.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.882 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:53.882 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719174+0000", 2024-04-20T01:47:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719174+0000", 2024-04-20T01:47:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719173+0000", 2024-04-20T01:47:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:53.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719197+0000", 2024-04-20T01:47:53.884 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4196e-05 2024-04-20T01:47:53.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.884 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:53.884 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719214+0000", 2024-04-20T01:47:53.884 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6276000000000001e-05 2024-04-20T01:47:53.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.884 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:53.884 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719223+0000", 2024-04-20T01:47:53.884 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6919999999999997e-06 2024-04-20T01:47:53.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.885 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:53.885 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720898+0000", 2024-04-20T01:47:53.885 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0016744259999999999 2024-04-20T01:47:53.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.885 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:53.885 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720905+0000", 2024-04-20T01:47:53.885 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6660000000000008e-06 2024-04-20T01:47:53.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.885 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:53.885 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.729531+0000", 2024-04-20T01:47:53.885 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086254020000000008 2024-04-20T01:47:53.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.886 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:53.886 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.729566+0000", 2024-04-20T01:47:53.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5621000000000003e-05 2024-04-20T01:47:53.886 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.886 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.886 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.886 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s4 47/30 ECSubWrite(tid=80, reqid=osd.8.0:80, at_version=47'65, trim_to=0'0, roll_forward_to=44'61))", 2024-04-20T01:47:53.886 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.720292+0000", 2024-04-20T01:47:53.886 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.1320455210000002, 2024-04-20T01:47:53.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0092900529999999995, 2024-04-20T01:47:53.887 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:53.887 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:53.887 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:53.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.887 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:53.887 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720292+0000", 2024-04-20T01:47:53.887 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.887 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:53.887 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720292+0000", 2024-04-20T01:47:53.887 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.887 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:53.888 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720292+0000", 2024-04-20T01:47:53.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:53.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.888 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:53.888 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720310+0000", 2024-04-20T01:47:53.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7717e-05 2024-04-20T01:47:53.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.888 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:53.888 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720318+0000", 2024-04-20T01:47:53.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3129999999999999e-06 2024-04-20T01:47:53.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:53.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720325+0000", 2024-04-20T01:47:53.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.832e-06 2024-04-20T01:47:53.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:53.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720837+0000", 2024-04-20T01:47:53.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051157600000000002 2024-04-20T01:47:53.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:53.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720846+0000", 2024-04-20T01:47:53.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3260000000000002e-06 2024-04-20T01:47:53.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:53.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.729552+0000", 2024-04-20T01:47:53.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0087057520000000006 2024-04-20T01:47:53.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:53.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.729583+0000", 2024-04-20T01:47:53.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0923999999999997e-05 2024-04-20T01:47:53.890 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.890 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.890 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.890 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.9s2 47/30 ECSubWrite(tid=15, reqid=client.4643.0:396, at_version=47'10, trim_to=0'0, roll_forward_to=47'8))", 2024-04-20T01:47:53.891 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.752334+0000", 2024-04-20T01:47:53.891 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.1000042429999999, 2024-04-20T01:47:53.891 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0069182169999999999, 2024-04-20T01:47:53.891 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:53.891 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:53.891 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:53.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.891 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:53.891 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.752334+0000", 2024-04-20T01:47:53.891 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.892 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:53.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.752334+0000", 2024-04-20T01:47:53.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.892 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:53.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.752333+0000", 2024-04-20T01:47:53.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:53.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.892 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:53.893 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.752746+0000", 2024-04-20T01:47:53.893 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00041369300000000003 2024-04-20T01:47:53.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.893 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:53.893 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.752761+0000", 2024-04-20T01:47:53.893 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4901e-05 2024-04-20T01:47:53.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.893 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:53.893 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.752772+0000", 2024-04-20T01:47:53.893 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0961000000000001e-05 2024-04-20T01:47:53.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.893 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:53.894 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.752826+0000", 2024-04-20T01:47:53.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4206000000000002e-05 2024-04-20T01:47:53.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.894 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:53.894 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.752836+0000", 2024-04-20T01:47:53.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7920000000000004e-06 2024-04-20T01:47:53.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.894 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:53.894 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.759223+0000", 2024-04-20T01:47:53.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0063869199999999999 2024-04-20T01:47:53.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.895 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:53.895 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.759252+0000", 2024-04-20T01:47:53.895 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8858000000000001e-05 2024-04-20T01:47:53.895 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.895 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.895 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.895 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.895 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s4 48/30 ECSubWrite(tid=84, reqid=osd.8.0:84, at_version=48'69, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:47:53.895 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.709259+0000", 2024-04-20T01:47:53.895 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.1430794180000001, 2024-04-20T01:47:53.895 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0070121159999999997, 2024-04-20T01:47:53.895 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:53.896 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:53.896 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:53.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:53.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.709259+0000", 2024-04-20T01:47:53.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:53.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.709259+0000", 2024-04-20T01:47:53.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:53.897 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.709257+0000", 2024-04-20T01:47:53.897 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:53.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.897 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:53.897 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.709296+0000", 2024-04-20T01:47:53.897 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8469e-05 2024-04-20T01:47:53.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.897 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:53.897 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.709315+0000", 2024-04-20T01:47:53.897 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9432999999999998e-05 2024-04-20T01:47:53.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.898 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:53.898 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.709326+0000", 2024-04-20T01:47:53.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1083e-05 2024-04-20T01:47:53.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.898 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:53.898 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.709401+0000", 2024-04-20T01:47:53.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5045999999999999e-05 2024-04-20T01:47:53.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.898 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:53.898 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.709412+0000", 2024-04-20T01:47:53.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1106e-05 2024-04-20T01:47:53.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.899 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.899 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:53.899 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.716241+0000", 2024-04-20T01:47:53.899 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068288690000000004 2024-04-20T01:47:53.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.899 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.899 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:53.899 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.716271+0000", 2024-04-20T01:47:53.899 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9532000000000001e-05 2024-04-20T01:47:53.899 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.899 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.899 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.899 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.899 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s4 48/30 ECSubWrite(tid=85, reqid=client.4643.0:399, at_version=48'71, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:47:53.900 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.711859+0000", 2024-04-20T01:47:53.900 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.1404786570000001, 2024-04-20T01:47:53.900 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0082489499999999997, 2024-04-20T01:47:53.900 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:53.900 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:53.900 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:53.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:53.900 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711859+0000", 2024-04-20T01:47:53.900 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:53.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711859+0000", 2024-04-20T01:47:53.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.901 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.901 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:53.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711858+0000", 2024-04-20T01:47:53.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:53.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.901 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.901 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:53.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711887+0000", 2024-04-20T01:47:53.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8487e-05 2024-04-20T01:47:53.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.901 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.901 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:53.902 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711908+0000", 2024-04-20T01:47:53.902 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1625e-05 2024-04-20T01:47:53.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.902 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:53.902 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711926+0000", 2024-04-20T01:47:53.902 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8113000000000001e-05 2024-04-20T01:47:53.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.902 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:53.902 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712063+0000", 2024-04-20T01:47:53.902 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000136907 2024-04-20T01:47:53.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.903 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.903 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:53.903 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712076+0000", 2024-04-20T01:47:53.903 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2145e-05 2024-04-20T01:47:53.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.903 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.903 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:53.903 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.720075+0000", 2024-04-20T01:47:53.903 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0079993730000000006 2024-04-20T01:47:53.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.903 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.903 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:53.903 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.720108+0000", 2024-04-20T01:47:53.903 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3386000000000002e-05 2024-04-20T01:47:53.904 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.904 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.904 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.904 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.904 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s4 48/30 ECSubWrite(tid=86, reqid=client.4643.0:400, at_version=48'72, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:47:53.904 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.713114+0000", 2024-04-20T01:47:53.904 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.1392243950000003, 2024-04-20T01:47:53.904 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0088368690000000007, 2024-04-20T01:47:53.904 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:53.904 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:53.905 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:53.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:53.905 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713114+0000", 2024-04-20T01:47:53.905 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:53.905 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713114+0000", 2024-04-20T01:47:53.905 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:53.905 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713113+0000", 2024-04-20T01:47:53.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:53.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.906 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.906 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:53.906 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713136+0000", 2024-04-20T01:47:53.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2974999999999999e-05 2024-04-20T01:47:53.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.906 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.906 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:53.906 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713149+0000", 2024-04-20T01:47:53.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2945000000000001e-05 2024-04-20T01:47:53.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:53.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713159+0000", 2024-04-20T01:47:53.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9909999999999997e-06 2024-04-20T01:47:53.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:53.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713217+0000", 2024-04-20T01:47:53.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8085999999999999e-05 2024-04-20T01:47:53.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:53.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713228+0000", 2024-04-20T01:47:53.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1345999999999999e-05 2024-04-20T01:47:53.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.908 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:53.908 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.721915+0000", 2024-04-20T01:47:53.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086870850000000006 2024-04-20T01:47:53.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.908 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.908 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:53.908 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.721950+0000", 2024-04-20T01:47:53.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5352999999999998e-05 2024-04-20T01:47:53.908 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.908 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.908 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.908 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.908 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s4 48/30 ECSubWrite(tid=87, reqid=client.4643.0:401, at_version=48'73, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:47:53.909 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.714184+0000", 2024-04-20T01:47:53.909 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.1381540699999997, 2024-04-20T01:47:53.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0077907779999999999, 2024-04-20T01:47:53.909 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:53.909 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:53.909 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:53.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.909 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:53.909 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714184+0000", 2024-04-20T01:47:53.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.909 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:53.909 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714184+0000", 2024-04-20T01:47:53.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:53.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714183+0000", 2024-04-20T01:47:53.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:53.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:53.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714204+0000", 2024-04-20T01:47:53.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0914e-05 2024-04-20T01:47:53.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:53.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714223+0000", 2024-04-20T01:47:53.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9363000000000001e-05 2024-04-20T01:47:53.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.911 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:53.911 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714238+0000", 2024-04-20T01:47:53.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4878000000000001e-05 2024-04-20T01:47:53.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.911 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:53.911 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.716413+0000", 2024-04-20T01:47:53.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021746059999999999 2024-04-20T01:47:53.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.911 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:53.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.716420+0000", 2024-04-20T01:47:53.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9280000000000004e-06 2024-04-20T01:47:53.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:53.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.721961+0000", 2024-04-20T01:47:53.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055417130000000002 2024-04-20T01:47:53.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:53.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.721975+0000", 2024-04-20T01:47:53.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3322999999999999e-05 2024-04-20T01:47:53.912 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.912 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.913 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.913 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.9s2 48/30 ECSubWrite(tid=17, reqid=osd.7.0:17, at_version=48'12, trim_to=0'0, roll_forward_to=47'10))", 2024-04-20T01:47:53.913 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.715213+0000", 2024-04-20T01:47:53.913 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.1371246929999996, 2024-04-20T01:47:53.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.006777188, 2024-04-20T01:47:53.913 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:53.913 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:53.914 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:53.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:53.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.715213+0000", 2024-04-20T01:47:53.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.914 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:53.914 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.715213+0000", 2024-04-20T01:47:53.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:53.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.715213+0000", 2024-04-20T01:47:53.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-04-20T01:47:53.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:53.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.715243+0000", 2024-04-20T01:47:53.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0252000000000001e-05 2024-04-20T01:47:53.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:53.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.715256+0000", 2024-04-20T01:47:53.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2948e-05 2024-04-20T01:47:53.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:53.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.715268+0000", 2024-04-20T01:47:53.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1827999999999999e-05 2024-04-20T01:47:53.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:53.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.715329+0000", 2024-04-20T01:47:53.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0949999999999998e-05 2024-04-20T01:47:53.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:53.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.715340+0000", 2024-04-20T01:47:53.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0499999999999999e-05 2024-04-20T01:47:53.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:53.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.721948+0000", 2024-04-20T01:47:53.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0066079570000000002 2024-04-20T01:47:53.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:53.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.721991+0000", 2024-04-20T01:47:53.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2873000000000001e-05 2024-04-20T01:47:53.918 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.918 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.918 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.919 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs1 48/30 ECSubWrite(tid=121, reqid=osd.4.0:121, at_version=48'53, trim_to=0'0, roll_forward_to=47'49))", 2024-04-20T01:47:53.919 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.718114+0000", 2024-04-20T01:47:53.919 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.1342241279999996, 2024-04-20T01:47:53.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072131039999999997, 2024-04-20T01:47:53.919 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:53.919 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:53.919 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:53.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:53.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.718114+0000", 2024-04-20T01:47:53.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:53.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.718114+0000", 2024-04-20T01:47:53.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:53.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.718113+0000", 2024-04-20T01:47:53.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:53.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:53.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.718135+0000", 2024-04-20T01:47:53.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2214000000000001e-05 2024-04-20T01:47:53.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:53.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.718148+0000", 2024-04-20T01:47:53.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2104e-05 2024-04-20T01:47:53.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:53.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.718157+0000", 2024-04-20T01:47:53.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6330000000000008e-06 2024-04-20T01:47:53.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:53.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.720662+0000", 2024-04-20T01:47:53.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025051040000000002 2024-04-20T01:47:53.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:53.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.720669+0000", 2024-04-20T01:47:53.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.041e-06 2024-04-20T01:47:53.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:53.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.725298+0000", 2024-04-20T01:47:53.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046288329999999997 2024-04-20T01:47:53.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:53.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.725327+0000", 2024-04-20T01:47:53.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8856e-05 2024-04-20T01:47:53.923 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.923 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.923 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.923 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.9s2 52/30 ECSubWrite(tid=21, reqid=client.4643.0:433, at_version=52'15, trim_to=0'0, roll_forward_to=48'14))", 2024-04-20T01:47:53.923 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:53.727940+0000", 2024-04-20T01:47:53.924 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.124398064, 2024-04-20T01:47:53.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0064603500000000001, 2024-04-20T01:47:53.924 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:53.924 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:53.924 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:53.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:53.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.727940+0000", 2024-04-20T01:47:53.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:53.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.727940+0000", 2024-04-20T01:47:53.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:53.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.727940+0000", 2024-04-20T01:47:53.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-04-20T01:47:53.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.925 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:53.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.728214+0000", 2024-04-20T01:47:53.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00027428000000000001 2024-04-20T01:47:53.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:53.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.728244+0000", 2024-04-20T01:47:53.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9490000000000001e-05 2024-04-20T01:47:53.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:53.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.728256+0000", 2024-04-20T01:47:53.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2592000000000001e-05 2024-04-20T01:47:53.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:53.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.728321+0000", 2024-04-20T01:47:53.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4726000000000003e-05 2024-04-20T01:47:53.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:53.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.728334+0000", 2024-04-20T01:47:53.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3217999999999999e-05 2024-04-20T01:47:53.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:53.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.734369+0000", 2024-04-20T01:47:53.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0060346469999999998 2024-04-20T01:47:53.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:53.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.734400+0000", 2024-04-20T01:47:53.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1542999999999999e-05 2024-04-20T01:47:53.929 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.929 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.929 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.929 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs1 52/30 ECSubWrite(tid=124, reqid=client.4643.0:434, at_version=52'55, trim_to=0'0, roll_forward_to=48'53))", 2024-04-20T01:47:53.929 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:53.736586+0000", 2024-04-20T01:47:53.929 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.115751517, 2024-04-20T01:47:53.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0094278839999999992, 2024-04-20T01:47:53.930 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:53.930 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:53.930 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:53.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:53.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736586+0000", 2024-04-20T01:47:53.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:53.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736586+0000", 2024-04-20T01:47:53.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:53.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:53.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736586+0000", 2024-04-20T01:47:53.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:53.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:53.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736681+0000", 2024-04-20T01:47:53.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4703000000000005e-05 2024-04-20T01:47:53.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:53.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736716+0000", 2024-04-20T01:47:53.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5434e-05 2024-04-20T01:47:53.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:53.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736737+0000", 2024-04-20T01:47:53.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0645999999999999e-05 2024-04-20T01:47:53.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:53.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736860+0000", 2024-04-20T01:47:53.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012254099999999999 2024-04-20T01:47:53.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.932 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:53.932 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736872+0000", 2024-04-20T01:47:53.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2824e-05 2024-04-20T01:47:53.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.933 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:53.933 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.745990+0000", 2024-04-20T01:47:53.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0091180250000000001 2024-04-20T01:47:53.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:53.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:53.933 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:53.933 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.746014+0000", 2024-04-20T01:47:53.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3983000000000001e-05 2024-04-20T01:47:53.933 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.933 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.933 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.933 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:53.933 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:53.933 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:53.935 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-20T01:47:53.936 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:47:53.936 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:47:53.936 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:47:53.936 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:47:53.936 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:47:53.949+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:47:53.946 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:47:53.967 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:47:53.988 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:47:54.008 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:47:54.029 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:47:54.035 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:54.050+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:47:54.049 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:47:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:47:54.065+0000 7fa36bfff700 0 -- 172.21.15.44:0/436196784 >> [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] conn(0x7fa36c21f310 msgr2=0x7fa36c221700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-20T01:47:54.090 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:54.090 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:47:54.090 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-04-20T01:47:54.090 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:54.102 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-20T01:47:54.136 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:54.151+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:54.237 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:47:54.252+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:47:54.272 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:47:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:47:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:47:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:47:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:47:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:47:54.275 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:47:54.275 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:47:54.275 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:47:54.275 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:47:54.275 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:47:54.275 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:47:54.275 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:47:54.276 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:47:54.276 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:47:54.276 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:47:54.276 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:47:54.296 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:54.296 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:47:54.296 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-04-20T01:47:54.296 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-04-20T01:47:54.296 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:54.308 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-20T01:47:54.339 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:54.353+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:54.439 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:47:54.454+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:47:54.498 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:54.498 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-04-20T01:47:54.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-04-20T01:47:54.498 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-04-20T01:47:54.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.498 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:210 3.7s0 3:eb044288:::smithi16579500-44:head [append 0~589824] snapc 3=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:47:54.498 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.776114+0000", 2024-04-20T01:47:54.498 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 16.734562706999998, 2024-04-20T01:47:54.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012748944, 2024-04-20T01:47:54.498 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:54.498 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:54.499 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:54.499 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:54.499 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:54.499 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 210 2024-04-20T01:47:54.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.499 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:54.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.499 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:54.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.776114+0000", 2024-04-20T01:47:54.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.499 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:54.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.776114+0000", 2024-04-20T01:47:54.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:54.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.776113+0000", 2024-04-20T01:47:54.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:47:54.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:54.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.776849+0000", 2024-04-20T01:47:54.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00073624699999999999 2024-04-20T01:47:54.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:54.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.776863+0000", 2024-04-20T01:47:54.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3749e-05 2024-04-20T01:47:54.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:54.501 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.776869+0000", 2024-04-20T01:47:54.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8889999999999996e-06 2024-04-20T01:47:54.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.501 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.776935+0000", 2024-04-20T01:47:54.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5565e-05 2024-04-20T01:47:54.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:54.501 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.777079+0000", 2024-04-20T01:47:54.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014380400000000001 2024-04-20T01:47:54.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.502 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:54.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778445+0000", 2024-04-20T01:47:54.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0013661999999999999 2024-04-20T01:47:54.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.502 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:54.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.786334+0000", 2024-04-20T01:47:54.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0078889330000000008 2024-04-20T01:47:54.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.502 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:47:54.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.788782+0000", 2024-04-20T01:47:54.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0024480389999999999 2024-04-20T01:47:54.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.503 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:54.503 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.788863+0000", 2024-04-20T01:47:54.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1198000000000004e-05 2024-04-20T01:47:54.503 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.503 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.503 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.503 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:211 3.7s0 3:eb044288:::smithi16579500-44:head [append 0~376832] snapc 3=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:47:54.503 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.778254+0000", 2024-04-20T01:47:54.503 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 16.732422663000001, 2024-04-20T01:47:54.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01403427, 2024-04-20T01:47:54.504 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:54.504 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:54.504 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:54.504 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:54.504 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:54.504 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 211 2024-04-20T01:47:54.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.504 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:54.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.504 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:54.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778254+0000", 2024-04-20T01:47:54.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:54.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778254+0000", 2024-04-20T01:47:54.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:54.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778252+0000", 2024-04-20T01:47:54.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999976 2024-04-20T01:47:54.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:54.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780895+0000", 2024-04-20T01:47:54.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002642658 2024-04-20T01:47:54.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:54.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780910+0000", 2024-04-20T01:47:54.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4949999999999999e-05 2024-04-20T01:47:54.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:54.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780919+0000", 2024-04-20T01:47:54.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7770000000000008e-06 2024-04-20T01:47:54.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780949+0000", 2024-04-20T01:47:54.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0639999999999998e-05 2024-04-20T01:47:54.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.507 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:54.507 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781077+0000", 2024-04-20T01:47:54.507 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000127565 2024-04-20T01:47:54.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.507 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:54.507 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781928+0000", 2024-04-20T01:47:54.507 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00085169599999999999 2024-04-20T01:47:54.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.507 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:54.507 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.787994+0000", 2024-04-20T01:47:54.507 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00606545 2024-04-20T01:47:54.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.508 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:47:54.508 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.792213+0000", 2024-04-20T01:47:54.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042187199999999996 2024-04-20T01:47:54.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.508 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:54.508 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.792289+0000", 2024-04-20T01:47:54.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5980000000000001e-05 2024-04-20T01:47:54.508 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.508 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.508 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.509 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:212 3.7s0 3:eb044288:::smithi16579500-44:head [setxattr _header (56) in=7b] snapc 3=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:47:54.509 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.780929+0000", 2024-04-20T01:47:54.509 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 16.729748205, 2024-04-20T01:47:54.509 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011579653000000001, 2024-04-20T01:47:54.509 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:54.509 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:54.509 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:54.509 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:54.509 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:54.509 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 212 2024-04-20T01:47:54.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.509 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:54.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:54.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780929+0000", 2024-04-20T01:47:54.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:54.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780929+0000", 2024-04-20T01:47:54.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:54.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780928+0000", 2024-04-20T01:47:54.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:54.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:54.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780933+0000", 2024-04-20T01:47:54.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5600000000000001e-06 2024-04-20T01:47:54.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:54.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780941+0000", 2024-04-20T01:47:54.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8140000000000001e-06 2024-04-20T01:47:54.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:54.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780947+0000", 2024-04-20T01:47:54.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0360000000000003e-06 2024-04-20T01:47:54.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.784823+0000", 2024-04-20T01:47:54.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0038761690000000001 2024-04-20T01:47:54.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:54.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.784935+0000", 2024-04-20T01:47:54.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011201600000000001 2024-04-20T01:47:54.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.512 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:54.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.785422+0000", 2024-04-20T01:47:54.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00048639400000000002 2024-04-20T01:47:54.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.513 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:54.513 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.787998+0000", 2024-04-20T01:47:54.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025759189999999999 2024-04-20T01:47:54.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.513 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:47:54.513 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.792414+0000", 2024-04-20T01:47:54.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0044166960000000003 2024-04-20T01:47:54.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.513 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:54.513 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.792508+0000", 2024-04-20T01:47:54.514 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4232000000000002e-05 2024-04-20T01:47:54.514 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.514 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.514 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.514 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.514 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.514 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:213 3.7s0 3:eb044288:::smithi16579500-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:47:54.514 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.780960+0000", 2024-04-20T01:47:54.514 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 16.729716582000002, 2024-04-20T01:47:54.514 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015313724000000001, 2024-04-20T01:47:54.514 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:54.514 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:47:54.514 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:54.514 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:54.514 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:54.515 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 213 2024-04-20T01:47:54.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.515 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:54.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.515 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:54.515 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780960+0000", 2024-04-20T01:47:54.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.515 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:54.515 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780960+0000", 2024-04-20T01:47:54.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.516 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:54.516 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780960+0000", 2024-04-20T01:47:54.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:54.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.516 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:54.516 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780964+0000", 2024-04-20T01:47:54.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6610000000000002e-06 2024-04-20T01:47:54.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.516 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:54.516 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780971+0000", 2024-04-20T01:47:54.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9270000000000005e-06 2024-04-20T01:47:54.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:54.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780978+0000", 2024-04-20T01:47:54.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1670000000000004e-06 2024-04-20T01:47:54.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.786222+0000", 2024-04-20T01:47:54.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052446979999999999 2024-04-20T01:47:54.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.518 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-04-20T01:47:54.518 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.786324+0000", 2024-04-20T01:47:54.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000101801 2024-04-20T01:47:54.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.518 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.518 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.793221+0000", 2024-04-20T01:47:54.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068970510000000004 2024-04-20T01:47:54.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.518 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:54.518 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.793323+0000", 2024-04-20T01:47:54.519 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102289 2024-04-20T01:47:54.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.519 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.519 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:54.519 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.796274+0000", 2024-04-20T01:47:54.519 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0029506409999999999 2024-04-20T01:47:54.519 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.519 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.519 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.519 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.519 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:214 3.bs0 3:dc0bbe39:::smithi16579500-13:head [stat,delete] snapc 3=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:47:54.519 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.781222+0000", 2024-04-20T01:47:54.520 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 16.729455134999998, 2024-04-20T01:47:54.520 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015584361, 2024-04-20T01:47:54.520 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:54.520 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:54.520 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:54.520 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:54.520 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:54.520 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 214 2024-04-20T01:47:54.520 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.520 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:54.520 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.520 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:54.521 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781222+0000", 2024-04-20T01:47:54.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.521 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:54.521 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781222+0000", 2024-04-20T01:47:54.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.521 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:54.521 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781221+0000", 2024-04-20T01:47:54.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:54.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:54.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781228+0000", 2024-04-20T01:47:54.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6699999999999997e-06 2024-04-20T01:47:54.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:54.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781239+0000", 2024-04-20T01:47:54.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1574e-05 2024-04-20T01:47:54.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:54.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781247+0000", 2024-04-20T01:47:54.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.593e-06 2024-04-20T01:47:54.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781288+0000", 2024-04-20T01:47:54.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1253e-05 2024-04-20T01:47:54.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:54.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781383+0000", 2024-04-20T01:47:54.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4788000000000001e-05 2024-04-20T01:47:54.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:54.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782250+0000", 2024-04-20T01:47:54.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00086664399999999996 2024-04-20T01:47:54.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:54.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.786611+0000", 2024-04-20T01:47:54.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004361714 2024-04-20T01:47:54.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:47:54.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.796739+0000", 2024-04-20T01:47:54.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010127519 2024-04-20T01:47:54.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:54.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.796806+0000", 2024-04-20T01:47:54.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7414000000000002e-05 2024-04-20T01:47:54.525 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.525 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.525 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.525 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:225 3.bs0 3:df90e529:::smithi16579500-45:head [append 0~622592] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:47:54.525 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.785410+0000", 2024-04-20T01:47:54.525 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 15.725266624, 2024-04-20T01:47:54.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013791425, 2024-04-20T01:47:54.525 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:54.525 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:54.525 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:54.525 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:54.526 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:54.526 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 225 2024-04-20T01:47:54.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.526 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:54.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:54.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.785410+0000", 2024-04-20T01:47:54.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:54.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.785410+0000", 2024-04-20T01:47:54.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.527 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:54.527 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.785407+0000", 2024-04-20T01:47:54.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999967 2024-04-20T01:47:54.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.527 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:54.527 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.786402+0000", 2024-04-20T01:47:54.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00099532000000000006 2024-04-20T01:47:54.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.527 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:54.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.786427+0000", 2024-04-20T01:47:54.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5304e-05 2024-04-20T01:47:54.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:54.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.786440+0000", 2024-04-20T01:47:54.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2238999999999999e-05 2024-04-20T01:47:54.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.786515+0000", 2024-04-20T01:47:54.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4923000000000001e-05 2024-04-20T01:47:54.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:54.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.786652+0000", 2024-04-20T01:47:54.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137522 2024-04-20T01:47:54.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:54.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788170+0000", 2024-04-20T01:47:54.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001518003 2024-04-20T01:47:54.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:54.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796858+0000", 2024-04-20T01:47:54.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086877320000000001 2024-04-20T01:47:54.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:47:54.530 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.799118+0000", 2024-04-20T01:47:54.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022599600000000001 2024-04-20T01:47:54.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:54.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.799202+0000", 2024-04-20T01:47:54.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3954000000000003e-05 2024-04-20T01:47:54.531 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.531 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.532 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.532 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:226 3.bs0 3:df90e529:::smithi16579500-45:head [append 0~376832] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:47:54.532 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787415+0000", 2024-04-20T01:47:54.532 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 15.723262352999999, 2024-04-20T01:47:54.532 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012386196, 2024-04-20T01:47:54.532 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:54.532 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:54.532 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:54.532 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:54.533 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:54.533 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 226 2024-04-20T01:47:54.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.533 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:54.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:54.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787415+0000", 2024-04-20T01:47:54.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:54.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787415+0000", 2024-04-20T01:47:54.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:54.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787413+0000", 2024-04-20T01:47:54.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:47:54.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.534 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:54.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787797+0000", 2024-04-20T01:47:54.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00038435999999999999 2024-04-20T01:47:54.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:54.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787813+0000", 2024-04-20T01:47:54.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6235999999999998e-05 2024-04-20T01:47:54.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:54.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787829+0000", 2024-04-20T01:47:54.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5730999999999999e-05 2024-04-20T01:47:54.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.536 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.789244+0000", 2024-04-20T01:47:54.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001414546 2024-04-20T01:47:54.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:54.536 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.789361+0000", 2024-04-20T01:47:54.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011783200000000001 2024-04-20T01:47:54.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:54.536 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.790143+0000", 2024-04-20T01:47:54.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00078198299999999998 2024-04-20T01:47:54.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:54.537 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796916+0000", 2024-04-20T01:47:54.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0067724949999999999 2024-04-20T01:47:54.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.537 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:47:54.537 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.799764+0000", 2024-04-20T01:47:54.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002848176 2024-04-20T01:47:54.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.537 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:54.537 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.799801+0000", 2024-04-20T01:47:54.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6715999999999999e-05 2024-04-20T01:47:54.537 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.537 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.538 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.538 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:227 3.bs0 3:df90e529:::smithi16579500-45:head [setxattr _header (56) in=7b] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:47:54.538 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787843+0000", 2024-04-20T01:47:54.538 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 15.722834346999999, 2024-04-20T01:47:54.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014686189000000001, 2024-04-20T01:47:54.538 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:54.538 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:54.538 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:54.538 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:54.538 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:54.538 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 227 2024-04-20T01:47:54.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.539 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:54.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:54.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787843+0000", 2024-04-20T01:47:54.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:54.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787843+0000", 2024-04-20T01:47:54.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:54.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787842+0000", 2024-04-20T01:47:54.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:54.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:54.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787847+0000", 2024-04-20T01:47:54.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8130000000000001e-06 2024-04-20T01:47:54.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:54.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787853+0000", 2024-04-20T01:47:54.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5119999999999999e-06 2024-04-20T01:47:54.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:54.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787859+0000", 2024-04-20T01:47:54.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8119999999999996e-06 2024-04-20T01:47:54.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.541 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.541 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.790718+0000", 2024-04-20T01:47:54.541 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002859267 2024-04-20T01:47:54.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.541 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:54.541 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.790827+0000", 2024-04-20T01:47:54.541 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000109471 2024-04-20T01:47:54.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.541 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:54.541 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791236+0000", 2024-04-20T01:47:54.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000408637 2024-04-20T01:47:54.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.542 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:54.542 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796922+0000", 2024-04-20T01:47:54.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0056861139999999999 2024-04-20T01:47:54.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.542 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:47:54.542 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.802464+0000", 2024-04-20T01:47:54.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055420449999999998 2024-04-20T01:47:54.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.542 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:54.543 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.802529+0000", 2024-04-20T01:47:54.543 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4554000000000006e-05 2024-04-20T01:47:54.543 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.543 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.543 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.543 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:228 3.bs0 3:df90e529:::smithi16579500-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:47:54.543 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787871+0000", 2024-04-20T01:47:54.543 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 15.722805632, 2024-04-20T01:47:54.543 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.016315307000000001, 2024-04-20T01:47:54.543 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:54.543 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:47:54.543 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:54.544 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:54.544 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:54.544 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 228 2024-04-20T01:47:54.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.544 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:54.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.544 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:54.544 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787871+0000", 2024-04-20T01:47:54.544 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.544 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:54.544 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787871+0000", 2024-04-20T01:47:54.544 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.545 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:54.545 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787871+0000", 2024-04-20T01:47:54.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:54.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.545 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:54.545 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787873+0000", 2024-04-20T01:47:54.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.08e-06 2024-04-20T01:47:54.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.545 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:54.545 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787878+0000", 2024-04-20T01:47:54.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7299999999999996e-06 2024-04-20T01:47:54.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.546 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:54.546 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787881+0000", 2024-04-20T01:47:54.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5209999999999998e-06 2024-04-20T01:47:54.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.546 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.546 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791461+0000", 2024-04-20T01:47:54.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003579905 2024-04-20T01:47:54.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-04-20T01:47:54.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791554+0000", 2024-04-20T01:47:54.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2485000000000006e-05 2024-04-20T01:47:54.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.802822+0000", 2024-04-20T01:47:54.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011268251 2024-04-20T01:47:54.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:54.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.802922+0000", 2024-04-20T01:47:54.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010029700000000001 2024-04-20T01:47:54.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:54.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.804187+0000", 2024-04-20T01:47:54.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001264362 2024-04-20T01:47:54.548 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.548 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.548 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.548 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:237 3.bs0 3:da81107c:::smithi16579500-34:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:47:54.548 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787884+0000", 2024-04-20T01:47:54.548 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 15.722792878, 2024-04-20T01:47:54.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011811317, 2024-04-20T01:47:54.549 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:54.549 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:47:54.549 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:54.549 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:54.549 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:54.549 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 237 2024-04-20T01:47:54.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.549 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:54.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:54.549 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787884+0000", 2024-04-20T01:47:54.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.550 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:54.550 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787884+0000", 2024-04-20T01:47:54.550 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.550 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.550 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:54.550 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787884+0000", 2024-04-20T01:47:54.550 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:54.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.550 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.550 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:54.550 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787891+0000", 2024-04-20T01:47:54.550 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7070000000000003e-06 2024-04-20T01:47:54.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.551 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:54.551 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787895+0000", 2024-04-20T01:47:54.551 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3900000000000003e-06 2024-04-20T01:47:54.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.551 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:54.551 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787901+0000", 2024-04-20T01:47:54.551 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4359999999999999e-06 2024-04-20T01:47:54.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791590+0000", 2024-04-20T01:47:54.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0036883620000000002 2024-04-20T01:47:54.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:54.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791675+0000", 2024-04-20T01:47:54.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4919999999999993e-05 2024-04-20T01:47:54.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:54.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.799695+0000", 2024-04-20T01:47:54.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0080207469999999999 2024-04-20T01:47:54.552 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.553 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.553 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.553 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:238 3.bs0 3:da81107c:::smithi16579500-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:47:54.553 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787905+0000", 2024-04-20T01:47:54.553 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 15.722771549000001, 2024-04-20T01:47:54.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012509012, 2024-04-20T01:47:54.553 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:54.553 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:47:54.553 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:54.553 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:54.554 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:54.554 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 238 2024-04-20T01:47:54.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.554 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:54.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:54.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787905+0000", 2024-04-20T01:47:54.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:54.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787905+0000", 2024-04-20T01:47:54.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.555 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:54.555 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787905+0000", 2024-04-20T01:47:54.555 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-04-20T01:47:54.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.555 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:54.555 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787913+0000", 2024-04-20T01:47:54.555 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4989999999999997e-06 2024-04-20T01:47:54.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.556 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:54.556 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787918+0000", 2024-04-20T01:47:54.556 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0170000000000004e-06 2024-04-20T01:47:54.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.556 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:54.556 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787920+0000", 2024-04-20T01:47:54.556 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.74e-06 2024-04-20T01:47:54.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.556 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.556 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.792326+0000", 2024-04-20T01:47:54.556 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004405544 2024-04-20T01:47:54.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:54.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.792420+0000", 2024-04-20T01:47:54.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3760999999999999e-05 2024-04-20T01:47:54.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:54.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.800414+0000", 2024-04-20T01:47:54.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0079946849999999996 2024-04-20T01:47:54.557 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.557 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.557 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.558 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:239 3.bs0 3:da81107c:::smithi16579500-34:head [read 0~557056,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:47:54.558 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787923+0000", 2024-04-20T01:47:54.558 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 15.722754093000001, 2024-04-20T01:47:54.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012954693, 2024-04-20T01:47:54.558 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:54.558 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:47:54.558 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:54.558 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:54.558 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:54.558 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 239 2024-04-20T01:47:54.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.559 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:54.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.559 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:54.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787923+0000", 2024-04-20T01:47:54.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.559 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:54.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787923+0000", 2024-04-20T01:47:54.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.560 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:54.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787923+0000", 2024-04-20T01:47:54.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-04-20T01:47:54.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.560 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.560 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:54.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787924+0000", 2024-04-20T01:47:54.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7179999999999999e-06 2024-04-20T01:47:54.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.560 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.560 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:54.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787928+0000", 2024-04-20T01:47:54.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3950000000000001e-06 2024-04-20T01:47:54.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.561 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:54.561 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787930+0000", 2024-04-20T01:47:54.561 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6240000000000002e-06 2024-04-20T01:47:54.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.561 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.561 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.794417+0000", 2024-04-20T01:47:54.561 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0064870850000000001 2024-04-20T01:47:54.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.561 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:54.562 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.794496+0000", 2024-04-20T01:47:54.562 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8215999999999998e-05 2024-04-20T01:47:54.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:54.562 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.800878+0000", 2024-04-20T01:47:54.562 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0063818870000000001 2024-04-20T01:47:54.562 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.562 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.562 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.562 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.0s6 37/30 ECSubWrite(tid=19, reqid=client.4643.0:235, at_version=37'9, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:54.563 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.796192+0000", 2024-04-20T01:47:54.563 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 15.714484503, 2024-04-20T01:47:54.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010261599, 2024-04-20T01:47:54.563 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:54.563 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:54.563 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:54.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.563 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:54.563 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796192+0000", 2024-04-20T01:47:54.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:54.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796192+0000", 2024-04-20T01:47:54.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:54.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796192+0000", 2024-04-20T01:47:54.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:54.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:54.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796214+0000", 2024-04-20T01:47:54.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1387999999999999e-05 2024-04-20T01:47:54.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:54.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796223+0000", 2024-04-20T01:47:54.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5990000000000008e-06 2024-04-20T01:47:54.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:54.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796232+0000", 2024-04-20T01:47:54.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9360000000000005e-06 2024-04-20T01:47:54.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.797530+0000", 2024-04-20T01:47:54.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0012980699999999999 2024-04-20T01:47:54.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:54.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.797539+0000", 2024-04-20T01:47:54.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0569999999999999e-06 2024-04-20T01:47:54.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:54.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.806434+0000", 2024-04-20T01:47:54.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0088950049999999992 2024-04-20T01:47:54.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.567 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:54.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.806454+0000", 2024-04-20T01:47:54.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9851e-05 2024-04-20T01:47:54.567 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.567 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.567 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.567 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.0s6 37/30 ECSubWrite(tid=20, reqid=client.4643.0:236, at_version=37'10, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:54.567 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.796240+0000", 2024-04-20T01:47:54.567 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 15.714437222000001, 2024-04-20T01:47:54.568 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010230095999999999, 2024-04-20T01:47:54.568 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:54.568 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:54.568 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:54.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.568 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:54.568 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796240+0000", 2024-04-20T01:47:54.568 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.568 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:54.568 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796240+0000", 2024-04-20T01:47:54.568 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.569 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:54.569 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796239+0000", 2024-04-20T01:47:54.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:54.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.569 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:54.569 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796259+0000", 2024-04-20T01:47:54.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9190000000000001e-05 2024-04-20T01:47:54.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:54.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796268+0000", 2024-04-20T01:47:54.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3870000000000001e-06 2024-04-20T01:47:54.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:54.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796278+0000", 2024-04-20T01:47:54.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6490000000000003e-06 2024-04-20T01:47:54.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.800317+0000", 2024-04-20T01:47:54.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0040391660000000003 2024-04-20T01:47:54.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:54.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.800327+0000", 2024-04-20T01:47:54.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0217000000000001e-05 2024-04-20T01:47:54.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:54.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.806458+0000", 2024-04-20T01:47:54.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0061307610000000002 2024-04-20T01:47:54.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:54.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.806470+0000", 2024-04-20T01:47:54.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2035000000000001e-05 2024-04-20T01:47:54.572 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.572 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.572 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.572 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:385 3.1s0 3:8bb12819:::smithi16579500-21:head [copy-from ver 2] snapc d=[c,b] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-20T01:47:54.572 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.710205+0000", 2024-04-20T01:47:54.572 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.800471581, 2024-04-20T01:47:54.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.016348672000000002, 2024-04-20T01:47:54.572 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:54.573 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:54.573 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:54.573 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:54.573 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:54.573 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 385 2024-04-20T01:47:54.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.573 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:54.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.573 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:54.573 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710205+0000", 2024-04-20T01:47:54.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.573 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:54.574 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710205+0000", 2024-04-20T01:47:54.574 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.574 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:54.574 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710204+0000", 2024-04-20T01:47:54.574 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:54.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.574 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:54.574 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710212+0000", 2024-04-20T01:47:54.574 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8250000000000005e-06 2024-04-20T01:47:54.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:54.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710222+0000", 2024-04-20T01:47:54.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0071000000000001e-05 2024-04-20T01:47:54.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:54.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710232+0000", 2024-04-20T01:47:54.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0583e-05 2024-04-20T01:47:54.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710273+0000", 2024-04-20T01:47:54.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0407999999999997e-05 2024-04-20T01:47:54.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.576 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:54.576 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710360+0000", 2024-04-20T01:47:54.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6778000000000002e-05 2024-04-20T01:47:54.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.576 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:54.576 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719733+0000", 2024-04-20T01:47:54.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0093733800000000006 2024-04-20T01:47:54.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.576 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:54.576 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.723689+0000", 2024-04-20T01:47:54.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0039562620000000003 2024-04-20T01:47:54.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:47:54.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.726462+0000", 2024-04-20T01:47:54.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027724989999999999 2024-04-20T01:47:54.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:54.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.726554+0000", 2024-04-20T01:47:54.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2262000000000005e-05 2024-04-20T01:47:54.577 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.577 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.577 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.578 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:386 3.1s0 3:8bb12819:::smithi16579500-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-04-20T01:47:54.578 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.710241+0000", 2024-04-20T01:47:54.578 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.8004355670000001, 2024-04-20T01:47:54.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011009879, 2024-04-20T01:47:54.578 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:54.578 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:47:54.578 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:54.578 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:54.578 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:54.578 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 386 2024-04-20T01:47:54.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.578 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:54.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.579 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:54.579 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710241+0000", 2024-04-20T01:47:54.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.579 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:54.579 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710241+0000", 2024-04-20T01:47:54.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.579 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:54.579 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710241+0000", 2024-04-20T01:47:54.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:54.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:54.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710243+0000", 2024-04-20T01:47:54.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4279999999999999e-06 2024-04-20T01:47:54.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:54.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710250+0000", 2024-04-20T01:47:54.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2899999999999999e-06 2024-04-20T01:47:54.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:54.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710253+0000", 2024-04-20T01:47:54.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5939999999999998e-06 2024-04-20T01:47:54.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710534+0000", 2024-04-20T01:47:54.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00028018900000000002 2024-04-20T01:47:54.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for blocked object", 2024-04-20T01:47:54.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710587+0000", 2024-04-20T01:47:54.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3934000000000002e-05 2024-04-20T01:47:54.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.721025+0000", 2024-04-20T01:47:54.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010437251 2024-04-20T01:47:54.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:54.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.721136+0000", 2024-04-20T01:47:54.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000111506 2024-04-20T01:47:54.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:54.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.721251+0000", 2024-04-20T01:47:54.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000115068 2024-04-20T01:47:54.582 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.582 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.583 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.583 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s2 48/30 ECSubWrite(tid=85, reqid=client.4643.0:399, at_version=48'71, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:47:54.583 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.711991+0000", 2024-04-20T01:47:54.583 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.798685925, 2024-04-20T01:47:54.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013597662, 2024-04-20T01:47:54.583 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:54.583 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:54.583 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:54.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:54.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711991+0000", 2024-04-20T01:47:54.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:54.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711991+0000", 2024-04-20T01:47:54.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:54.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711990+0000", 2024-04-20T01:47:54.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:54.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:54.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712015+0000", 2024-04-20T01:47:54.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4516999999999999e-05 2024-04-20T01:47:54.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:54.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712032+0000", 2024-04-20T01:47:54.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7425999999999999e-05 2024-04-20T01:47:54.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:54.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712042+0000", 2024-04-20T01:47:54.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9000000000000001e-06 2024-04-20T01:47:54.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713222+0000", 2024-04-20T01:47:54.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001179776 2024-04-20T01:47:54.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:54.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713248+0000", 2024-04-20T01:47:54.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5990999999999999e-05 2024-04-20T01:47:54.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:54.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.725562+0000", 2024-04-20T01:47:54.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01231369 2024-04-20T01:47:54.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:54.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.725589+0000", 2024-04-20T01:47:54.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7137000000000001e-05 2024-04-20T01:47:54.587 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.587 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.587 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.587 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s2 48/30 ECSubWrite(tid=86, reqid=client.4643.0:400, at_version=48'72, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:47:54.588 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.713256+0000", 2024-04-20T01:47:54.588 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.7974208330000003, 2024-04-20T01:47:54.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.017021191000000001, 2024-04-20T01:47:54.588 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:54.588 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:54.588 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:54.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:54.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713256+0000", 2024-04-20T01:47:54.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:54.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713256+0000", 2024-04-20T01:47:54.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:54.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713255+0000", 2024-04-20T01:47:54.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:54.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:54.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713277+0000", 2024-04-20T01:47:54.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1348e-05 2024-04-20T01:47:54.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:54.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713289+0000", 2024-04-20T01:47:54.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2692e-05 2024-04-20T01:47:54.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:54.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713299+0000", 2024-04-20T01:47:54.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0071999999999999e-05 2024-04-20T01:47:54.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714467+0000", 2024-04-20T01:47:54.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001167581 2024-04-20T01:47:54.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:54.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714478+0000", 2024-04-20T01:47:54.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1029999999999999e-05 2024-04-20T01:47:54.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:54.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730237+0000", 2024-04-20T01:47:54.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015758635 2024-04-20T01:47:54.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:54.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730277+0000", 2024-04-20T01:47:54.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0788999999999999e-05 2024-04-20T01:47:54.592 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.592 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.592 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.592 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s2 48/30 ECSubWrite(tid=87, reqid=client.4643.0:401, at_version=48'73, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:47:54.592 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.714498+0000", 2024-04-20T01:47:54.592 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.7961792399999998, 2024-04-20T01:47:54.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015921002, 2024-04-20T01:47:54.592 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:54.592 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:54.593 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:54.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:54.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714498+0000", 2024-04-20T01:47:54.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:54.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714498+0000", 2024-04-20T01:47:54.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:54.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714497+0000", 2024-04-20T01:47:54.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:54.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:54.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714519+0000", 2024-04-20T01:47:54.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2574000000000001e-05 2024-04-20T01:47:54.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:54.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714531+0000", 2024-04-20T01:47:54.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1361e-05 2024-04-20T01:47:54.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:54.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714540+0000", 2024-04-20T01:47:54.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4769999999999992e-06 2024-04-20T01:47:54.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.716648+0000", 2024-04-20T01:47:54.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002108183 2024-04-20T01:47:54.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:54.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.716661+0000", 2024-04-20T01:47:54.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2490000000000001e-05 2024-04-20T01:47:54.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:54.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730395+0000", 2024-04-20T01:47:54.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013734333 2024-04-20T01:47:54.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:54.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730419+0000", 2024-04-20T01:47:54.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3489000000000002e-05 2024-04-20T01:47:54.596 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.596 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.596 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.596 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.3s2 48/30 ECSubWrite(tid=254, reqid=osd.13.0:251, at_version=0'0, trim_to=0'0, roll_forward_to=48'22))", 2024-04-20T01:47:54.597 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.719206+0000", 2024-04-20T01:47:54.597 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.7914707410000004, 2024-04-20T01:47:54.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014281428000000001, 2024-04-20T01:47:54.597 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:54.597 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:54.597 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:54.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:54.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719206+0000", 2024-04-20T01:47:54.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:54.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719206+0000", 2024-04-20T01:47:54.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:54.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:54.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719206+0000", 2024-04-20T01:47:54.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:54.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:54.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719215+0000", 2024-04-20T01:47:54.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6229999999999992e-06 2024-04-20T01:47:54.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.599 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:54.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719237+0000", 2024-04-20T01:47:54.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1877000000000001e-05 2024-04-20T01:47:54.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.599 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:54.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719249+0000", 2024-04-20T01:47:54.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1736000000000001e-05 2024-04-20T01:47:54.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.599 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:54.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.726201+0000", 2024-04-20T01:47:54.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0069514930000000004 2024-04-20T01:47:54.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:54.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.726206+0000", 2024-04-20T01:47:54.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7830000000000004e-06 2024-04-20T01:47:54.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:54.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.733466+0000", 2024-04-20T01:47:54.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072593550000000003 2024-04-20T01:47:54.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:54.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:54.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:54.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.733488+0000", 2024-04-20T01:47:54.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2007000000000001e-05 2024-04-20T01:47:54.600 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.601 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.601 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.601 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:54.601 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:54.601 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:54.603 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-20T01:47:54.603 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:47:54.617+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:47:54.695 INFO:tasks.rados.rados.0.smithi165.stdout:179: done (3 left) 2024-04-20T01:47:54.695 INFO:tasks.rados.rados.0.smithi165.stdout:182: done (2 left) 2024-04-20T01:47:54.695 INFO:tasks.rados.rados.0.smithi165.stdout:183: done (1 left) 2024-04-20T01:47:54.695 INFO:tasks.rados.rados.0.smithi165.stdout:184: done (0 left) 2024-04-20T01:47:54.695 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 30 current snap is 12 2024-04-20T01:47:54.695 INFO:tasks.rados.rados.0.smithi165.stdout:185: seq_num 88 ranges {0=720896,720896=442368,1163264=327680} 2024-04-20T01:47:54.698 INFO:tasks.rados.rados.0.smithi165.stdout:185: writing smithi16579500-30 from 0 to 720896 tid 1 2024-04-20T01:47:54.701 INFO:tasks.rados.rados.0.smithi165.stdout:185: writing smithi16579500-30 from 720896 to 1163264 tid 2 2024-04-20T01:47:54.703 INFO:tasks.rados.rados.0.smithi165.stdout:185: writing smithi16579500-30 from 1163264 to 1490944 tid 3 2024-04-20T01:47:54.704 INFO:tasks.rados.rados.0.smithi165.stdout:186: snap_remove snap 11 2024-04-20T01:47:54.704 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:47:54.718+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:47:54.798 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:54.799 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:47:54.799 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-04-20T01:47:54.799 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:54.804 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:54.819+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:54.810 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-20T01:47:54.905 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:47:54.919+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:47:54.955 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:54.956 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:47:54.956 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-04-20T01:47:54.956 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-04-20T01:47:54.956 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:54.967 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-20T01:47:55.006 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:55.021+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:55.108 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:55.122+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:55.157 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:55.157 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-04-20T01:47:55.157 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-04-20T01:47:55.157 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-04-20T01:47:55.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.158 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.7s4 36/30 ECSubWrite(tid=57, reqid=client.4643.0:210, at_version=36'7, trim_to=0'0, roll_forward_to=33'5))", 2024-04-20T01:47:55.158 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.778787+0000", 2024-04-20T01:47:55.158 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 17.391404323, 2024-04-20T01:47:55.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0078786210000000006, 2024-04-20T01:47:55.158 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.158 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.158 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.158 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.158 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778787+0000", 2024-04-20T01:47:55.158 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.158 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778787+0000", 2024-04-20T01:47:55.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.159 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778787+0000", 2024-04-20T01:47:55.159 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-04-20T01:47:55.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.159 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.159 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778852+0000", 2024-04-20T01:47:55.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4950999999999995e-05 2024-04-20T01:47:55.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778875+0000", 2024-04-20T01:47:55.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.247e-05 2024-04-20T01:47:55.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.160 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.160 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778886+0000", 2024-04-20T01:47:55.160 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1756999999999999e-05 2024-04-20T01:47:55.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.779834+0000", 2024-04-20T01:47:55.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00094809500000000004 2024-04-20T01:47:55.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.779847+0000", 2024-04-20T01:47:55.161 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.24e-05 2024-04-20T01:47:55.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.161 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.161 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.786624+0000", 2024-04-20T01:47:55.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0067774719999999997 2024-04-20T01:47:55.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.162 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.162 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.786666+0000", 2024-04-20T01:47:55.162 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1705000000000001e-05 2024-04-20T01:47:55.162 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.162 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.162 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.162 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.7s4 36/30 ECSubWrite(tid=58, reqid=client.4643.0:211, at_version=36'8, trim_to=0'0, roll_forward_to=33'5))", 2024-04-20T01:47:55.162 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.782105+0000", 2024-04-20T01:47:55.162 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 17.388086270999999, 2024-04-20T01:47:55.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096109860000000002, 2024-04-20T01:47:55.163 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.163 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.163 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782105+0000", 2024-04-20T01:47:55.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.163 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782105+0000", 2024-04-20T01:47:55.163 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.163 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782104+0000", 2024-04-20T01:47:55.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:55.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782192+0000", 2024-04-20T01:47:55.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7806000000000006e-05 2024-04-20T01:47:55.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.164 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782206+0000", 2024-04-20T01:47:55.164 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3767e-05 2024-04-20T01:47:55.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.164 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.165 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782217+0000", 2024-04-20T01:47:55.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0808999999999999e-05 2024-04-20T01:47:55.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.165 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.165 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782307+0000", 2024-04-20T01:47:55.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0904999999999998e-05 2024-04-20T01:47:55.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.165 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.165 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782319+0000", 2024-04-20T01:47:55.165 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1932e-05 2024-04-20T01:47:55.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.791680+0000", 2024-04-20T01:47:55.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.009360808 2024-04-20T01:47:55.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.166 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.166 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.791716+0000", 2024-04-20T01:47:55.166 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6186000000000002e-05 2024-04-20T01:47:55.166 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.166 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.166 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.166 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:215 3.4s0 3:28069c2b:::smithi16579500-39:head [copy-from ver 4] snapc 3=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:47:55.167 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.797216+0000", 2024-04-20T01:47:55.167 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 17.372975675999999, 2024-04-20T01:47:55.167 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.020029964000000001, 2024-04-20T01:47:55.167 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.167 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:55.167 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:55.167 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:55.167 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:55.167 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 215 2024-04-20T01:47:55.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.167 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.167 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797216+0000", 2024-04-20T01:47:55.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797216+0000", 2024-04-20T01:47:55.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.168 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797214+0000", 2024-04-20T01:47:55.168 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999976 2024-04-20T01:47:55.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.168 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797223+0000", 2024-04-20T01:47:55.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9709999999999993e-06 2024-04-20T01:47:55.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797240+0000", 2024-04-20T01:47:55.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6872e-05 2024-04-20T01:47:55.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.169 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.169 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797249+0000", 2024-04-20T01:47:55.169 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1500000000000005e-06 2024-04-20T01:47:55.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.170 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.170 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797298+0000", 2024-04-20T01:47:55.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9715000000000001e-05 2024-04-20T01:47:55.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.170 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:55.170 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797408+0000", 2024-04-20T01:47:55.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000109883 2024-04-20T01:47:55.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.170 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.170 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808715+0000", 2024-04-20T01:47:55.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011306145 2024-04-20T01:47:55.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.171 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.171 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.812376+0000", 2024-04-20T01:47:55.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0036618029999999999 2024-04-20T01:47:55.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.171 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:47:55.171 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.817198+0000", 2024-04-20T01:47:55.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048218749999999998 2024-04-20T01:47:55.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.171 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.171 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.817246+0000", 2024-04-20T01:47:55.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7787000000000002e-05 2024-04-20T01:47:55.172 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.172 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.172 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.172 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:216 3.4s0 3:28069c2b:::smithi16579500-39:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:47:55.172 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.797261+0000", 2024-04-20T01:47:55.172 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 17.372930948, 2024-04-20T01:47:55.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012431373000000001, 2024-04-20T01:47:55.172 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.172 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:47:55.172 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:55.172 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:55.172 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:55.173 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 216 2024-04-20T01:47:55.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.173 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797261+0000", 2024-04-20T01:47:55.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797261+0000", 2024-04-20T01:47:55.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797260+0000", 2024-04-20T01:47:55.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:55.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797266+0000", 2024-04-20T01:47:55.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2140000000000001e-06 2024-04-20T01:47:55.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797275+0000", 2024-04-20T01:47:55.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9309999999999997e-06 2024-04-20T01:47:55.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797282+0000", 2024-04-20T01:47:55.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5169999999999996e-06 2024-04-20T01:47:55.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797635+0000", 2024-04-20T01:47:55.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035326500000000001 2024-04-20T01:47:55.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for blocked object", 2024-04-20T01:47:55.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797689+0000", 2024-04-20T01:47:55.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3687999999999998e-05 2024-04-20T01:47:55.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.809488+0000", 2024-04-20T01:47:55.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011799417 2024-04-20T01:47:55.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.176 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:55.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.809591+0000", 2024-04-20T01:47:55.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102729 2024-04-20T01:47:55.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.176 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.176 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.809692+0000", 2024-04-20T01:47:55.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000101202 2024-04-20T01:47:55.176 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.176 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.176 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.177 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:217 3.4s0 3:27f742c3:::smithi16579500-41:head [create,setxattr STaaXsn_tt`[noPeao^kO]r]qhc]OnRdXWqXa]RQ (421) in=40b,setxattr QccpemYdNR_fUkaT_Zk (1613) in=19b,setxattr e]rWtr\\spa\\[`\\etXP]UVYl (1613) in=23b,setxattr ctjQdgZSs[oQ^YekRX]`P_kg`lYfeo\\]jOW\\Ncgj (105) in=40b,setxattr ]UUsac^QNTgnq\\c_amO]eQ^UmNmdPVNbpsche]nq (1366) in=40b,setxattr tsV]fVpQmWmtatroVb]qjW\\n\\^hTba^e (1613) in=32b,setxattr plUORnNkpnPqade]gn^TNqootWi^\\Ygrc[O[dZfX (105) in=40b,setxattr ]aq]PelqnUhOnhUcccRY_o`UqoXf`_^fRi`adYdm (630) in=40b,setxattr lobeSZbbYgcdsPag__QR]`_ih[ (1613) in=26b,setxattr cmbU]Vpc[onnejbfSlj]qNT]`Yspgre^bQ[go^so (138) in=40b,setxattr pkmgTn^hlhmYnjkbSslNYhkOctl_ctnq]^Wm (1613) in=36b,setxattr jrdhlkfStVpRdk]N[rjbfWgQ_RYkY^ZYdg]\\l[dV (1367) in=40b,setxattr `jS`f[NpneY_j`inTVdpWq`ntO_diaj[folX`ibo (1473) in=40b,setxattr aj_mddZOpYlRadmQjbSnSRVdY\\gTgZ]RNfWSSbbo (1858) in=40b,setxattr YegfaljqWjdPp\\]^ejiXpWObnZd]dQTlihpU_Q`q (1649) in=40b,setxattr UsbSqlQaZndaoThjTR\\i_ei]Zs[hUpZasNRfZkaO (1611) in=40b,setxattr oZqjdnYqTXfkcaTmagUaNpZTgXfVkphqmiUTlmOf (1612) in=40b,setxattr [PdlRk]`\\iiigl[fRo`obkeWlZ]W`eqS[nmRn[kg (841) in=40b,setxattr eQXfldiUZkeo^Sjbkb_shi[kNcVQe^W[loVl[g^d (1753) in=40b,setxattr aYNbp]cNSQRqR\\PWsdUPp[Qp]`Zl\\fdnSPeUOdn\\ (13964) in=40b] snapc 3=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:47:55.177 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.798958+0000", 2024-04-20T01:47:55.177 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 17.371233614000001, 2024-04-20T01:47:55.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0078906770000000005, 2024-04-20T01:47:55.177 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.177 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:55.177 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:55.177 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:55.177 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:55.177 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 217 2024-04-20T01:47:55.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.177 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.798958+0000", 2024-04-20T01:47:55.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.798958+0000", 2024-04-20T01:47:55.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.798956+0000", 2024-04-20T01:47:55.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:47:55.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799150+0000", 2024-04-20T01:47:55.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000193735 2024-04-20T01:47:55.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799162+0000", 2024-04-20T01:47:55.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1705999999999999e-05 2024-04-20T01:47:55.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799170+0000", 2024-04-20T01:47:55.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1480000000000006e-06 2024-04-20T01:47:55.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799329+0000", 2024-04-20T01:47:55.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000158622 2024-04-20T01:47:55.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.180 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:55.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.799445+0000", 2024-04-20T01:47:55.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000116106 2024-04-20T01:47:55.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.180 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.800456+0000", 2024-04-20T01:47:55.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0010113190000000001 2024-04-20T01:47:55.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.180 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.804628+0000", 2024-04-20T01:47:55.181 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0041720530000000002 2024-04-20T01:47:55.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.181 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:47:55.181 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.806746+0000", 2024-04-20T01:47:55.181 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021178870000000002 2024-04-20T01:47:55.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.181 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.181 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.806849+0000", 2024-04-20T01:47:55.181 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102847 2024-04-20T01:47:55.181 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.181 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.181 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.182 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:240 3.4s0 3:28069c2b:::smithi16579500-39:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:47:55.182 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787662+0000", 2024-04-20T01:47:55.182 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 16.382529535, 2024-04-20T01:47:55.182 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013455973, 2024-04-20T01:47:55.182 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.182 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:55.182 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:55.182 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:55.182 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:55.182 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 240 2024-04-20T01:47:55.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.182 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787662+0000", 2024-04-20T01:47:55.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787662+0000", 2024-04-20T01:47:55.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787659+0000", 2024-04-20T01:47:55.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999971 2024-04-20T01:47:55.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787676+0000", 2024-04-20T01:47:55.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6371999999999998e-05 2024-04-20T01:47:55.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787703+0000", 2024-04-20T01:47:55.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6902000000000001e-05 2024-04-20T01:47:55.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787718+0000", 2024-04-20T01:47:55.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5231999999999999e-05 2024-04-20T01:47:55.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788023+0000", 2024-04-20T01:47:55.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000304825 2024-04-20T01:47:55.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op must wait for map", 2024-04-20T01:47:55.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788031+0000", 2024-04-20T01:47:55.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7560000000000005e-06 2024-04-20T01:47:55.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791096+0000", 2024-04-20T01:47:55.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0030642849999999999 2024-04-20T01:47:55.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.186 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:55.186 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791195+0000", 2024-04-20T01:47:55.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9277000000000005e-05 2024-04-20T01:47:55.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.186 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.186 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.792028+0000", 2024-04-20T01:47:55.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00083305200000000003 2024-04-20T01:47:55.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.186 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.186 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.800720+0000", 2024-04-20T01:47:55.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086918859999999994 2024-04-20T01:47:55.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.187 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:47:55.187 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.801035+0000", 2024-04-20T01:47:55.187 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00031470199999999998 2024-04-20T01:47:55.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.187 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.187 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.801118+0000", 2024-04-20T01:47:55.187 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3541999999999994e-05 2024-04-20T01:47:55.187 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.187 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.187 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.187 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:241 3.4s0 3:28069c2b:::smithi16579500-39:head [rollback 2] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:47:55.187 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787738+0000", 2024-04-20T01:47:55.188 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 16.382453593000001, 2024-04-20T01:47:55.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01358739, 2024-04-20T01:47:55.188 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.188 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:55.188 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:55.188 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:55.188 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:55.188 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 241 2024-04-20T01:47:55.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.188 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787738+0000", 2024-04-20T01:47:55.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.189 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.189 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787738+0000", 2024-04-20T01:47:55.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.189 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.189 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787737+0000", 2024-04-20T01:47:55.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:55.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.189 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.189 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787742+0000", 2024-04-20T01:47:55.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4050000000000002e-06 2024-04-20T01:47:55.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787751+0000", 2024-04-20T01:47:55.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3710000000000006e-06 2024-04-20T01:47:55.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787757+0000", 2024-04-20T01:47:55.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2600000000000002e-06 2024-04-20T01:47:55.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788094+0000", 2024-04-20T01:47:55.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033620200000000002 2024-04-20T01:47:55.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.191 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:47:55.191 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788099+0000", 2024-04-20T01:47:55.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2469999999999999e-06 2024-04-20T01:47:55.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.191 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.191 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.792538+0000", 2024-04-20T01:47:55.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0044393949999999996 2024-04-20T01:47:55.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.191 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:55.191 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.792627+0000", 2024-04-20T01:47:55.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8646999999999993e-05 2024-04-20T01:47:55.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.192 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.192 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.793450+0000", 2024-04-20T01:47:55.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00082310900000000004 2024-04-20T01:47:55.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.192 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.192 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.800753+0000", 2024-04-20T01:47:55.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073032540000000003 2024-04-20T01:47:55.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.192 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:47:55.192 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.801274+0000", 2024-04-20T01:47:55.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052095799999999997 2024-04-20T01:47:55.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.192 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.193 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.801325+0000", 2024-04-20T01:47:55.193 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1251000000000001e-05 2024-04-20T01:47:55.193 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.193 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.193 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.193 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:242 3.4s0 3:28069c2b:::smithi16579500-39:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:47:55.193 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787763+0000", 2024-04-20T01:47:55.193 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 16.382428865000001, 2024-04-20T01:47:55.193 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013716078, 2024-04-20T01:47:55.193 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.193 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:55.193 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:55.193 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:55.193 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:55.194 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 242 2024-04-20T01:47:55.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.194 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787763+0000", 2024-04-20T01:47:55.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787763+0000", 2024-04-20T01:47:55.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787762+0000", 2024-04-20T01:47:55.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:55.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787765+0000", 2024-04-20T01:47:55.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.084e-06 2024-04-20T01:47:55.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787771+0000", 2024-04-20T01:47:55.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9020000000000004e-06 2024-04-20T01:47:55.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787781+0000", 2024-04-20T01:47:55.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1630000000000002e-06 2024-04-20T01:47:55.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788118+0000", 2024-04-20T01:47:55.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033771499999999999 2024-04-20T01:47:55.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:47:55.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788123+0000", 2024-04-20T01:47:55.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-04-20T01:47:55.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.793864+0000", 2024-04-20T01:47:55.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0057404309999999998 2024-04-20T01:47:55.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.197 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:55.197 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.793978+0000", 2024-04-20T01:47:55.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000114274 2024-04-20T01:47:55.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.197 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.197 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.794417+0000", 2024-04-20T01:47:55.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00043935000000000003 2024-04-20T01:47:55.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.197 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.197 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.800777+0000", 2024-04-20T01:47:55.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0063594230000000003 2024-04-20T01:47:55.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.198 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:47:55.198 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.801420+0000", 2024-04-20T01:47:55.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00064355900000000004 2024-04-20T01:47:55.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.198 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.198 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.801479+0000", 2024-04-20T01:47:55.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8693000000000002e-05 2024-04-20T01:47:55.198 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.198 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.198 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.199 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.ds5 38/30 ECSubWrite(tid=19, reqid=osd.6.0:18, at_version=0'0, trim_to=0'0, roll_forward_to=38'10))", 2024-04-20T01:47:55.199 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:39.794346+0000", 2024-04-20T01:47:55.199 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 15.37584521, 2024-04-20T01:47:55.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0074716360000000002, 2024-04-20T01:47:55.199 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.199 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.199 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.199 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.794346+0000", 2024-04-20T01:47:55.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.794346+0000", 2024-04-20T01:47:55.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.794346+0000", 2024-04-20T01:47:55.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:55.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.794363+0000", 2024-04-20T01:47:55.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7765e-05 2024-04-20T01:47:55.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.794381+0000", 2024-04-20T01:47:55.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7975999999999999e-05 2024-04-20T01:47:55.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.794392+0000", 2024-04-20T01:47:55.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.008e-05 2024-04-20T01:47:55.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.794448+0000", 2024-04-20T01:47:55.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6681000000000002e-05 2024-04-20T01:47:55.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.794462+0000", 2024-04-20T01:47:55.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3949e-05 2024-04-20T01:47:55.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.202 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.202 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.801780+0000", 2024-04-20T01:47:55.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073180299999999997 2024-04-20T01:47:55.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.202 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.202 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.801818+0000", 2024-04-20T01:47:55.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7920000000000003e-05 2024-04-20T01:47:55.202 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.202 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.202 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.203 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:291 3.4s0 3:27ac9303:::smithi16579500-19:head [read 0~1261568,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:47:55.203 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:43.807349+0000", 2024-04-20T01:47:55.203 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.362842694999999, 2024-04-20T01:47:55.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010722239, 2024-04-20T01:47:55.203 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.203 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:47:55.203 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:55.203 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:55.203 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:55.203 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 291 2024-04-20T01:47:55.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.203 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.203 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.204 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807349+0000", 2024-04-20T01:47:55.204 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.204 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.204 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807349+0000", 2024-04-20T01:47:55.204 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.204 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.204 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807346+0000", 2024-04-20T01:47:55.204 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999976 2024-04-20T01:47:55.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.204 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.204 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807364+0000", 2024-04-20T01:47:55.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7674999999999999e-05 2024-04-20T01:47:55.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.205 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.205 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807417+0000", 2024-04-20T01:47:55.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2698000000000003e-05 2024-04-20T01:47:55.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.205 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.205 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807439+0000", 2024-04-20T01:47:55.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2459e-05 2024-04-20T01:47:55.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.205 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.205 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807524+0000", 2024-04-20T01:47:55.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4544999999999998e-05 2024-04-20T01:47:55.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op must wait for map", 2024-04-20T01:47:55.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807533+0000", 2024-04-20T01:47:55.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8259999999999999e-06 2024-04-20T01:47:55.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.811990+0000", 2024-04-20T01:47:55.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0044573290000000003 2024-04-20T01:47:55.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:55.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.812079+0000", 2024-04-20T01:47:55.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8614000000000002e-05 2024-04-20T01:47:55.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.207 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.207 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.818071+0000", 2024-04-20T01:47:55.207 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0059925940000000004 2024-04-20T01:47:55.207 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.207 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.207 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.207 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:292 3.4s0 3:27ac9303:::smithi16579500-19:head [read 0~1261568,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:47:55.207 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:43.807455+0000", 2024-04-20T01:47:55.207 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.36273624, 2024-04-20T01:47:55.207 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01155932, 2024-04-20T01:47:55.207 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.208 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:47:55.208 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:55.208 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:55.208 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:55.208 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 292 2024-04-20T01:47:55.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.208 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.208 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.208 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807455+0000", 2024-04-20T01:47:55.208 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.208 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.208 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807455+0000", 2024-04-20T01:47:55.208 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807455+0000", 2024-04-20T01:47:55.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:55.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807460+0000", 2024-04-20T01:47:55.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7879999999999997e-06 2024-04-20T01:47:55.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807465+0000", 2024-04-20T01:47:55.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3260000000000003e-06 2024-04-20T01:47:55.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807469+0000", 2024-04-20T01:47:55.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.703e-06 2024-04-20T01:47:55.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807605+0000", 2024-04-20T01:47:55.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013600600000000001 2024-04-20T01:47:55.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:47:55.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807612+0000", 2024-04-20T01:47:55.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3969999999999997e-06 2024-04-20T01:47:55.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.211 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.211 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.812370+0000", 2024-04-20T01:47:55.211 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047582049999999997 2024-04-20T01:47:55.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.211 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:55.211 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.812466+0000", 2024-04-20T01:47:55.211 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5373999999999999e-05 2024-04-20T01:47:55.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.211 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.211 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.819015+0000", 2024-04-20T01:47:55.211 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.006549048 2024-04-20T01:47:55.212 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.212 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.212 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.212 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:293 3.4s0 3:27ac9303:::smithi16579500-19:head [read 0~1261568,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:47:55.212 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:43.807567+0000", 2024-04-20T01:47:55.212 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.362624402, 2024-04-20T01:47:55.212 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012280444999999999, 2024-04-20T01:47:55.212 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.212 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:47:55.212 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:55.212 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:55.213 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:55.213 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 293 2024-04-20T01:47:55.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.213 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.213 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.213 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807567+0000", 2024-04-20T01:47:55.213 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.213 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.213 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807567+0000", 2024-04-20T01:47:55.213 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.213 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.214 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807567+0000", 2024-04-20T01:47:55.214 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:55.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.214 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.214 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807573+0000", 2024-04-20T01:47:55.214 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6050000000000003e-06 2024-04-20T01:47:55.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.214 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.214 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807583+0000", 2024-04-20T01:47:55.214 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0274e-05 2024-04-20T01:47:55.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.214 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.215 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807591+0000", 2024-04-20T01:47:55.215 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.892e-06 2024-04-20T01:47:55.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.215 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.215 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807645+0000", 2024-04-20T01:47:55.215 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3202000000000003e-05 2024-04-20T01:47:55.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.215 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:47:55.215 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807651+0000", 2024-04-20T01:47:55.215 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6699999999999997e-06 2024-04-20T01:47:55.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.215 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.216 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.812731+0000", 2024-04-20T01:47:55.216 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0050797480000000002 2024-04-20T01:47:55.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.216 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:55.216 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.812817+0000", 2024-04-20T01:47:55.216 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6415999999999994e-05 2024-04-20T01:47:55.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.216 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.216 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.819848+0000", 2024-04-20T01:47:55.216 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0070303809999999996 2024-04-20T01:47:55.216 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.216 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.216 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.217 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:317 3.4s0 3:27ac9303:::smithi16579500-19:head [append 0~786432] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:47:55.217 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:43.934117+0000", 2024-04-20T01:47:55.217 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.236074976999999, 2024-04-20T01:47:55.217 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0081074609999999998, 2024-04-20T01:47:55.217 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.217 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:55.217 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:55.217 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:55.217 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:55.217 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 317 2024-04-20T01:47:55.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.217 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.218 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.218 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.934117+0000", 2024-04-20T01:47:55.218 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.218 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.218 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.934117+0000", 2024-04-20T01:47:55.218 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.218 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.218 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.934115+0000", 2024-04-20T01:47:55.218 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:47:55.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.219 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.219 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.934946+0000", 2024-04-20T01:47:55.219 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00083129500000000002 2024-04-20T01:47:55.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.219 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.219 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.934982+0000", 2024-04-20T01:47:55.219 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5636999999999998e-05 2024-04-20T01:47:55.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.219 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.219 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.934995+0000", 2024-04-20T01:47:55.219 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3013000000000001e-05 2024-04-20T01:47:55.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.219 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.220 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.935043+0000", 2024-04-20T01:47:55.220 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7859999999999999e-05 2024-04-20T01:47:55.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.220 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:55.220 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.935132+0000", 2024-04-20T01:47:55.220 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9090000000000003e-05 2024-04-20T01:47:55.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.220 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.220 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.936136+0000", 2024-04-20T01:47:55.220 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0010040820000000001 2024-04-20T01:47:55.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.220 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.221 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.939927+0000", 2024-04-20T01:47:55.221 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0037916130000000001 2024-04-20T01:47:55.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.221 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:47:55.221 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.942132+0000", 2024-04-20T01:47:55.221 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022043219999999999 2024-04-20T01:47:55.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.221 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.221 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.942224+0000", 2024-04-20T01:47:55.221 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2487999999999999e-05 2024-04-20T01:47:55.221 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.221 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.221 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.222 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:318 3.4s0 3:27ac9303:::smithi16579500-19:head [append 0~376832] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:47:55.222 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:43.936253+0000", 2024-04-20T01:47:55.222 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.233938366, 2024-04-20T01:47:55.222 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007015093, 2024-04-20T01:47:55.222 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.222 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:55.222 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:55.222 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:55.222 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:55.222 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 318 2024-04-20T01:47:55.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.222 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.222 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.223 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.936253+0000", 2024-04-20T01:47:55.223 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.223 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.223 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.936253+0000", 2024-04-20T01:47:55.223 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.223 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.223 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.936251+0000", 2024-04-20T01:47:55.223 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:47:55.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.223 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.224 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.936760+0000", 2024-04-20T01:47:55.224 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00050877300000000002 2024-04-20T01:47:55.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.224 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.224 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.936776+0000", 2024-04-20T01:47:55.224 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6283999999999999e-05 2024-04-20T01:47:55.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.224 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.224 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.936787+0000", 2024-04-20T01:47:55.224 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0696000000000001e-05 2024-04-20T01:47:55.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.224 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.225 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.937376+0000", 2024-04-20T01:47:55.225 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00058907199999999995 2024-04-20T01:47:55.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.225 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:55.225 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.937511+0000", 2024-04-20T01:47:55.225 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013469999999999999 2024-04-20T01:47:55.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.225 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.225 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.938330+0000", 2024-04-20T01:47:55.225 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00081859200000000004 2024-04-20T01:47:55.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.225 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.226 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.939929+0000", 2024-04-20T01:47:55.226 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001599318 2024-04-20T01:47:55.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.226 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:47:55.226 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943197+0000", 2024-04-20T01:47:55.226 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0032686270000000001 2024-04-20T01:47:55.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.230 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.230 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943268+0000", 2024-04-20T01:47:55.230 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0951000000000006e-05 2024-04-20T01:47:55.230 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.230 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.230 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.231 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:320 3.4s0 3:27ac9303:::smithi16579500-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:47:55.231 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:43.936823+0000", 2024-04-20T01:47:55.231 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.233369149, 2024-04-20T01:47:55.231 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010320714, 2024-04-20T01:47:55.231 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.231 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:47:55.231 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:55.231 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:55.231 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:55.231 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 320 2024-04-20T01:47:55.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.231 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.231 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.231 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.936823+0000", 2024-04-20T01:47:55.232 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.232 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.232 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.936823+0000", 2024-04-20T01:47:55.232 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.232 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.232 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.936822+0000", 2024-04-20T01:47:55.232 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:55.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.232 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.232 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.936831+0000", 2024-04-20T01:47:55.232 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6929999999999996e-06 2024-04-20T01:47:55.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.233 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.233 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.936839+0000", 2024-04-20T01:47:55.233 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9559999999999997e-06 2024-04-20T01:47:55.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.233 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.233 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.936845+0000", 2024-04-20T01:47:55.233 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.178e-06 2024-04-20T01:47:55.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.233 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.233 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.940007+0000", 2024-04-20T01:47:55.233 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0031617059999999998 2024-04-20T01:47:55.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.234 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-04-20T01:47:55.234 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.940114+0000", 2024-04-20T01:47:55.234 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000107737 2024-04-20T01:47:55.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.234 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.234 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.944146+0000", 2024-04-20T01:47:55.234 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0040320260000000002 2024-04-20T01:47:55.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.234 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:55.234 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.944305+0000", 2024-04-20T01:47:55.234 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000159066 2024-04-20T01:47:55.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.235 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.235 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.947143+0000", 2024-04-20T01:47:55.235 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0028377879999999999 2024-04-20T01:47:55.235 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.235 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.235 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.235 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs1 47/30 ECSubWrite(tid=38, reqid=osd.14.0:38, at_version=47'31, trim_to=0'0, roll_forward_to=46'29))", 2024-04-20T01:47:55.235 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.712012+0000", 2024-04-20T01:47:55.235 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 6.4581799359999996, 2024-04-20T01:47:55.235 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0085004929999999996, 2024-04-20T01:47:55.235 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.235 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.236 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.236 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.236 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.236 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.712012+0000", 2024-04-20T01:47:55.236 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.236 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.236 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.236 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.712012+0000", 2024-04-20T01:47:55.236 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.236 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.236 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.236 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.712011+0000", 2024-04-20T01:47:55.236 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:55.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.237 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.237 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.712039+0000", 2024-04-20T01:47:55.237 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7098999999999999e-05 2024-04-20T01:47:55.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.237 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.237 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.712052+0000", 2024-04-20T01:47:55.237 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3183999999999999e-05 2024-04-20T01:47:55.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.237 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.237 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.712062+0000", 2024-04-20T01:47:55.237 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0507e-05 2024-04-20T01:47:55.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.238 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.238 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.712191+0000", 2024-04-20T01:47:55.238 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000128746 2024-04-20T01:47:55.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.238 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.238 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.712206+0000", 2024-04-20T01:47:55.238 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4959e-05 2024-04-20T01:47:55.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.238 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.238 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720412+0000", 2024-04-20T01:47:55.238 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0082055600000000006 2024-04-20T01:47:55.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.239 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.239 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720512+0000", 2024-04-20T01:47:55.239 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000100728 2024-04-20T01:47:55.239 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.239 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.239 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.239 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs1 47/30 ECSubWrite(tid=39, reqid=osd.14.0:39, at_version=47'33, trim_to=0'0, roll_forward_to=46'29))", 2024-04-20T01:47:55.239 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.713032+0000", 2024-04-20T01:47:55.239 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 6.4571598530000003, 2024-04-20T01:47:55.239 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0075027779999999999, 2024-04-20T01:47:55.239 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.239 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.240 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.240 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.240 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.713032+0000", 2024-04-20T01:47:55.240 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.240 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.240 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.713032+0000", 2024-04-20T01:47:55.240 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.240 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.240 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.713031+0000", 2024-04-20T01:47:55.240 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:55.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.241 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.241 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.713059+0000", 2024-04-20T01:47:55.241 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8090999999999999e-05 2024-04-20T01:47:55.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.241 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.241 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.713072+0000", 2024-04-20T01:47:55.241 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3229e-05 2024-04-20T01:47:55.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.241 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.241 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.713080+0000", 2024-04-20T01:47:55.241 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.233e-06 2024-04-20T01:47:55.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.242 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.713123+0000", 2024-04-20T01:47:55.242 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3544999999999997e-05 2024-04-20T01:47:55.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.242 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.713130+0000", 2024-04-20T01:47:55.242 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7880000000000001e-06 2024-04-20T01:47:55.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.242 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720520+0000", 2024-04-20T01:47:55.242 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.007390087 2024-04-20T01:47:55.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.243 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.243 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720535+0000", 2024-04-20T01:47:55.243 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.467e-05 2024-04-20T01:47:55.243 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.243 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.243 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.243 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:399 3.4s0 3:2d59a0da:::smithi16579500-10:head [append 0~0] snapc d=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-04-20T01:47:55.243 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.711018+0000", 2024-04-20T01:47:55.243 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.4591735999999997, 2024-04-20T01:47:55.243 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014803088000000001, 2024-04-20T01:47:55.243 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.244 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:55.244 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:55.244 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:55.244 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:55.244 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 399 2024-04-20T01:47:55.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.244 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.244 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.244 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711018+0000", 2024-04-20T01:47:55.244 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.244 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.244 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711018+0000", 2024-04-20T01:47:55.245 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.245 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.245 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711017+0000", 2024-04-20T01:47:55.245 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:55.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.245 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.245 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711023+0000", 2024-04-20T01:47:55.245 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7760000000000003e-06 2024-04-20T01:47:55.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.245 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.245 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711036+0000", 2024-04-20T01:47:55.245 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3754000000000001e-05 2024-04-20T01:47:55.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.246 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.246 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711044+0000", 2024-04-20T01:47:55.246 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.435e-06 2024-04-20T01:47:55.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.246 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.246 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711095+0000", 2024-04-20T01:47:55.246 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1038999999999997e-05 2024-04-20T01:47:55.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.246 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:55.246 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711221+0000", 2024-04-20T01:47:55.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000126515 2024-04-20T01:47:55.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711890+0000", 2024-04-20T01:47:55.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00066828000000000005 2024-04-20T01:47:55.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714226+0000", 2024-04-20T01:47:55.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023359040000000002 2024-04-20T01:47:55.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:47:55.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.725772+0000", 2024-04-20T01:47:55.248 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011546529 2024-04-20T01:47:55.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.248 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.248 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.725821+0000", 2024-04-20T01:47:55.248 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9129999999999999e-05 2024-04-20T01:47:55.248 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.248 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.248 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.248 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:400 3.4s0 3:2d59a0da:::smithi16579500-10:head [rollback b] snapc d=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-04-20T01:47:55.248 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.711049+0000", 2024-04-20T01:47:55.248 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.459142977, 2024-04-20T01:47:55.248 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.019555619, 2024-04-20T01:47:55.248 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.249 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:55.249 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:55.249 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:55.249 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:55.249 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 400 2024-04-20T01:47:55.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.249 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.249 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.249 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711049+0000", 2024-04-20T01:47:55.249 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.249 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.249 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711049+0000", 2024-04-20T01:47:55.250 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.250 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.250 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.250 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711048+0000", 2024-04-20T01:47:55.250 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:55.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.250 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.250 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.250 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711050+0000", 2024-04-20T01:47:55.250 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.905e-06 2024-04-20T01:47:55.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.250 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.250 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711054+0000", 2024-04-20T01:47:55.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0280000000000002e-06 2024-04-20T01:47:55.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.251 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711058+0000", 2024-04-20T01:47:55.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5159999999999999e-06 2024-04-20T01:47:55.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.251 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712434+0000", 2024-04-20T01:47:55.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001375937 2024-04-20T01:47:55.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.251 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:55.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712543+0000", 2024-04-20T01:47:55.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010908000000000001 2024-04-20T01:47:55.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713115+0000", 2024-04-20T01:47:55.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00057183500000000003 2024-04-20T01:47:55.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.715444+0000", 2024-04-20T01:47:55.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023292930000000001 2024-04-20T01:47:55.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:47:55.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730509+0000", 2024-04-20T01:47:55.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015064546 2024-04-20T01:47:55.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.253 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730604+0000", 2024-04-20T01:47:55.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5814000000000001e-05 2024-04-20T01:47:55.253 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.253 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.253 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.253 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:401 3.4s0 3:2d59a0da:::smithi16579500-10:head [append 0~0] snapc d=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-04-20T01:47:55.253 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.711061+0000", 2024-04-20T01:47:55.253 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.4591306240000002, 2024-04-20T01:47:55.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.019877855999999999, 2024-04-20T01:47:55.254 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.254 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:55.254 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:47:55.254 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:47:55.254 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:55.254 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 401 2024-04-20T01:47:55.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.254 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711061+0000", 2024-04-20T01:47:55.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711061+0000", 2024-04-20T01:47:55.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711061+0000", 2024-04-20T01:47:55.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:55.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711064+0000", 2024-04-20T01:47:55.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8480000000000001e-06 2024-04-20T01:47:55.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.256 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.256 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711067+0000", 2024-04-20T01:47:55.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5559999999999999e-06 2024-04-20T01:47:55.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.256 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.256 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711070+0000", 2024-04-20T01:47:55.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.044e-06 2024-04-20T01:47:55.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.256 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.256 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713427+0000", 2024-04-20T01:47:55.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002356585 2024-04-20T01:47:55.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:47:55.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713534+0000", 2024-04-20T01:47:55.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010668200000000001 2024-04-20T01:47:55.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713981+0000", 2024-04-20T01:47:55.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00044727999999999999 2024-04-20T01:47:55.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.718977+0000", 2024-04-20T01:47:55.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0049960330000000004 2024-04-20T01:47:55.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:47:55.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730869+0000", 2024-04-20T01:47:55.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011892606 2024-04-20T01:47:55.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730939+0000", 2024-04-20T01:47:55.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9470999999999999e-05 2024-04-20T01:47:55.258 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.258 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.258 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.258 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.258 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.258 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:55.260 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-20T01:47:55.260 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:55.275+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:55.361 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:55.376+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:47:55.410 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:55.410 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:47:55.410 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-04-20T01:47:55.410 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:55.422 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-20T01:47:55.462 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:55.476+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:55.562 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:55.577+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:55.564 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:55.564 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:47:55.564 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-04-20T01:47:55.564 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-04-20T01:47:55.565 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:55.576 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-20T01:47:55.664 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:55.678+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:55.698 INFO:tasks.rados.rados.0.smithi165.stdout:186: done (1 left) 2024-04-20T01:47:55.698 INFO:tasks.rados.rados.0.smithi165.stdout:187: snap_remove snap 9 2024-04-20T01:47:55.723 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:55.723 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-04-20T01:47:55.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-04-20T01:47:55.723 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-04-20T01:47:55.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.723 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.5s4 33/30 ECSubWrite(tid=2, reqid=client.4643.0:11, at_version=0'0, trim_to=0'0, roll_forward_to=32'2))", 2024-04-20T01:47:55.723 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.765877+0000", 2024-04-20T01:47:55.723 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 20.970662927999999, 2024-04-20T01:47:55.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0075032049999999998, 2024-04-20T01:47:55.724 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.724 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.724 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765877+0000", 2024-04-20T01:47:55.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765877+0000", 2024-04-20T01:47:55.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.725 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.725 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765877+0000", 2024-04-20T01:47:55.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:55.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.725 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.725 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765904+0000", 2024-04-20T01:47:55.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7461e-05 2024-04-20T01:47:55.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.725 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.725 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765927+0000", 2024-04-20T01:47:55.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2697999999999999e-05 2024-04-20T01:47:55.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.726 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765939+0000", 2024-04-20T01:47:55.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2142e-05 2024-04-20T01:47:55.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.726 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.768205+0000", 2024-04-20T01:47:55.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022662239999999998 2024-04-20T01:47:55.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.726 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.768221+0000", 2024-04-20T01:47:55.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6161e-05 2024-04-20T01:47:55.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.727 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.773346+0000", 2024-04-20T01:47:55.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051250469999999998 2024-04-20T01:47:55.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.727 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.773380+0000", 2024-04-20T01:47:55.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3760999999999997e-05 2024-04-20T01:47:55.727 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.727 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.727 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.728 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/30 ECSubWrite(tid=2, reqid=client.4643.0:38, at_version=0'0, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:47:55.728 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.777270+0000", 2024-04-20T01:47:55.728 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 20.959270024999999, 2024-04-20T01:47:55.728 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0057051339999999997, 2024-04-20T01:47:55.728 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.728 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.728 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.728 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.728 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777270+0000", 2024-04-20T01:47:55.728 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777270+0000", 2024-04-20T01:47:55.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777268+0000", 2024-04-20T01:47:55.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:47:55.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777287+0000", 2024-04-20T01:47:55.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9128999999999999e-05 2024-04-20T01:47:55.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777309+0000", 2024-04-20T01:47:55.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1475999999999999e-05 2024-04-20T01:47:55.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777323+0000", 2024-04-20T01:47:55.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4076e-05 2024-04-20T01:47:55.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777375+0000", 2024-04-20T01:47:55.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2685999999999997e-05 2024-04-20T01:47:55.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777389+0000", 2024-04-20T01:47:55.731 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3369e-05 2024-04-20T01:47:55.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.731 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.731 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.731 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.782943+0000", 2024-04-20T01:47:55.731 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055546110000000001 2024-04-20T01:47:55.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.731 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.731 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.731 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.782975+0000", 2024-04-20T01:47:55.731 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1569000000000002e-05 2024-04-20T01:47:55.731 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.731 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.731 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.732 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/30 ECSubWrite(tid=4, reqid=client.4643.0:58, at_version=33'3, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:47:55.732 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.780211+0000", 2024-04-20T01:47:55.732 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 20.956329339, 2024-04-20T01:47:55.732 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0098815680000000003, 2024-04-20T01:47:55.732 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.732 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.732 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.732 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.732 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780211+0000", 2024-04-20T01:47:55.732 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.733 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.733 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780211+0000", 2024-04-20T01:47:55.733 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.733 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.733 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.733 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780209+0000", 2024-04-20T01:47:55.733 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:47:55.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.733 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.733 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.733 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780307+0000", 2024-04-20T01:47:55.733 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8575999999999999e-05 2024-04-20T01:47:55.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.733 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.734 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.734 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780328+0000", 2024-04-20T01:47:55.734 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0537999999999999e-05 2024-04-20T01:47:55.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.734 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.734 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.734 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780340+0000", 2024-04-20T01:47:55.734 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1997999999999999e-05 2024-04-20T01:47:55.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.734 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.734 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.734 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780423+0000", 2024-04-20T01:47:55.734 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2812999999999995e-05 2024-04-20T01:47:55.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780434+0000", 2024-04-20T01:47:55.735 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1028e-05 2024-04-20T01:47:55.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790048+0000", 2024-04-20T01:47:55.735 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096147339999999998 2024-04-20T01:47:55.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790092+0000", 2024-04-20T01:47:55.735 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3649999999999997e-05 2024-04-20T01:47:55.735 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.735 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.736 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.736 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/30 ECSubWrite(tid=5, reqid=client.4643.0:59, at_version=33'4, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:47:55.736 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.781201+0000", 2024-04-20T01:47:55.736 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 20.955339101, 2024-04-20T01:47:55.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0089164740000000006, 2024-04-20T01:47:55.736 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.736 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.736 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.736 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.736 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781201+0000", 2024-04-20T01:47:55.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.737 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.737 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.737 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781201+0000", 2024-04-20T01:47:55.737 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.737 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.737 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.737 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781200+0000", 2024-04-20T01:47:55.737 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:55.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.737 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.737 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.737 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781219+0000", 2024-04-20T01:47:55.737 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8748000000000001e-05 2024-04-20T01:47:55.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.738 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.738 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.738 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781236+0000", 2024-04-20T01:47:55.738 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7251000000000001e-05 2024-04-20T01:47:55.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.738 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.738 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.738 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781246+0000", 2024-04-20T01:47:55.738 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0026e-05 2024-04-20T01:47:55.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.738 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.738 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.738 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.782742+0000", 2024-04-20T01:47:55.738 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001496079 2024-04-20T01:47:55.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.739 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.739 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.739 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.782753+0000", 2024-04-20T01:47:55.739 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0767999999999999e-05 2024-04-20T01:47:55.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.739 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.739 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.739 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790103+0000", 2024-04-20T01:47:55.739 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073499259999999997 2024-04-20T01:47:55.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.739 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.739 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.739 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790117+0000", 2024-04-20T01:47:55.739 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4729000000000001e-05 2024-04-20T01:47:55.739 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.740 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.740 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.740 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.5s4 33/30 ECSubWrite(tid=5, reqid=client.4643.0:65, at_version=33'4, trim_to=0'0, roll_forward_to=32'2))", 2024-04-20T01:47:55.740 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.783666+0000", 2024-04-20T01:47:55.740 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 20.952874080000001, 2024-04-20T01:47:55.740 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0063804899999999999, 2024-04-20T01:47:55.740 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.740 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.740 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.740 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.740 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783666+0000", 2024-04-20T01:47:55.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783666+0000", 2024-04-20T01:47:55.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783665+0000", 2024-04-20T01:47:55.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:55.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783685+0000", 2024-04-20T01:47:55.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9735e-05 2024-04-20T01:47:55.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783701+0000", 2024-04-20T01:47:55.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6175e-05 2024-04-20T01:47:55.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783710+0000", 2024-04-20T01:47:55.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3649999999999993e-06 2024-04-20T01:47:55.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783800+0000", 2024-04-20T01:47:55.743 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0061000000000004e-05 2024-04-20T01:47:55.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.743 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.743 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783809+0000", 2024-04-20T01:47:55.743 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0750000000000004e-06 2024-04-20T01:47:55.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.743 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.743 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790003+0000", 2024-04-20T01:47:55.743 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0061941590000000003 2024-04-20T01:47:55.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.743 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.743 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790046+0000", 2024-04-20T01:47:55.744 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2831999999999999e-05 2024-04-20T01:47:55.744 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.744 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.744 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.744 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.744 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/30 ECSubWrite(tid=25, reqid=client.4643.0:101, at_version=33'10, trim_to=0'0, roll_forward_to=33'8))", 2024-04-20T01:47:55.744 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.819138+0000", 2024-04-20T01:47:55.744 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 20.917402088999999, 2024-04-20T01:47:55.744 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004933425, 2024-04-20T01:47:55.744 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.744 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.744 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.744 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.744 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.819138+0000", 2024-04-20T01:47:55.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.819138+0000", 2024-04-20T01:47:55.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.819137+0000", 2024-04-20T01:47:55.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:55.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.746 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.746 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.819158+0000", 2024-04-20T01:47:55.746 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.048e-05 2024-04-20T01:47:55.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.746 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.746 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.819169+0000", 2024-04-20T01:47:55.746 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1922999999999999e-05 2024-04-20T01:47:55.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.746 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.746 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.819180+0000", 2024-04-20T01:47:55.746 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0995000000000001e-05 2024-04-20T01:47:55.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.819244+0000", 2024-04-20T01:47:55.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3212000000000004e-05 2024-04-20T01:47:55.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.819254+0000", 2024-04-20T01:47:55.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0198e-05 2024-04-20T01:47:55.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.824053+0000", 2024-04-20T01:47:55.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047987969999999996 2024-04-20T01:47:55.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.748 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.748 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.824071+0000", 2024-04-20T01:47:55.748 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8576999999999998e-05 2024-04-20T01:47:55.748 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.748 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.748 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.748 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs5 33/30 ECSubWrite(tid=0, reqid=client.4643.0:106, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:55.748 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.825844+0000", 2024-04-20T01:47:55.748 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 20.910696118000001, 2024-04-20T01:47:55.748 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0062702069999999999, 2024-04-20T01:47:55.748 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.748 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.749 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.749 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.749 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.749 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.825844+0000", 2024-04-20T01:47:55.749 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.749 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.749 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.749 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.825844+0000", 2024-04-20T01:47:55.749 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.749 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.749 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.749 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.825843+0000", 2024-04-20T01:47:55.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:55.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826258+0000", 2024-04-20T01:47:55.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00041487099999999998 2024-04-20T01:47:55.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826269+0000", 2024-04-20T01:47:55.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0264e-05 2024-04-20T01:47:55.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826275+0000", 2024-04-20T01:47:55.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3010000000000003e-06 2024-04-20T01:47:55.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826329+0000", 2024-04-20T01:47:55.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4036999999999998e-05 2024-04-20T01:47:55.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826337+0000", 2024-04-20T01:47:55.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7470000000000005e-06 2024-04-20T01:47:55.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.752 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.832089+0000", 2024-04-20T01:47:55.752 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0057520339999999996 2024-04-20T01:47:55.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.752 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.752 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.832114+0000", 2024-04-20T01:47:55.752 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5267999999999999e-05 2024-04-20T01:47:55.752 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.752 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.752 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.752 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs5 33/30 ECSubWrite(tid=1, reqid=client.4643.0:107, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:55.752 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.826758+0000", 2024-04-20T01:47:55.753 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 20.909781918, 2024-04-20T01:47:55.753 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0053877639999999997, 2024-04-20T01:47:55.753 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.753 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.753 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.753 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.753 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.753 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826758+0000", 2024-04-20T01:47:55.753 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.753 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.753 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.753 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826758+0000", 2024-04-20T01:47:55.754 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.754 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.754 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826757+0000", 2024-04-20T01:47:55.754 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:55.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.754 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.754 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826771+0000", 2024-04-20T01:47:55.754 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3735e-05 2024-04-20T01:47:55.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826782+0000", 2024-04-20T01:47:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1071e-05 2024-04-20T01:47:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826792+0000", 2024-04-20T01:47:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4970000000000007e-06 2024-04-20T01:47:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826933+0000", 2024-04-20T01:47:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014126699999999999 2024-04-20T01:47:55.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.756 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.756 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826942+0000", 2024-04-20T01:47:55.756 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5229999999999995e-06 2024-04-20T01:47:55.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.756 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.756 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.832129+0000", 2024-04-20T01:47:55.756 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051871479999999999 2024-04-20T01:47:55.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.756 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.756 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.832146+0000", 2024-04-20T01:47:55.756 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7081000000000001e-05 2024-04-20T01:47:55.756 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.757 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.757 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.757 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/30 ECSubWrite(tid=32, reqid=client.4643.0:134, at_version=33'12, trim_to=0'0, roll_forward_to=33'10))", 2024-04-20T01:47:55.757 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.862933+0000", 2024-04-20T01:47:55.757 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 20.873606645999999, 2024-04-20T01:47:55.757 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0049452619999999997, 2024-04-20T01:47:55.757 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.757 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.757 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.757 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.757 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.862933+0000", 2024-04-20T01:47:55.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.862933+0000", 2024-04-20T01:47:55.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.862932+0000", 2024-04-20T01:47:55.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:55.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.862957+0000", 2024-04-20T01:47:55.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4629000000000001e-05 2024-04-20T01:47:55.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.759 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.862975+0000", 2024-04-20T01:47:55.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7618999999999999e-05 2024-04-20T01:47:55.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.759 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.862990+0000", 2024-04-20T01:47:55.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5551999999999999e-05 2024-04-20T01:47:55.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.759 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.760 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.863224+0000", 2024-04-20T01:47:55.760 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00023420400000000001 2024-04-20T01:47:55.760 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.760 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.760 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.760 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.863234+0000", 2024-04-20T01:47:55.760 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6700000000000006e-06 2024-04-20T01:47:55.760 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.760 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.760 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.760 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.867852+0000", 2024-04-20T01:47:55.760 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0046182419999999998 2024-04-20T01:47:55.760 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.760 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.761 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.761 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.867879+0000", 2024-04-20T01:47:55.761 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6241000000000002e-05 2024-04-20T01:47:55.761 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.761 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.761 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.761 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.761 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.761 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.es6 33/30 ECSubWrite(tid=2, reqid=client.4643.0:103, at_version=33'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:55.761 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.965527+0000", 2024-04-20T01:47:55.761 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 20.771012613, 2024-04-20T01:47:55.761 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048030770000000002, 2024-04-20T01:47:55.761 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.761 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.761 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.762 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.762 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.762 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.965527+0000", 2024-04-20T01:47:55.762 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.762 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.762 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.762 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.965527+0000", 2024-04-20T01:47:55.762 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.762 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.762 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.762 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.965527+0000", 2024-04-20T01:47:55.762 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:55.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.763 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.763 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.965977+0000", 2024-04-20T01:47:55.763 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00044985499999999998 2024-04-20T01:47:55.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.763 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.763 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.965991+0000", 2024-04-20T01:47:55.763 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4572999999999999e-05 2024-04-20T01:47:55.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.763 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.763 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966001+0000", 2024-04-20T01:47:55.763 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9110000000000005e-06 2024-04-20T01:47:55.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.764 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.764 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966059+0000", 2024-04-20T01:47:55.764 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8248999999999997e-05 2024-04-20T01:47:55.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.764 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.764 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.764 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966075+0000", 2024-04-20T01:47:55.764 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5347999999999999e-05 2024-04-20T01:47:55.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.764 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.764 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.764 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.970302+0000", 2024-04-20T01:47:55.764 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042272509999999996 2024-04-20T01:47:55.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.764 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.764 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.765 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.970330+0000", 2024-04-20T01:47:55.765 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8396000000000001e-05 2024-04-20T01:47:55.765 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.765 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.765 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.765 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.es6 33/30 ECSubWrite(tid=3, reqid=client.4643.0:104, at_version=33'4, trim_to=0'0, roll_forward_to=33'1))", 2024-04-20T01:47:55.765 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.966859+0000", 2024-04-20T01:47:55.765 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 20.769680421, 2024-04-20T01:47:55.765 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0063969730000000002, 2024-04-20T01:47:55.765 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.765 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.765 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.766 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.766 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966859+0000", 2024-04-20T01:47:55.766 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.766 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.766 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.766 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.766 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966859+0000", 2024-04-20T01:47:55.766 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.766 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.766 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.766 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.766 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966859+0000", 2024-04-20T01:47:55.766 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:55.766 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.766 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.766 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.767 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966876+0000", 2024-04-20T01:47:55.767 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7116999999999999e-05 2024-04-20T01:47:55.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.767 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.767 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.767 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966887+0000", 2024-04-20T01:47:55.767 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1508999999999999e-05 2024-04-20T01:47:55.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.767 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.767 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.767 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966894+0000", 2024-04-20T01:47:55.767 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4219999999999997e-06 2024-04-20T01:47:55.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.767 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.767 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.767 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.967342+0000", 2024-04-20T01:47:55.767 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00044805799999999998 2024-04-20T01:47:55.768 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.768 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.768 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.768 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.967352+0000", 2024-04-20T01:47:55.768 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0497e-05 2024-04-20T01:47:55.768 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.768 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.768 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.768 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.973223+0000", 2024-04-20T01:47:55.768 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0058706749999999997 2024-04-20T01:47:55.768 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.768 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.768 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.768 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.973256+0000", 2024-04-20T01:47:55.768 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3544999999999998e-05 2024-04-20T01:47:55.768 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.769 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.769 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.769 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.769 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.769 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 37/30 ECSubWrite(tid=64, reqid=client.4643.0:225, at_version=37'16, trim_to=0'0, roll_forward_to=36'14))", 2024-04-20T01:47:55.769 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.788511+0000", 2024-04-20T01:47:55.769 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 16.948029196, 2024-04-20T01:47:55.769 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0049701729999999996, 2024-04-20T01:47:55.769 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.769 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.769 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.769 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.769 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.769 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788511+0000", 2024-04-20T01:47:55.770 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.770 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.770 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.770 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.770 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788511+0000", 2024-04-20T01:47:55.770 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.770 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.770 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.770 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.770 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788510+0000", 2024-04-20T01:47:55.770 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:55.770 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.770 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.770 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.771 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788672+0000", 2024-04-20T01:47:55.771 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016171800000000001 2024-04-20T01:47:55.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.771 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.771 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.771 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788693+0000", 2024-04-20T01:47:55.771 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1066000000000001e-05 2024-04-20T01:47:55.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.771 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.771 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.771 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788705+0000", 2024-04-20T01:47:55.771 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1627000000000001e-05 2024-04-20T01:47:55.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.772 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.772 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.772 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.789578+0000", 2024-04-20T01:47:55.772 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00087352100000000004 2024-04-20T01:47:55.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.772 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.772 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.772 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.789588+0000", 2024-04-20T01:47:55.772 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3700000000000001e-06 2024-04-20T01:47:55.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.772 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.772 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.772 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.793450+0000", 2024-04-20T01:47:55.772 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0038620540000000002 2024-04-20T01:47:55.773 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.773 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.773 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.773 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.793481+0000", 2024-04-20T01:47:55.773 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1102999999999997e-05 2024-04-20T01:47:55.773 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.773 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.773 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.773 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.773 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.773 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 38/30 ECSubWrite(tid=73, reqid=osd.4.0:73, at_version=38'22, trim_to=0'0, roll_forward_to=37'18))", 2024-04-20T01:47:55.773 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:39.792927+0000", 2024-04-20T01:47:55.773 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 15.943612438000001, 2024-04-20T01:47:55.773 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.005772124, 2024-04-20T01:47:55.773 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.774 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.774 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.774 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.774 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.774 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.792927+0000", 2024-04-20T01:47:55.774 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.774 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.774 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.774 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.792927+0000", 2024-04-20T01:47:55.774 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.774 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.774 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.774 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.792926+0000", 2024-04-20T01:47:55.775 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:55.775 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.775 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.775 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.775 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.792962+0000", 2024-04-20T01:47:55.775 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6053999999999997e-05 2024-04-20T01:47:55.775 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.775 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.775 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.775 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.792982+0000", 2024-04-20T01:47:55.775 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0200999999999999e-05 2024-04-20T01:47:55.775 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.775 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.775 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.776 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.792998+0000", 2024-04-20T01:47:55.776 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6044000000000001e-05 2024-04-20T01:47:55.776 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.776 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.776 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.776 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.793081+0000", 2024-04-20T01:47:55.776 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3146999999999995e-05 2024-04-20T01:47:55.776 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.776 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.776 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.776 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.793094+0000", 2024-04-20T01:47:55.776 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2741999999999999e-05 2024-04-20T01:47:55.776 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.776 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.776 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.776 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.798662+0000", 2024-04-20T01:47:55.777 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055676750000000002 2024-04-20T01:47:55.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.777 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.777 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.777 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.798700+0000", 2024-04-20T01:47:55.777 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7832e-05 2024-04-20T01:47:55.777 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.777 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.777 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.777 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.777 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.777 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 38/30 ECSubWrite(tid=74, reqid=osd.4.0:73, at_version=0'0, trim_to=0'0, roll_forward_to=38'22))", 2024-04-20T01:47:55.777 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:39.799501+0000", 2024-04-20T01:47:55.777 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 15.937038469000001, 2024-04-20T01:47:55.777 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0056449669999999999, 2024-04-20T01:47:55.777 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.777 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.778 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.778 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.778 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.799501+0000", 2024-04-20T01:47:55.778 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.778 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.778 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.799501+0000", 2024-04-20T01:47:55.778 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.778 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.778 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.778 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.799500+0000", 2024-04-20T01:47:55.779 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:55.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.779 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.779 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.799514+0000", 2024-04-20T01:47:55.779 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3157e-05 2024-04-20T01:47:55.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.779 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.779 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.799533+0000", 2024-04-20T01:47:55.779 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9879000000000001e-05 2024-04-20T01:47:55.779 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.779 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.779 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.780 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.799544+0000", 2024-04-20T01:47:55.780 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0051000000000001e-05 2024-04-20T01:47:55.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.780 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.780 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.799598+0000", 2024-04-20T01:47:55.780 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4001e-05 2024-04-20T01:47:55.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.780 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.780 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.799608+0000", 2024-04-20T01:47:55.780 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0234999999999999e-05 2024-04-20T01:47:55.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.780 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.781 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.781 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.805112+0000", 2024-04-20T01:47:55.781 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055047100000000003 2024-04-20T01:47:55.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.781 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.781 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.805146+0000", 2024-04-20T01:47:55.781 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3939000000000002e-05 2024-04-20T01:47:55.781 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.781 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.781 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.781 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.781 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.es6 38/30 ECSubWrite(tid=19, reqid=client.4643.0:245, at_version=38'12, trim_to=0'0, roll_forward_to=34'11))", 2024-04-20T01:47:55.781 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:39.805812+0000", 2024-04-20T01:47:55.782 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 15.930727726000001, 2024-04-20T01:47:55.782 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047353940000000004, 2024-04-20T01:47:55.782 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.782 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.782 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.782 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.782 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.805812+0000", 2024-04-20T01:47:55.782 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.782 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.782 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.782 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.805812+0000", 2024-04-20T01:47:55.782 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.783 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.783 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.805812+0000", 2024-04-20T01:47:55.783 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:55.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.783 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.783 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.806321+0000", 2024-04-20T01:47:55.783 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00050948900000000001 2024-04-20T01:47:55.783 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.783 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.783 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.783 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.806339+0000", 2024-04-20T01:47:55.783 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7299000000000002e-05 2024-04-20T01:47:55.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.784 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.784 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.784 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.806352+0000", 2024-04-20T01:47:55.784 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3516000000000001e-05 2024-04-20T01:47:55.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.784 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.784 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.784 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.806426+0000", 2024-04-20T01:47:55.784 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4068999999999998e-05 2024-04-20T01:47:55.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.784 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.784 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.784 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.806441+0000", 2024-04-20T01:47:55.784 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5062000000000001e-05 2024-04-20T01:47:55.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.785 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.785 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.785 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.810512+0000", 2024-04-20T01:47:55.785 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004070578 2024-04-20T01:47:55.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.785 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.785 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.785 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.810548+0000", 2024-04-20T01:47:55.785 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5644e-05 2024-04-20T01:47:55.785 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.785 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.785 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.785 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.786 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 42/30 ECSubWrite(tid=98, reqid=client.4643.0:321, at_version=42'36, trim_to=0'0, roll_forward_to=42'35))", 2024-04-20T01:47:55.786 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:43.943374+0000", 2024-04-20T01:47:55.786 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.79316567, 2024-04-20T01:47:55.786 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0059029729999999997, 2024-04-20T01:47:55.786 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.786 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.786 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.786 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.786 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.786 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943374+0000", 2024-04-20T01:47:55.786 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.786 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.786 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.787 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943374+0000", 2024-04-20T01:47:55.787 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.787 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.787 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.787 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943372+0000", 2024-04-20T01:47:55.787 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:47:55.787 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.787 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.787 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943540+0000", 2024-04-20T01:47:55.787 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016738300000000001 2024-04-20T01:47:55.787 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.787 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.787 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.788 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943564+0000", 2024-04-20T01:47:55.788 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4371000000000001e-05 2024-04-20T01:47:55.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.788 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.788 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943577+0000", 2024-04-20T01:47:55.788 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3254e-05 2024-04-20T01:47:55.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.788 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.788 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943661+0000", 2024-04-20T01:47:55.788 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3727000000000007e-05 2024-04-20T01:47:55.788 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.788 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.788 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.788 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943674+0000", 2024-04-20T01:47:55.789 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3097e-05 2024-04-20T01:47:55.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.789 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.789 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.789 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.949246+0000", 2024-04-20T01:47:55.789 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055718210000000002 2024-04-20T01:47:55.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.789 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.789 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.789 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.949277+0000", 2024-04-20T01:47:55.789 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1226999999999998e-05 2024-04-20T01:47:55.789 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.789 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.789 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.790 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.790 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 42/30 ECSubWrite(tid=99, reqid=client.4643.0:322, at_version=42'37, trim_to=0'0, roll_forward_to=42'35))", 2024-04-20T01:47:55.790 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:43.945628+0000", 2024-04-20T01:47:55.790 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.790911852000001, 2024-04-20T01:47:55.790 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.005004133, 2024-04-20T01:47:55.790 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.790 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.790 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.790 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.790 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.790 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.945628+0000", 2024-04-20T01:47:55.790 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.790 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.790 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.790 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.791 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.945628+0000", 2024-04-20T01:47:55.791 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.791 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.791 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.791 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.945627+0000", 2024-04-20T01:47:55.791 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:55.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.791 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.791 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.791 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.945749+0000", 2024-04-20T01:47:55.791 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012226500000000001 2024-04-20T01:47:55.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.791 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.791 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.792 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.945764+0000", 2024-04-20T01:47:55.792 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4566000000000001e-05 2024-04-20T01:47:55.792 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.792 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.792 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.945775+0000", 2024-04-20T01:47:55.792 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0885e-05 2024-04-20T01:47:55.792 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.792 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.792 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.946099+0000", 2024-04-20T01:47:55.792 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00032390799999999998 2024-04-20T01:47:55.792 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.792 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.792 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.793 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.946113+0000", 2024-04-20T01:47:55.793 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4462e-05 2024-04-20T01:47:55.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.793 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.793 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.793 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.950599+0000", 2024-04-20T01:47:55.793 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0044862690000000002 2024-04-20T01:47:55.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.793 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.793 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.793 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.950632+0000", 2024-04-20T01:47:55.793 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2886999999999999e-05 2024-04-20T01:47:55.793 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.793 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.793 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.794 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.794 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 47/30 ECSubWrite(tid=110, reqid=osd.4.0:110, at_version=47'47, trim_to=0'0, roll_forward_to=44'43))", 2024-04-20T01:47:55.794 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.707966+0000", 2024-04-20T01:47:55.794 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 7.0285738630000001, 2024-04-20T01:47:55.794 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051086999999999999, 2024-04-20T01:47:55.794 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.794 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.794 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.794 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.794 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.794 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.707966+0000", 2024-04-20T01:47:55.794 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.794 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.795 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.795 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.707966+0000", 2024-04-20T01:47:55.795 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.795 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.795 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.707965+0000", 2024-04-20T01:47:55.795 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:55.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.795 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.795 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.707993+0000", 2024-04-20T01:47:55.795 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7506999999999999e-05 2024-04-20T01:47:55.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.795 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.796 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.796 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.708011+0000", 2024-04-20T01:47:55.796 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8312999999999999e-05 2024-04-20T01:47:55.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.796 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.796 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.708022+0000", 2024-04-20T01:47:55.796 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1005999999999999e-05 2024-04-20T01:47:55.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.796 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.796 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.796 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.708110+0000", 2024-04-20T01:47:55.796 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8467999999999994e-05 2024-04-20T01:47:55.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.797 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.797 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.797 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.708124+0000", 2024-04-20T01:47:55.797 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3750999999999999e-05 2024-04-20T01:47:55.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.797 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.797 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.797 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.713042+0000", 2024-04-20T01:47:55.797 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004918192 2024-04-20T01:47:55.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.797 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.797 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.797 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.713075+0000", 2024-04-20T01:47:55.797 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2422999999999998e-05 2024-04-20T01:47:55.798 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.798 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.798 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.798 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.798 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.798 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs5 47/30 ECSubWrite(tid=39, reqid=osd.14.0:39, at_version=47'33, trim_to=0'0, roll_forward_to=46'29))", 2024-04-20T01:47:55.798 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.713027+0000", 2024-04-20T01:47:55.798 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 7.0235125729999996, 2024-04-20T01:47:55.798 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0050638180000000003, 2024-04-20T01:47:55.798 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.798 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.798 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.798 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.798 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.799 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.713027+0000", 2024-04-20T01:47:55.799 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.799 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.799 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.799 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.713027+0000", 2024-04-20T01:47:55.799 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.799 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.799 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.799 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.713026+0000", 2024-04-20T01:47:55.799 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:55.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.799 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.799 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.800 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.713054+0000", 2024-04-20T01:47:55.800 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7362e-05 2024-04-20T01:47:55.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.800 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.800 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.800 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.713068+0000", 2024-04-20T01:47:55.800 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4239e-05 2024-04-20T01:47:55.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.800 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.800 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.800 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.713080+0000", 2024-04-20T01:47:55.800 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2106000000000001e-05 2024-04-20T01:47:55.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.800 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.800 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.801 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.713159+0000", 2024-04-20T01:47:55.801 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9028000000000003e-05 2024-04-20T01:47:55.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.801 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.801 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.713171+0000", 2024-04-20T01:47:55.801 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2002e-05 2024-04-20T01:47:55.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.801 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.801 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.718051+0000", 2024-04-20T01:47:55.801 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048800110000000001 2024-04-20T01:47:55.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.801 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.801 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.802 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.718091+0000", 2024-04-20T01:47:55.802 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9959999999999997e-05 2024-04-20T01:47:55.802 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.802 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.802 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.802 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.802 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.802 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 47/30 ECSubWrite(tid=114, reqid=client.4643.0:392, at_version=47'49, trim_to=0'0, roll_forward_to=47'47))", 2024-04-20T01:47:55.802 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.746767+0000", 2024-04-20T01:47:55.802 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 6.9897729169999998, 2024-04-20T01:47:55.802 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0061268629999999998, 2024-04-20T01:47:55.802 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:47:55.802 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:47:55.803 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:47:55.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.803 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:47:55.803 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746767+0000", 2024-04-20T01:47:55.803 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.803 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:47:55.803 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746767+0000", 2024-04-20T01:47:55.803 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:47:55.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.803 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.803 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:47:55.803 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746766+0000", 2024-04-20T01:47:55.803 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:47:55.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.804 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.804 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:47:55.804 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746823+0000", 2024-04-20T01:47:55.804 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6743999999999997e-05 2024-04-20T01:47:55.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.804 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.804 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:47:55.804 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746840+0000", 2024-04-20T01:47:55.804 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7300999999999999e-05 2024-04-20T01:47:55.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.804 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.804 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:47:55.804 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746850+0000", 2024-04-20T01:47:55.804 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0356999999999999e-05 2024-04-20T01:47:55.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.805 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.805 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:47:55.805 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746919+0000", 2024-04-20T01:47:55.805 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8180000000000001e-05 2024-04-20T01:47:55.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.805 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.805 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:47:55.805 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746927+0000", 2024-04-20T01:47:55.805 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3659999999999992e-06 2024-04-20T01:47:55.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.805 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.805 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:47:55.805 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.752855+0000", 2024-04-20T01:47:55.805 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0059277779999999999 2024-04-20T01:47:55.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:55.806 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:55.806 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:47:55.806 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.752894+0000", 2024-04-20T01:47:55.806 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9057999999999998e-05 2024-04-20T01:47:55.806 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.806 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.806 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.806 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:55.806 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:55.806 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:55.806 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-20T01:47:55.808 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:55.822+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:55.809 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-20T01:47:55.909 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:55.924+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:55.951 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:55.951 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:47:55.951 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 0 2024-04-20T01:47:55.951 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:55.962 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-20T01:47:56.010 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:47:56.025+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:47:56.090 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:56.090 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:47:56.091 INFO:teuthology.orchestra.run.smithi121.stdout: "complaint_time": 30, 2024-04-20T01:47:56.091 INFO:teuthology.orchestra.run.smithi121.stdout: "num_blocked_ops": 0 2024-04-20T01:47:56.091 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:56.100 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-20T01:47:56.111 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:56.126+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:47:56.211 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:47:56.226+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:47:56.229 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:56.229 INFO:teuthology.orchestra.run.smithi121.stdout: "size": 20, 2024-04-20T01:47:56.229 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 600, 2024-04-20T01:47:56.229 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-04-20T01:47:56.230 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.230 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.9s1 47/30 ECSubWrite(tid=13, reqid=osd.7.0:13, at_version=47'8, trim_to=0'0, roll_forward_to=42'6))", 2024-04-20T01:47:56.230 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.708967+0000", 2024-04-20T01:47:56.230 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 7.5339576910000003, 2024-04-20T01:47:56.230 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0053408939999999997, 2024-04-20T01:47:56.230 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:56.230 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:56.230 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:56.230 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.230 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:56.230 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.708967+0000", 2024-04-20T01:47:56.230 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.230 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.231 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.231 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:56.231 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.708967+0000", 2024-04-20T01:47:56.231 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.231 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.231 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.231 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:56.231 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.708966+0000", 2024-04-20T01:47:56.231 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:56.231 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.231 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.231 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:56.231 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.708986+0000", 2024-04-20T01:47:56.231 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9799000000000001e-05 2024-04-20T01:47:56.232 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.232 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.232 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:56.232 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.708996+0000", 2024-04-20T01:47:56.232 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.9939999999999995e-06 2024-04-20T01:47:56.232 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.232 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.232 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:56.232 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.709004+0000", 2024-04-20T01:47:56.232 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.2390000000000002e-06 2024-04-20T01:47:56.232 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.232 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.232 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:56.232 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.709244+0000", 2024-04-20T01:47:56.232 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000240113 2024-04-20T01:47:56.233 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.233 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.233 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:56.233 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.709250+0000", 2024-04-20T01:47:56.233 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.4739999999999996e-06 2024-04-20T01:47:56.233 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.233 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.233 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:56.233 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.714281+0000", 2024-04-20T01:47:56.233 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0050313019999999996 2024-04-20T01:47:56.233 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.233 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.233 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:56.233 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.714308+0000", 2024-04-20T01:47:56.234 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6342e-05 2024-04-20T01:47:56.234 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.234 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.234 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.234 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.234 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.234 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.as2 47/30 ECSubWrite(tid=32, reqid=osd.6.0:31, at_version=0'0, trim_to=0'0, roll_forward_to=47'12))", 2024-04-20T01:47:56.234 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.722942+0000", 2024-04-20T01:47:56.234 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 7.5199823639999996, 2024-04-20T01:47:56.234 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0050181330000000001, 2024-04-20T01:47:56.234 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:56.234 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:56.234 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:56.234 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.235 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:56.235 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.722942+0000", 2024-04-20T01:47:56.235 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.235 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.235 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.235 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:56.235 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.722942+0000", 2024-04-20T01:47:56.235 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.235 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.235 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.235 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:56.235 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.722941+0000", 2024-04-20T01:47:56.235 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:56.235 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.235 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.235 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:56.236 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.722952+0000", 2024-04-20T01:47:56.236 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0556e-05 2024-04-20T01:47:56.236 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.236 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.236 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:56.236 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.722965+0000", 2024-04-20T01:47:56.236 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3369e-05 2024-04-20T01:47:56.236 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.236 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.236 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:56.236 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.722974+0000", 2024-04-20T01:47:56.236 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.6740000000000003e-06 2024-04-20T01:47:56.236 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.237 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.237 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:56.237 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.723093+0000", 2024-04-20T01:47:56.237 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00011913000000000001 2024-04-20T01:47:56.237 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.237 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.237 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:56.237 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.723103+0000", 2024-04-20T01:47:56.237 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.8770000000000005e-06 2024-04-20T01:47:56.237 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.237 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.237 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:56.237 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.727939+0000", 2024-04-20T01:47:56.237 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0048359450000000003 2024-04-20T01:47:56.237 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.238 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.238 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:56.238 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.727960+0000", 2024-04-20T01:47:56.238 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1390999999999999e-05 2024-04-20T01:47:56.238 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.238 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.238 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.238 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.238 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.238 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 47/30 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=47'35, trim_to=0'0, roll_forward_to=44'31))", 2024-04-20T01:47:56.238 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.723003+0000", 2024-04-20T01:47:56.238 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 7.5199213289999998, 2024-04-20T01:47:56.238 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0049761140000000002, 2024-04-20T01:47:56.238 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:56.239 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:56.239 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:56.239 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.239 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:56.239 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.723003+0000", 2024-04-20T01:47:56.239 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.239 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.239 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.239 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:56.239 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.723003+0000", 2024-04-20T01:47:56.239 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.239 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.239 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.239 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:56.239 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.723003+0000", 2024-04-20T01:47:56.240 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:56.240 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.240 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.240 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:56.240 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.723021+0000", 2024-04-20T01:47:56.240 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8777e-05 2024-04-20T01:47:56.240 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.240 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.240 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:56.240 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.723032+0000", 2024-04-20T01:47:56.240 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0139000000000001e-05 2024-04-20T01:47:56.240 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.240 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.240 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:56.241 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.723038+0000", 2024-04-20T01:47:56.241 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.674e-06 2024-04-20T01:47:56.241 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.241 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.241 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:56.241 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.723269+0000", 2024-04-20T01:47:56.241 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00023070600000000001 2024-04-20T01:47:56.241 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.241 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.241 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:56.241 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.723278+0000", 2024-04-20T01:47:56.241 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.867e-06 2024-04-20T01:47:56.241 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.241 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.241 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:56.241 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.727963+0000", 2024-04-20T01:47:56.242 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0046851009999999997 2024-04-20T01:47:56.242 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.242 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.242 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:56.242 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.727979+0000", 2024-04-20T01:47:56.242 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6243999999999999e-05 2024-04-20T01:47:56.242 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.242 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.242 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.242 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.242 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.242 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 47/30 ECSubWrite(tid=246, reqid=client.4643.0:388, at_version=47'38, trim_to=0'0, roll_forward_to=47'35))", 2024-04-20T01:47:56.242 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.745219+0000", 2024-04-20T01:47:56.242 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 7.4977049610000002, 2024-04-20T01:47:56.243 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0073977050000000001, 2024-04-20T01:47:56.243 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:56.243 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:56.243 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:56.243 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.243 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:56.243 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745219+0000", 2024-04-20T01:47:56.243 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.243 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.243 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.243 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:56.243 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745219+0000", 2024-04-20T01:47:56.243 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.243 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.244 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.244 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:56.244 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745218+0000", 2024-04-20T01:47:56.244 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:56.244 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.244 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.244 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:56.244 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745266+0000", 2024-04-20T01:47:56.244 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.7012999999999999e-05 2024-04-20T01:47:56.244 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.244 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.244 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:56.244 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745277+0000", 2024-04-20T01:47:56.244 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1348999999999999e-05 2024-04-20T01:47:56.244 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.245 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.245 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:56.245 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745284+0000", 2024-04-20T01:47:56.245 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0060000000000003e-06 2024-04-20T01:47:56.245 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.245 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.245 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:56.245 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745340+0000", 2024-04-20T01:47:56.245 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6023000000000002e-05 2024-04-20T01:47:56.245 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.245 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.245 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:56.245 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745361+0000", 2024-04-20T01:47:56.245 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1356999999999999e-05 2024-04-20T01:47:56.245 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.246 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.246 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:56.246 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.752597+0000", 2024-04-20T01:47:56.246 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0072354910000000001 2024-04-20T01:47:56.246 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.246 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.246 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:56.246 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.752617+0000", 2024-04-20T01:47:56.246 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0341000000000001e-05 2024-04-20T01:47:56.246 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.246 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.246 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.246 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.246 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.247 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 47/30 ECSubWrite(tid=247, reqid=client.4643.0:389, at_version=47'39, trim_to=0'0, roll_forward_to=47'35))", 2024-04-20T01:47:56.247 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.746671+0000", 2024-04-20T01:47:56.247 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 7.4962532980000001, 2024-04-20T01:47:56.247 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0059586839999999997, 2024-04-20T01:47:56.247 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:56.247 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:56.247 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:56.247 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.247 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:56.247 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.746671+0000", 2024-04-20T01:47:56.247 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.247 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.247 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.247 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:56.247 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.746671+0000", 2024-04-20T01:47:56.248 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.248 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.248 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.248 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:56.248 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.746670+0000", 2024-04-20T01:47:56.248 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:56.248 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.248 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.248 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:56.248 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.746736+0000", 2024-04-20T01:47:56.248 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.5914999999999995e-05 2024-04-20T01:47:56.248 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.248 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.248 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:56.248 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.746749+0000", 2024-04-20T01:47:56.248 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2245999999999999e-05 2024-04-20T01:47:56.249 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.249 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.249 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:56.249 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.746758+0000", 2024-04-20T01:47:56.249 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4590000000000004e-06 2024-04-20T01:47:56.249 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.249 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.249 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:56.249 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.747358+0000", 2024-04-20T01:47:56.249 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00060028299999999996 2024-04-20T01:47:56.249 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.249 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.249 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:56.249 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.747367+0000", 2024-04-20T01:47:56.250 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.2670000000000006e-06 2024-04-20T01:47:56.250 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.250 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.250 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:56.250 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.752621+0000", 2024-04-20T01:47:56.250 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0052539919999999999 2024-04-20T01:47:56.250 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.250 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.250 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:56.250 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.752630+0000", 2024-04-20T01:47:56.250 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.1940000000000004e-06 2024-04-20T01:47:56.250 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.251 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.251 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.251 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.251 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.251 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 47/30 ECSubWrite(tid=248, reqid=client.4643.0:390, at_version=47'40, trim_to=0'0, roll_forward_to=47'35))", 2024-04-20T01:47:56.251 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.747763+0000", 2024-04-20T01:47:56.251 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 7.4951613070000001, 2024-04-20T01:47:56.251 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.004878088, 2024-04-20T01:47:56.251 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:56.251 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:56.251 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:56.251 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.251 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:56.251 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.747763+0000", 2024-04-20T01:47:56.252 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.252 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.252 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.252 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:56.252 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.747763+0000", 2024-04-20T01:47:56.252 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.252 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.252 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.252 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:56.252 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.747762+0000", 2024-04-20T01:47:56.252 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:56.252 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.252 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.252 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:56.252 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.747779+0000", 2024-04-20T01:47:56.253 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7272e-05 2024-04-20T01:47:56.253 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.253 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.253 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:56.253 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.747792+0000", 2024-04-20T01:47:56.253 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2323999999999999e-05 2024-04-20T01:47:56.253 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.253 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.253 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:56.253 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.747801+0000", 2024-04-20T01:47:56.253 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8089999999999999e-06 2024-04-20T01:47:56.253 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.253 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.253 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:56.253 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.747843+0000", 2024-04-20T01:47:56.254 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2876000000000001e-05 2024-04-20T01:47:56.254 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.254 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.254 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:56.254 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.747853+0000", 2024-04-20T01:47:56.254 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4930000000000004e-06 2024-04-20T01:47:56.254 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.254 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.254 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:56.254 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.752632+0000", 2024-04-20T01:47:56.254 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0047787890000000003 2024-04-20T01:47:56.254 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.254 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.254 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:56.255 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.752641+0000", 2024-04-20T01:47:56.255 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3449999999999995e-06 2024-04-20T01:47:56.255 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.255 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.255 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.255 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.255 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.255 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 48/30 ECSubWrite(tid=84, reqid=osd.8.0:84, at_version=48'69, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:47:56.255 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.709204+0000", 2024-04-20T01:47:56.255 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 6.5337203969999997, 2024-04-20T01:47:56.255 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0059724779999999998, 2024-04-20T01:47:56.255 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:56.255 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:56.255 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:56.255 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.256 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:56.256 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.709204+0000", 2024-04-20T01:47:56.256 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.256 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.256 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.256 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:56.256 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.709204+0000", 2024-04-20T01:47:56.256 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.256 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.256 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.256 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:56.256 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.709203+0000", 2024-04-20T01:47:56.256 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:56.256 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.256 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.257 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:56.257 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.709222+0000", 2024-04-20T01:47:56.257 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8780999999999999e-05 2024-04-20T01:47:56.257 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.257 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.257 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:56.257 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.709233+0000", 2024-04-20T01:47:56.257 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.045e-05 2024-04-20T01:47:56.257 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.257 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.257 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:56.257 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.709243+0000", 2024-04-20T01:47:56.257 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0111e-05 2024-04-20T01:47:56.257 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.258 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.258 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:56.258 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.709284+0000", 2024-04-20T01:47:56.258 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.1495999999999997e-05 2024-04-20T01:47:56.258 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.258 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.258 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:56.258 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.709294+0000", 2024-04-20T01:47:56.258 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3689999999999996e-06 2024-04-20T01:47:56.258 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.258 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.258 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:56.258 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715152+0000", 2024-04-20T01:47:56.258 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0058586300000000001 2024-04-20T01:47:56.259 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.259 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.259 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:56.259 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715176+0000", 2024-04-20T01:47:56.259 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4139999999999999e-05 2024-04-20T01:47:56.259 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.259 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.259 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.259 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.259 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.259 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 48/30 ECSubWrite(tid=252, reqid=osd.13.0:252, at_version=48'42, trim_to=0'0, roll_forward_to=47'40))", 2024-04-20T01:47:56.259 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.710137+0000", 2024-04-20T01:47:56.259 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 6.5327871130000004, 2024-04-20T01:47:56.259 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0082943410000000002, 2024-04-20T01:47:56.259 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:56.260 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:56.260 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:56.260 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.260 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:56.260 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710137+0000", 2024-04-20T01:47:56.260 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.260 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.260 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.260 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:56.260 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710137+0000", 2024-04-20T01:47:56.260 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.260 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.260 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.260 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:56.261 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710137+0000", 2024-04-20T01:47:56.261 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:56.261 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.261 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.261 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:56.261 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710162+0000", 2024-04-20T01:47:56.261 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4768000000000001e-05 2024-04-20T01:47:56.261 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.261 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.261 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:56.261 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710174+0000", 2024-04-20T01:47:56.261 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2150999999999999e-05 2024-04-20T01:47:56.261 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.261 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.262 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:56.262 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710183+0000", 2024-04-20T01:47:56.262 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8200000000000003e-06 2024-04-20T01:47:56.262 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.262 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.262 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:56.262 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710238+0000", 2024-04-20T01:47:56.262 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.4954999999999998e-05 2024-04-20T01:47:56.262 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.262 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.262 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:56.262 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710248+0000", 2024-04-20T01:47:56.262 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0222e-05 2024-04-20T01:47:56.262 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.262 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.263 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:56.263 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718408+0000", 2024-04-20T01:47:56.263 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0081605570000000006 2024-04-20T01:47:56.263 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.263 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.263 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:56.263 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718432+0000", 2024-04-20T01:47:56.263 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3129000000000002e-05 2024-04-20T01:47:56.263 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.263 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.263 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.263 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.263 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.263 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 48/30 ECSubWrite(tid=253, reqid=osd.13.0:253, at_version=48'44, trim_to=0'0, roll_forward_to=47'40))", 2024-04-20T01:47:56.263 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.711047+0000", 2024-04-20T01:47:56.263 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 6.5318776950000004, 2024-04-20T01:47:56.264 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0093898609999999993, 2024-04-20T01:47:56.264 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:56.264 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:56.264 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:56.264 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.264 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:56.264 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711047+0000", 2024-04-20T01:47:56.264 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.264 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.264 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.264 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:56.264 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711047+0000", 2024-04-20T01:47:56.264 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.264 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.264 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.265 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:56.265 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711046+0000", 2024-04-20T01:47:56.265 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:56.265 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.265 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.265 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:56.265 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711066+0000", 2024-04-20T01:47:56.265 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0421999999999999e-05 2024-04-20T01:47:56.265 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.265 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.265 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:56.265 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711082+0000", 2024-04-20T01:47:56.265 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5231999999999999e-05 2024-04-20T01:47:56.265 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.265 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.266 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:56.266 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711091+0000", 2024-04-20T01:47:56.266 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.1679999999999993e-06 2024-04-20T01:47:56.266 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.266 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.266 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:56.266 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.712543+0000", 2024-04-20T01:47:56.266 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001452545 2024-04-20T01:47:56.266 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.266 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.266 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:56.266 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.712549+0000", 2024-04-20T01:47:56.266 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.7450000000000001e-06 2024-04-20T01:47:56.266 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.266 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.267 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:56.267 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720419+0000", 2024-04-20T01:47:56.267 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0078696630000000007 2024-04-20T01:47:56.267 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.267 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.267 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:56.267 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720436+0000", 2024-04-20T01:47:56.267 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7869000000000001e-05 2024-04-20T01:47:56.267 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.267 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.267 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.267 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.267 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.267 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 48/30 ECSubWrite(tid=85, reqid=client.4643.0:399, at_version=48'71, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:47:56.268 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.711873+0000", 2024-04-20T01:47:56.268 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 6.5310508919999997, 2024-04-20T01:47:56.268 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0060631469999999996, 2024-04-20T01:47:56.268 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:56.268 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:56.268 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:56.268 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.268 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:56.268 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711873+0000", 2024-04-20T01:47:56.268 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.268 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.268 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.268 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:56.268 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711873+0000", 2024-04-20T01:47:56.268 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.269 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.269 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.269 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:56.269 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711873+0000", 2024-04-20T01:47:56.269 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:47:56.269 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.269 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.269 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:56.269 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711898+0000", 2024-04-20T01:47:56.269 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5327999999999999e-05 2024-04-20T01:47:56.269 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.269 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.269 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:56.269 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711907+0000", 2024-04-20T01:47:56.269 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.7839999999999992e-06 2024-04-20T01:47:56.270 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.270 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.270 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:56.270 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711913+0000", 2024-04-20T01:47:56.270 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.0599999999999996e-06 2024-04-20T01:47:56.270 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.270 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.270 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:56.270 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711972+0000", 2024-04-20T01:47:56.270 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.9212000000000002e-05 2024-04-20T01:47:56.270 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.270 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.270 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:56.270 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711983+0000", 2024-04-20T01:47:56.271 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1126e-05 2024-04-20T01:47:56.271 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.271 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.271 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:56.271 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.717902+0000", 2024-04-20T01:47:56.271 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0059184529999999997 2024-04-20T01:47:56.271 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.271 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.271 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:56.271 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.717937+0000", 2024-04-20T01:47:56.271 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.4909000000000001e-05 2024-04-20T01:47:56.271 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.271 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.271 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.271 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.272 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.272 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 48/30 ECSubWrite(tid=86, reqid=client.4643.0:400, at_version=48'72, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:47:56.272 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.713378+0000", 2024-04-20T01:47:56.272 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 6.5295462119999996, 2024-04-20T01:47:56.272 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0070552389999999996, 2024-04-20T01:47:56.272 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:56.272 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:56.272 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:56.272 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.272 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:56.272 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713378+0000", 2024-04-20T01:47:56.272 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.272 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.272 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.272 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:56.273 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713378+0000", 2024-04-20T01:47:56.273 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.273 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.273 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.273 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:56.273 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713378+0000", 2024-04-20T01:47:56.273 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:56.273 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.273 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.273 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:56.273 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713397+0000", 2024-04-20T01:47:56.273 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9165999999999999e-05 2024-04-20T01:47:56.273 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.273 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.273 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:56.274 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713408+0000", 2024-04-20T01:47:56.274 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0854e-05 2024-04-20T01:47:56.274 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.274 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.274 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:56.274 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713414+0000", 2024-04-20T01:47:56.274 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.1850000000000001e-06 2024-04-20T01:47:56.274 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.274 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.274 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:56.274 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713458+0000", 2024-04-20T01:47:56.274 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.4162999999999997e-05 2024-04-20T01:47:56.274 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.274 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.274 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:56.275 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713469+0000", 2024-04-20T01:47:56.275 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0789e-05 2024-04-20T01:47:56.275 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.275 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.275 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:56.275 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720404+0000", 2024-04-20T01:47:56.275 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0069350109999999996 2024-04-20T01:47:56.275 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.275 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.275 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:56.275 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720433+0000", 2024-04-20T01:47:56.275 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9660000000000001e-05 2024-04-20T01:47:56.275 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.275 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.275 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.276 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.276 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.276 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 48/30 ECSubWrite(tid=87, reqid=client.4643.0:401, at_version=48'73, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:47:56.276 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.713964+0000", 2024-04-20T01:47:56.276 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 6.5289604509999997, 2024-04-20T01:47:56.276 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0065393960000000003, 2024-04-20T01:47:56.276 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:56.276 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:56.276 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:56.276 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.276 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:56.276 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713964+0000", 2024-04-20T01:47:56.276 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.276 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.276 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.277 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:56.277 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713964+0000", 2024-04-20T01:47:56.277 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.277 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.277 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.277 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:56.277 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713963+0000", 2024-04-20T01:47:56.277 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:56.277 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.277 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.277 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:56.277 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713977+0000", 2024-04-20T01:47:56.277 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3828999999999999e-05 2024-04-20T01:47:56.277 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.277 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.277 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:56.278 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713988+0000", 2024-04-20T01:47:56.278 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0728e-05 2024-04-20T01:47:56.278 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.278 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.278 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:56.278 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713994+0000", 2024-04-20T01:47:56.278 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.2949999999999999e-06 2024-04-20T01:47:56.278 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.278 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.278 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:56.278 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715264+0000", 2024-04-20T01:47:56.278 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00126958 2024-04-20T01:47:56.278 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.278 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.279 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:56.279 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715273+0000", 2024-04-20T01:47:56.279 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2720000000000003e-06 2024-04-20T01:47:56.279 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.279 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.279 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:56.279 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720484+0000", 2024-04-20T01:47:56.279 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0052106820000000003 2024-04-20T01:47:56.279 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.279 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.279 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:56.279 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720503+0000", 2024-04-20T01:47:56.279 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9706000000000001e-05 2024-04-20T01:47:56.279 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.279 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.279 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.280 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.280 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.280 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.9s1 48/30 ECSubWrite(tid=17, reqid=osd.7.0:17, at_version=48'12, trim_to=0'0, roll_forward_to=47'10))", 2024-04-20T01:47:56.280 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.715248+0000", 2024-04-20T01:47:56.280 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 6.5276761709999995, 2024-04-20T01:47:56.280 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.005232039, 2024-04-20T01:47:56.280 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:56.280 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:56.280 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:56.280 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.280 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:56.280 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715248+0000", 2024-04-20T01:47:56.280 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.280 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.280 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.281 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:56.281 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715248+0000", 2024-04-20T01:47:56.281 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.281 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.281 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.281 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:56.281 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715248+0000", 2024-04-20T01:47:56.281 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:47:56.281 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.281 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.281 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:56.281 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715269+0000", 2024-04-20T01:47:56.281 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1322999999999999e-05 2024-04-20T01:47:56.281 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.281 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.282 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:56.282 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715278+0000", 2024-04-20T01:47:56.282 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.0869999999999996e-06 2024-04-20T01:47:56.282 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.282 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.282 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:56.282 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715286+0000", 2024-04-20T01:47:56.282 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.1000000000000004e-06 2024-04-20T01:47:56.282 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.282 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.282 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:56.282 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715336+0000", 2024-04-20T01:47:56.282 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.9503999999999999e-05 2024-04-20T01:47:56.282 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.283 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.283 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:56.283 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715347+0000", 2024-04-20T01:47:56.283 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0577000000000001e-05 2024-04-20T01:47:56.283 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.283 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.283 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:56.283 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720458+0000", 2024-04-20T01:47:56.283 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0051115600000000002 2024-04-20T01:47:56.283 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.283 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.283 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:56.283 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720480+0000", 2024-04-20T01:47:56.283 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2113e-05 2024-04-20T01:47:56.284 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.284 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.284 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.284 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.284 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.284 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.as2 48/30 ECSubWrite(tid=33, reqid=osd.6.0:33, at_version=48'14, trim_to=0'0, roll_forward_to=47'12))", 2024-04-20T01:47:56.284 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.716701+0000", 2024-04-20T01:47:56.284 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 6.5262230370000003, 2024-04-20T01:47:56.284 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.006490778, 2024-04-20T01:47:56.284 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:56.284 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:56.284 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:56.284 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.284 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:56.284 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.716701+0000", 2024-04-20T01:47:56.285 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.285 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.285 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.285 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:56.285 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.716701+0000", 2024-04-20T01:47:56.285 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.285 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.285 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.285 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:56.285 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.716701+0000", 2024-04-20T01:47:56.285 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:47:56.285 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.285 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.285 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:56.285 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.716719+0000", 2024-04-20T01:47:56.286 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7574000000000001e-05 2024-04-20T01:47:56.286 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.286 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.286 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:56.286 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.716731+0000", 2024-04-20T01:47:56.286 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1949e-05 2024-04-20T01:47:56.286 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.286 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.286 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:56.286 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.716739+0000", 2024-04-20T01:47:56.286 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4190000000000002e-06 2024-04-20T01:47:56.286 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.286 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.286 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:56.287 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.716782+0000", 2024-04-20T01:47:56.287 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2803000000000003e-05 2024-04-20T01:47:56.287 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.287 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.287 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:56.287 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.716789+0000", 2024-04-20T01:47:56.287 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.3100000000000003e-06 2024-04-20T01:47:56.287 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.287 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.287 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:56.287 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723173+0000", 2024-04-20T01:47:56.287 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0063837140000000004 2024-04-20T01:47:56.287 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.287 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.287 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:56.287 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723192+0000", 2024-04-20T01:47:56.288 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9154999999999999e-05 2024-04-20T01:47:56.288 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.288 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.288 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.288 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.288 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.288 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 48/30 ECSubWrite(tid=120, reqid=osd.4.0:120, at_version=48'51, trim_to=0'0, roll_forward_to=47'49))", 2024-04-20T01:47:56.288 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.717383+0000", 2024-04-20T01:47:56.288 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 6.5255416190000002, 2024-04-20T01:47:56.288 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0058028560000000003, 2024-04-20T01:47:56.288 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:56.288 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:56.288 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:56.288 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.288 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:56.289 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.717383+0000", 2024-04-20T01:47:56.289 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.289 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.289 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.289 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:56.289 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.717383+0000", 2024-04-20T01:47:56.289 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.289 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.289 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.289 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:56.289 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.717382+0000", 2024-04-20T01:47:56.289 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:56.289 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.289 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.289 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:56.290 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.717404+0000", 2024-04-20T01:47:56.290 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1586999999999998e-05 2024-04-20T01:47:56.290 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.290 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.290 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:56.290 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.717419+0000", 2024-04-20T01:47:56.290 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5180999999999999e-05 2024-04-20T01:47:56.290 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.290 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.290 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:56.290 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.717429+0000", 2024-04-20T01:47:56.290 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4290000000000007e-06 2024-04-20T01:47:56.290 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.290 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.291 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:56.291 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.717502+0000", 2024-04-20T01:47:56.291 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.3638000000000002e-05 2024-04-20T01:47:56.291 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.291 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.291 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:56.291 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.717511+0000", 2024-04-20T01:47:56.291 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.0729999999999994e-06 2024-04-20T01:47:56.291 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.291 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.291 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:56.291 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723164+0000", 2024-04-20T01:47:56.291 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0056525639999999997 2024-04-20T01:47:56.291 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.291 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.292 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:56.292 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723186+0000", 2024-04-20T01:47:56.292 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1637999999999998e-05 2024-04-20T01:47:56.292 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.292 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.292 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.292 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.292 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.292 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 48/30 ECSubWrite(tid=121, reqid=osd.4.0:121, at_version=48'53, trim_to=0'0, roll_forward_to=47'49))", 2024-04-20T01:47:56.292 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.718319+0000", 2024-04-20T01:47:56.292 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 6.5246057400000002, 2024-04-20T01:47:56.292 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0075908039999999996, 2024-04-20T01:47:56.292 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:56.292 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:56.293 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:56.293 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.293 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:56.293 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718319+0000", 2024-04-20T01:47:56.293 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.293 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.293 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.293 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:56.293 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718319+0000", 2024-04-20T01:47:56.293 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.293 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.293 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.293 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:56.293 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718318+0000", 2024-04-20T01:47:56.293 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:56.294 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.294 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.294 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:56.294 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718351+0000", 2024-04-20T01:47:56.294 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.3179000000000002e-05 2024-04-20T01:47:56.294 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.294 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.294 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:56.294 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718362+0000", 2024-04-20T01:47:56.294 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0689999999999999e-05 2024-04-20T01:47:56.294 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.294 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.294 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:56.294 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718370+0000", 2024-04-20T01:47:56.294 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4610000000000008e-06 2024-04-20T01:47:56.295 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.295 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.295 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:56.295 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718703+0000", 2024-04-20T01:47:56.295 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00033314100000000001 2024-04-20T01:47:56.295 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.295 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.295 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:56.295 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718713+0000", 2024-04-20T01:47:56.295 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.5459999999999997e-06 2024-04-20T01:47:56.295 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.295 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.295 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:56.295 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.725887+0000", 2024-04-20T01:47:56.295 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0071743909999999996 2024-04-20T01:47:56.296 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.296 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.296 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:56.296 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.725909+0000", 2024-04-20T01:47:56.296 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1965999999999999e-05 2024-04-20T01:47:56.296 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.296 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.296 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.296 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.296 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.296 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.9s1 48/30 ECSubWrite(tid=18, reqid=osd.7.0:17, at_version=0'0, trim_to=0'0, roll_forward_to=48'12))", 2024-04-20T01:47:56.296 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.723162+0000", 2024-04-20T01:47:56.296 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 6.5197626990000002, 2024-04-20T01:47:56.296 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.005487034, 2024-04-20T01:47:56.297 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:56.297 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:56.297 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:56.297 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.297 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:56.297 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723162+0000", 2024-04-20T01:47:56.297 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.297 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.297 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.297 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:56.297 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723162+0000", 2024-04-20T01:47:56.297 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.297 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.297 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.297 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:56.298 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723161+0000", 2024-04-20T01:47:56.298 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:56.298 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.298 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.298 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:56.298 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723171+0000", 2024-04-20T01:47:56.298 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4960000000000002e-06 2024-04-20T01:47:56.298 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.298 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.298 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:56.298 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723179+0000", 2024-04-20T01:47:56.298 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.3999999999999992e-06 2024-04-20T01:47:56.298 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.298 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.299 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:56.299 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723191+0000", 2024-04-20T01:47:56.299 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1878999999999999e-05 2024-04-20T01:47:56.299 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.299 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.299 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:56.299 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723236+0000", 2024-04-20T01:47:56.299 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5611000000000001e-05 2024-04-20T01:47:56.299 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.299 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.299 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:56.299 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723246+0000", 2024-04-20T01:47:56.299 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.9019999999999994e-06 2024-04-20T01:47:56.299 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.299 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.300 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:56.300 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.728625+0000", 2024-04-20T01:47:56.300 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0053787369999999998 2024-04-20T01:47:56.300 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.300 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.300 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:56.300 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.728649+0000", 2024-04-20T01:47:56.300 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3589000000000001e-05 2024-04-20T01:47:56.300 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.300 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.300 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.300 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.300 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.300 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.as2 48/30 ECSubWrite(tid=34, reqid=osd.6.0:33, at_version=0'0, trim_to=0'0, roll_forward_to=48'14))", 2024-04-20T01:47:56.300 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.726124+0000", 2024-04-20T01:47:56.301 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 6.5167999119999998, 2024-04-20T01:47:56.301 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0051488189999999998, 2024-04-20T01:47:56.301 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:56.301 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:56.301 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:56.301 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.301 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:56.301 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.726124+0000", 2024-04-20T01:47:56.301 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.301 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.301 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.301 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:56.301 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.726124+0000", 2024-04-20T01:47:56.301 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.301 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.302 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.302 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:56.302 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.726124+0000", 2024-04-20T01:47:56.302 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:56.302 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.302 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.302 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:56.302 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.726134+0000", 2024-04-20T01:47:56.302 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.9040000000000004e-06 2024-04-20T01:47:56.302 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.302 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.302 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:56.302 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.726147+0000", 2024-04-20T01:47:56.302 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3366e-05 2024-04-20T01:47:56.302 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.303 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.303 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:56.303 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.726158+0000", 2024-04-20T01:47:56.303 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.06e-05 2024-04-20T01:47:56.303 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.303 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.303 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:56.303 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.726207+0000", 2024-04-20T01:47:56.303 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.9153000000000002e-05 2024-04-20T01:47:56.303 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.303 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.303 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:56.303 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.726214+0000", 2024-04-20T01:47:56.303 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.9630000000000001e-06 2024-04-20T01:47:56.304 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.304 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.304 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:56.304 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.731248+0000", 2024-04-20T01:47:56.304 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0050344830000000002 2024-04-20T01:47:56.304 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.304 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.304 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:56.304 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.731273+0000", 2024-04-20T01:47:56.304 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5040000000000001e-05 2024-04-20T01:47:56.304 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.304 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.304 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.304 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.304 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.304 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s3 53/30 ECSubWrite(tid=32, reqid=client.4643.0:443, at_version=53'21, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:47:56.305 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:54.718764+0000", 2024-04-20T01:47:56.305 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 1.524160103, 2024-04-20T01:47:56.305 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0048456610000000002, 2024-04-20T01:47:56.305 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:56.305 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:56.305 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:56.305 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.305 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:56.305 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.718764+0000", 2024-04-20T01:47:56.305 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.305 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.305 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.305 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:56.305 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.718764+0000", 2024-04-20T01:47:56.305 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.306 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.306 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.306 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:56.306 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.718764+0000", 2024-04-20T01:47:56.306 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:56.306 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.306 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.306 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:56.306 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.719094+0000", 2024-04-20T01:47:56.306 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000330534 2024-04-20T01:47:56.306 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.306 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.306 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:56.306 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.719111+0000", 2024-04-20T01:47:56.306 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7027000000000001e-05 2024-04-20T01:47:56.307 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.307 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.307 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:56.307 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.719125+0000", 2024-04-20T01:47:56.307 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3675e-05 2024-04-20T01:47:56.307 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.307 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.307 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:56.307 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.719231+0000", 2024-04-20T01:47:56.307 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0001058 2024-04-20T01:47:56.307 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.307 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.307 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:56.307 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.719251+0000", 2024-04-20T01:47:56.307 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9698999999999999e-05 2024-04-20T01:47:56.308 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.308 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.308 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:56.308 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723583+0000", 2024-04-20T01:47:56.308 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.004331957 2024-04-20T01:47:56.308 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.308 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.308 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:56.308 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723610+0000", 2024-04-20T01:47:56.308 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7285e-05 2024-04-20T01:47:56.308 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.308 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.308 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.308 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.308 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.309 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s3 53/30 ECSubWrite(tid=34, reqid=client.4643.0:445, at_version=53'23, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:47:56.309 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:54.722837+0000", 2024-04-20T01:47:56.309 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 1.5200873029999999, 2024-04-20T01:47:56.309 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0058785790000000001, 2024-04-20T01:47:56.309 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:47:56.309 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:47:56.309 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:47:56.309 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.309 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:47:56.309 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.722837+0000", 2024-04-20T01:47:56.309 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.309 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.309 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.309 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:47:56.309 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.722837+0000", 2024-04-20T01:47:56.309 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:47:56.310 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.310 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.310 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:47:56.310 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.722836+0000", 2024-04-20T01:47:56.310 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:56.310 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.310 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.310 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:47:56.310 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723118+0000", 2024-04-20T01:47:56.310 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00028123299999999998 2024-04-20T01:47:56.310 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.310 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.310 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:47:56.311 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723136+0000", 2024-04-20T01:47:56.311 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8400999999999999e-05 2024-04-20T01:47:56.311 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.311 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.311 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:47:56.311 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723145+0000", 2024-04-20T01:47:56.311 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.5140000000000001e-06 2024-04-20T01:47:56.311 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.311 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.311 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:47:56.311 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723204+0000", 2024-04-20T01:47:56.311 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.9077999999999999e-05 2024-04-20T01:47:56.311 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.311 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.311 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:47:56.312 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723213+0000", 2024-04-20T01:47:56.312 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3980000000000005e-06 2024-04-20T01:47:56.312 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.312 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.312 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:47:56.312 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.728692+0000", 2024-04-20T01:47:56.312 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0054791789999999998 2024-04-20T01:47:56.312 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:47:56.312 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:47:56.312 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:47:56.312 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.728716+0000", 2024-04-20T01:47:56.312 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3388e-05 2024-04-20T01:47:56.312 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.312 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.312 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.313 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:47:56.313 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:47:56.313 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:56.313 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-20T01:47:56.315 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:47:56.329+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:47:56.315 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-20T01:47:56.415 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:56.430+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:47:56.443 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:56.444 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:47:56.444 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 0 2024-04-20T01:47:56.444 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:56.454 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-20T01:47:56.480 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:47:56.502 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:47:56.516 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:56.531+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:56.523 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:47:56.545 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:47:56.565 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:47:56.582 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:47:56.582 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:47:56.582 INFO:teuthology.orchestra.run.smithi121.stdout: "complaint_time": 30, 2024-04-20T01:47:56.582 INFO:teuthology.orchestra.run.smithi121.stdout: "num_blocked_ops": 0 2024-04-20T01:47:56.583 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:47:56.586 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:47:56.593 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-20T01:47:56.607 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:47:56.617 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:47:56.632+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:47:56.628 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:47:56.648 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:47:56.670 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:47:56.691 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:47:56.701 INFO:tasks.rados.rados.0.smithi165.stdout:187: done (1 left) 2024-04-20T01:47:56.701 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 5 current snap is 12 2024-04-20T01:47:56.701 INFO:tasks.rados.rados.0.smithi165.stdout:185: finishing write tid 1 to smithi16579500-30 2024-04-20T01:47:56.701 INFO:tasks.rados.rados.0.smithi165.stdout:188: seq_num 89 ranges {81920=737280,819200=573440,1392640=196608} 2024-04-20T01:47:56.704 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is set 2024-04-20T01:47:56.706 INFO:tasks.rados.rados.0.smithi165.stdout:188: writing smithi16579500-5 from 81920 to 819200 tid 1 2024-04-20T01:47:56.709 INFO:tasks.rados.rados.0.smithi165.stdout:188: writing smithi16579500-5 from 819200 to 1392640 tid 2 2024-04-20T01:47:56.711 INFO:tasks.rados.rados.0.smithi165.stdout:188: writing smithi16579500-5 from 1392640 to 1589248 tid 3 2024-04-20T01:47:56.711 INFO:tasks.rados.rados.0.smithi165.stdout:189: rollback oid 37 current snap is 12 2024-04-20T01:47:56.711 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 37 to 10 2024-04-20T01:47:56.711 INFO:tasks.rados.rados.0.smithi165.stdout:185: finishing write tid 2 to smithi16579500-30 2024-04-20T01:47:56.711 INFO:tasks.rados.rados.0.smithi165.stdout:185: finishing write tid 3 to smithi16579500-30 2024-04-20T01:47:56.711 INFO:tasks.rados.rados.0.smithi165.stdout:185: finishing write tid 4 to smithi16579500-30 2024-04-20T01:47:56.711 INFO:tasks.rados.rados.0.smithi165.stdout:185: finishing write tid 5 to smithi16579500-30 2024-04-20T01:47:56.712 INFO:tasks.rados.rados.0.smithi165.stdout:185: oid 30 updating version 0 to 21 2024-04-20T01:47:56.712 INFO:tasks.rados.rados.0.smithi165.stdout:185: oid 30 updating version 21 to 23 2024-04-20T01:47:56.712 INFO:tasks.rados.rados.0.smithi165.stdout:185: oid 30 updating version 23 to 24 2024-04-20T01:47:56.712 INFO:tasks.rados.rados.0.smithi165.stdout:185: oid 30 version 24 is already newer than 22 2024-04-20T01:47:56.712 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 30 v 24 (ObjNum 88 snap 12 seq_num 88) dirty exists 2024-04-20T01:47:56.712 INFO:tasks.rados.rados.0.smithi165.stdout:185: left oid 30 (ObjNum 88 snap 12 seq_num 88) 2024-04-20T01:47:56.712 INFO:tasks.rados.rados.0.smithi165.stdout:185: done (2 left) 2024-04-20T01:47:56.712 INFO:tasks.rados.rados.0.smithi165.stdout:190: delete oid 32 current snap is 12 2024-04-20T01:47:56.715 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:47:56.718 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:56.732+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:56.722 INFO:tasks.rados.rados.0.smithi165.stdout:190: done (2 left) 2024-04-20T01:47:56.723 INFO:tasks.rados.rados.0.smithi165.stdout:191: copy_from oid 39 from oid 26 current snap is 12 2024-04-20T01:47:56.723 INFO:tasks.rados.rados.0.smithi165.stdout:192: rollback oid 11 current snap is 12 2024-04-20T01:47:56.723 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 11 to 5 2024-04-20T01:47:56.723 INFO:tasks.rados.rados.0.smithi165.stdout:193: rmattr oid 6 current snap is 12 2024-04-20T01:47:56.723 INFO:tasks.rados.rados.0.smithi165.stdout:193: done (4 left) 2024-04-20T01:47:56.723 INFO:tasks.rados.rados.0.smithi165.stdout:194: read oid 41 snap 10 2024-04-20T01:47:56.723 INFO:tasks.rados.rados.0.smithi165.stdout:194: expect (ObjNum 75 snap 7 seq_num 75) 2024-04-20T01:47:56.723 INFO:tasks.rados.rados.0.smithi165.stdout:195: delete oid 43 current snap is 12 2024-04-20T01:47:56.724 INFO:tasks.rados.rados.0.smithi165.stdout:188: finishing write tid 1 to smithi16579500-5 2024-04-20T01:47:56.729 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:56.729 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:56.731 INFO:tasks.rados.rados.0.smithi165.stdout:189: finishing rollback tid 0 to smithi16579500-37 2024-04-20T01:47:56.732 INFO:tasks.rados.rados.0.smithi165.stdout:189: finishing rollback tid 1 to smithi16579500-37 2024-04-20T01:47:56.732 INFO:tasks.rados.rados.0.smithi165.stdout:189: finishing rollback tid 2 to smithi16579500-37 2024-04-20T01:47:56.732 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 37 v 27 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-04-20T01:47:56.733 INFO:tasks.rados.rados.0.smithi165.stdout:192: finishing rollback tid 0 to smithi16579500-11 2024-04-20T01:47:56.734 INFO:tasks.rados.rados.0.smithi165.stdout:192: finishing rollback tid 1 to smithi16579500-11 2024-04-20T01:47:56.734 INFO:tasks.rados.rados.0.smithi165.stdout:192: finishing rollback tid 2 to smithi16579500-11 2024-04-20T01:47:56.734 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 11 v 37 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-04-20T01:47:56.736 INFO:tasks.rados.rados.0.smithi165.stdout:188: finishing write tid 2 to smithi16579500-5 2024-04-20T01:47:56.736 INFO:tasks.rados.rados.0.smithi165.stdout:188: finishing write tid 3 to smithi16579500-5 2024-04-20T01:47:56.738 INFO:tasks.rados.rados.0.smithi165.stdout:188: finishing write tid 4 to smithi16579500-5 2024-04-20T01:47:56.740 INFO:tasks.rados.rados.0.smithi165.stdout:188: finishing write tid 5 to smithi16579500-5 2024-04-20T01:47:56.740 INFO:tasks.rados.rados.0.smithi165.stdout:188: oid 5 updating version 0 to 31 2024-04-20T01:47:56.740 INFO:tasks.rados.rados.0.smithi165.stdout:188: oid 5 updating version 31 to 32 2024-04-20T01:47:56.740 INFO:tasks.rados.rados.0.smithi165.stdout:188: oid 5 updating version 32 to 33 2024-04-20T01:47:56.740 INFO:tasks.rados.rados.0.smithi165.stdout:188: oid 5 version 33 is already newer than 28 2024-04-20T01:47:56.740 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 5 v 33 (ObjNum 89 snap 12 seq_num 89) dirty exists 2024-04-20T01:47:56.740 INFO:tasks.rados.rados.0.smithi165.stdout:188: left oid 5 (ObjNum 89 snap 12 seq_num 89) 2024-04-20T01:47:56.743 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-20T01:47:56.747 INFO:tasks.rados.rados.0.smithi165.stdout:194: expect (ObjNum 75 snap 7 seq_num 75) 2024-04-20T01:47:56.750 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:47:56.770 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:47:56.791 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:47:56.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:47:56.807+0000 7f4e72d9d700 0 -- 172.21.15.44:0/2734242086 >> [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] conn(0x7f4e7421f460 msgr2=0x7f4e74221850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-20T01:47:56.808 INFO:tasks.rados.rados.0.smithi165.stdout:191: finishing copy_from racing read to smithi16579500-39 2024-04-20T01:47:56.809 INFO:tasks.rados.rados.0.smithi165.stdout:191: finishing copy_from to smithi16579500-39 2024-04-20T01:47:56.809 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 39 v 58 (ObjNum 87 snap 11 seq_num 87) dirty exists 2024-04-20T01:47:56.809 INFO:tasks.rados.rados.0.smithi165.stdout:188: done (5 left) 2024-04-20T01:47:56.809 INFO:tasks.rados.rados.0.smithi165.stdout:189: done (4 left) 2024-04-20T01:47:56.809 INFO:tasks.rados.rados.0.smithi165.stdout:191: done (3 left) 2024-04-20T01:47:56.809 INFO:tasks.rados.rados.0.smithi165.stdout:192: done (2 left) 2024-04-20T01:47:56.809 INFO:tasks.rados.rados.0.smithi165.stdout:194: done (1 left) 2024-04-20T01:47:56.809 INFO:tasks.rados.rados.0.smithi165.stdout:195: done (0 left) 2024-04-20T01:47:56.809 INFO:tasks.rados.rados.0.smithi165.stdout:196: rollback oid 22 current snap is 12 2024-04-20T01:47:56.809 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 22 to 5 2024-04-20T01:47:56.809 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 36 current snap is 12 2024-04-20T01:47:56.809 INFO:tasks.rados.rados.0.smithi165.stdout:197: seq_num 92 ranges {1785856=786432,2572288=491520,3063808=278528} 2024-04-20T01:47:56.815 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:47:56.817 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:47:56.817 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:47:56.817 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:47:56.817 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:47:56.817 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:47:56.817 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:47:56.818 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:47:56.818 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:47:56.818 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:47:56.818 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:47:56.818 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:47:56.818 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:47:56.818 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:47:56.818 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:47:56.818 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:47:56.818 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:47:56.819 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:47:56.833+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:47:56.823 INFO:tasks.rados.rados.0.smithi165.stdout:197: writing smithi16579500-36 from 1785856 to 2572288 tid 1 2024-04-20T01:47:56.826 INFO:tasks.rados.rados.0.smithi165.stdout:197: writing smithi16579500-36 from 2572288 to 3063808 tid 2 2024-04-20T01:47:56.827 INFO:tasks.rados.rados.0.smithi165.stdout:197: writing smithi16579500-36 from 3063808 to 3342336 tid 3 2024-04-20T01:47:56.828 INFO:tasks.rados.rados.0.smithi165.stdout:198: snap_create 2024-04-20T01:47:56.828 INFO:tasks.rados.rados.0.smithi165.stdout:196: finishing rollback tid 0 to smithi16579500-22 2024-04-20T01:47:56.828 INFO:tasks.rados.rados.0.smithi165.stdout:196: finishing rollback tid 1 to smithi16579500-22 2024-04-20T01:47:56.828 INFO:tasks.rados.rados.0.smithi165.stdout:196: finishing rollback tid 2 to smithi16579500-22 2024-04-20T01:47:56.828 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 22 v 9 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-04-20T01:47:56.830 INFO:tasks.rados.rados.0.smithi165.stdout:197: finishing write tid 1 to smithi16579500-36 2024-04-20T01:47:56.831 INFO:tasks.rados.rados.0.smithi165.stdout:197: finishing write tid 2 to smithi16579500-36 2024-04-20T01:47:56.835 INFO:tasks.rados.rados.0.smithi165.stdout:197: finishing write tid 3 to smithi16579500-36 2024-04-20T01:47:56.835 INFO:tasks.rados.rados.0.smithi165.stdout:197: finishing write tid 4 to smithi16579500-36 2024-04-20T01:47:56.836 INFO:tasks.rados.rados.0.smithi165.stdout:197: finishing write tid 5 to smithi16579500-36 2024-04-20T01:47:56.836 INFO:tasks.rados.rados.0.smithi165.stdout:197: oid 36 updating version 0 to 39 2024-04-20T01:47:56.836 INFO:tasks.rados.rados.0.smithi165.stdout:197: oid 36 updating version 39 to 41 2024-04-20T01:47:56.836 INFO:tasks.rados.rados.0.smithi165.stdout:197: oid 36 version 41 is already newer than 38 2024-04-20T01:47:56.836 INFO:tasks.rados.rados.0.smithi165.stdout:197: oid 36 version 41 is already newer than 40 2024-04-20T01:47:56.836 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 36 v 41 (ObjNum 92 snap 12 seq_num 92) dirty exists 2024-04-20T01:47:56.836 INFO:tasks.rados.rados.0.smithi165.stdout:197: left oid 36 (ObjNum 92 snap 12 seq_num 92) 2024-04-20T01:47:56.913 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:56.913 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:56.920 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:47:56.935+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:47:56.923 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-20T01:47:57.021 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:47:57.035+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:47:57.051 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:57.051 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:57.061 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-20T01:47:57.122 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:47:57.137+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:47:57.189 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:57.189 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:57.199 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-20T01:47:57.223 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:47:57.238+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:47:57.324 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:47:57.338+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:47:57.327 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:57.327 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:57.337 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-20T01:47:57.425 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:47:57.439+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:47:57.464 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:57.464 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:57.474 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-20T01:47:57.526 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:57.541+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:47:57.602 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:57.602 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:57.612 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-20T01:47:57.627 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:57.641+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:57.710 INFO:tasks.rados.rados.0.smithi165.stdout:196: done (2 left) 2024-04-20T01:47:57.710 INFO:tasks.rados.rados.0.smithi165.stdout:197: done (1 left) 2024-04-20T01:47:57.710 INFO:tasks.rados.rados.0.smithi165.stdout:198: done (0 left) 2024-04-20T01:47:57.710 INFO:tasks.rados.rados.0.smithi165.stdout:199: snap_create 2024-04-20T01:47:57.714 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-20T01:47:57.714 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-20T01:47:57.715 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-04-20T01:47:57.715 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-20T01:47:57.728 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:57.743+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:47:57.762 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:57.762 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:57.773 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-20T01:47:57.829 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:47:57.843+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:47:57.902 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:57.902 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:57.912 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-20T01:47:57.930 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:47:57.944+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:47:58.031 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:58.045+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:47:58.040 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:58.040 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:58.050 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-20T01:47:58.089 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:47:58.089 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:47:58.089 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 56, 2024-04-20T01:47:58.089 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "9c2f64c1-aa35-4337-9f5c-081b9e3a2938", 2024-04-20T01:47:58.089 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2024-04-20T01:47:02.934241+0000", 2024-04-20T01:47:58.089 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-04-20T01:47:57.714018+0000", 2024-04-20T01:47:58.090 INFO:teuthology.orchestra.run.smithi044.stdout: "last_up_change": "2024-04-20T01:47:15.986754+0000", 2024-04-20T01:47:58.090 INFO:teuthology.orchestra.run.smithi044.stdout: "last_in_change": "2024-04-20T01:47:07.894719+0000", 2024-04-20T01:47:58.090 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-20T01:47:58.090 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_num": 5806080, 2024-04-20T01:47:58.090 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_set": [ 2024-04-20T01:47:58.090 INFO:teuthology.orchestra.run.smithi044.stdout: "nodeep-scrub", 2024-04-20T01:47:58.090 INFO:teuthology.orchestra.run.smithi044.stdout: "noscrub", 2024-04-20T01:47:58.090 INFO:teuthology.orchestra.run.smithi044.stdout: "pglog_hardlimit", 2024-04-20T01:47:58.090 INFO:teuthology.orchestra.run.smithi044.stdout: "purged_snapdirs", 2024-04-20T01:47:58.090 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_deletes", 2024-04-20T01:47:58.090 INFO:teuthology.orchestra.run.smithi044.stdout: "sortbitwise" 2024-04-20T01:47:58.090 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:47:58.090 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_version": 5, 2024-04-20T01:47:58.090 INFO:teuthology.orchestra.run.smithi044.stdout: "full_ratio": 0.94999998807907104, 2024-04-20T01:47:58.091 INFO:teuthology.orchestra.run.smithi044.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-20T01:47:58.091 INFO:teuthology.orchestra.run.smithi044.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-20T01:47:58.091 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_snapshot": "", 2024-04-20T01:47:58.091 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_max": 3, 2024-04-20T01:47:58.091 INFO:teuthology.orchestra.run.smithi044.stdout: "max_osd": 16, 2024-04-20T01:47:58.091 INFO:teuthology.orchestra.run.smithi044.stdout: "require_min_compat_client": "luminous", 2024-04-20T01:47:58.091 INFO:teuthology.orchestra.run.smithi044.stdout: "min_compat_client": "luminous", 2024-04-20T01:47:58.091 INFO:teuthology.orchestra.run.smithi044.stdout: "require_osd_release": "reef", 2024-04-20T01:47:58.091 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_crimson": false, 2024-04-20T01:47:58.091 INFO:teuthology.orchestra.run.smithi044.stdout: "pools": [ 2024-04-20T01:47:58.091 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.091 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 1, 2024-04-20T01:47:58.091 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": ".mgr", 2024-04-20T01:47:58.093 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-04-20T01:47:15.195181+0000", 2024-04-20T01:47:58.093 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T01:47:58.093 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool", 2024-04-20T01:47:58.093 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 1, 2024-04-20T01:47:58.093 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 2, 2024-04-20T01:47:58.093 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 1, 2024-04-20T01:47:58.093 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 0, 2024-04-20T01:47:58.093 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-04-20T01:47:58.093 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-04-20T01:47:58.094 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-04-20T01:47:58.094 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-20T01:47:58.094 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-04-20T01:47:58.094 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-04-20T01:47:58.094 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 1, 2024-04-20T01:47:58.094 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 1, 2024-04-20T01:47:58.094 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 1, 2024-04-20T01:47:58.094 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 1, 2024-04-20T01:47:58.094 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 1, 2024-04-20T01:47:58.094 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-04-20T01:47:58.094 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-04-20T01:47:58.094 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-04-20T01:47:58.094 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-04-20T01:47:58.095 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-04-20T01:47:58.095 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-04-20T01:47:58.095 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-04-20T01:47:58.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.095 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "24", 2024-04-20T01:47:58.095 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-04-20T01:47:58.095 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-20T01:47:58.095 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-04-20T01:47:58.095 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-04-20T01:47:58.095 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-04-20T01:47:58.095 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 0, 2024-04-20T01:47:58.095 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 0, 2024-04-20T01:47:58.095 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-04-20T01:47:58.096 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-04-20T01:47:58.096 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-04-20T01:47:58.096 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-04-20T01:47:58.096 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-04-20T01:47:58.096 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-04-20T01:47:58.096 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-04-20T01:47:58.096 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-04-20T01:47:58.096 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-04-20T01:47:58.096 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-04-20T01:47:58.096 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-04-20T01:47:58.096 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-20T01:47:58.096 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-20T01:47:58.096 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-20T01:47:58.096 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-04-20T01:47:58.096 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-04-20T01:47:58.097 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "", 2024-04-20T01:47:58.097 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-04-20T01:47:58.097 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-04-20T01:47:58.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.097 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-04-20T01:47:58.097 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-04-20T01:47:58.097 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-04-20T01:47:58.097 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-04-20T01:47:58.097 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-04-20T01:47:58.097 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-04-20T01:47:58.098 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-04-20T01:47:58.098 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-04-20T01:47:58.098 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 0, 2024-04-20T01:47:58.098 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-04-20T01:47:58.098 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-04-20T01:47:58.098 INFO:teuthology.orchestra.run.smithi044.stdout: "options": { 2024-04-20T01:47:58.098 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_max": 32, 2024-04-20T01:47:58.098 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_min": 1 2024-04-20T01:47:58.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.098 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-04-20T01:47:58.098 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": {} 2024-04-20T01:47:58.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.098 INFO:teuthology.orchestra.run.smithi044.stdout: "read_balance": { 2024-04-20T01:47:58.098 INFO:teuthology.orchestra.run.smithi044.stdout: "score_acting": 15.380000114440918, 2024-04-20T01:47:58.099 INFO:teuthology.orchestra.run.smithi044.stdout: "score_stable": 15.380000114440918, 2024-04-20T01:47:58.099 INFO:teuthology.orchestra.run.smithi044.stdout: "optimal_score": 0.12999999523162842, 2024-04-20T01:47:58.099 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_acting": 2, 2024-04-20T01:47:58.099 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_stable": 2, 2024-04-20T01:47:58.099 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity_weighted": 1, 2024-04-20T01:47:58.099 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity": 1, 2024-04-20T01:47:58.099 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity_weighted": 1 2024-04-20T01:47:58.099 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.099 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.099 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 2, 2024-04-20T01:47:58.099 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": "rbd", 2024-04-20T01:47:58.099 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-04-20T01:47:18.448004+0000", 2024-04-20T01:47:58.099 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T01:47:58.100 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool", 2024-04-20T01:47:58.100 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 1, 2024-04-20T01:47:58.100 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 2, 2024-04-20T01:47:58.100 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 1, 2024-04-20T01:47:58.100 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 0, 2024-04-20T01:47:58.100 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-04-20T01:47:58.100 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-04-20T01:47:58.100 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-04-20T01:47:58.100 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-20T01:47:58.100 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-04-20T01:47:58.100 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-04-20T01:47:58.100 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 8, 2024-04-20T01:47:58.100 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 8, 2024-04-20T01:47:58.101 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 8, 2024-04-20T01:47:58.101 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 8, 2024-04-20T01:47:58.101 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 8, 2024-04-20T01:47:58.101 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-04-20T01:47:58.101 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-04-20T01:47:58.101 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-04-20T01:47:58.101 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-04-20T01:47:58.101 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-04-20T01:47:58.101 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-04-20T01:47:58.101 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-04-20T01:47:58.101 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.101 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "27", 2024-04-20T01:47:58.101 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-04-20T01:47:58.102 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-20T01:47:58.102 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-04-20T01:47:58.102 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-04-20T01:47:58.102 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-04-20T01:47:58.102 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 0, 2024-04-20T01:47:58.102 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 0, 2024-04-20T01:47:58.102 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-04-20T01:47:58.102 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-04-20T01:47:58.102 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-04-20T01:47:58.102 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-04-20T01:47:58.102 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-04-20T01:47:58.102 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-04-20T01:47:58.103 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-04-20T01:47:58.103 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-04-20T01:47:58.103 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-04-20T01:47:58.103 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-04-20T01:47:58.103 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-04-20T01:47:58.103 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-20T01:47:58.103 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-20T01:47:58.103 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-20T01:47:58.103 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-04-20T01:47:58.103 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-04-20T01:47:58.103 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "", 2024-04-20T01:47:58.103 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-04-20T01:47:58.103 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-04-20T01:47:58.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.104 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-04-20T01:47:58.104 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-04-20T01:47:58.104 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-04-20T01:47:58.104 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-04-20T01:47:58.104 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-04-20T01:47:58.104 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-04-20T01:47:58.104 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-04-20T01:47:58.104 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-04-20T01:47:58.104 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 0, 2024-04-20T01:47:58.104 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-04-20T01:47:58.104 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-04-20T01:47:58.105 INFO:teuthology.orchestra.run.smithi044.stdout: "options": {}, 2024-04-20T01:47:58.105 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-04-20T01:47:58.105 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd": {} 2024-04-20T01:47:58.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.105 INFO:teuthology.orchestra.run.smithi044.stdout: "read_balance": { 2024-04-20T01:47:58.105 INFO:teuthology.orchestra.run.smithi044.stdout: "score_acting": 3.9900000095367432, 2024-04-20T01:47:58.105 INFO:teuthology.orchestra.run.smithi044.stdout: "score_stable": 3.9900000095367432, 2024-04-20T01:47:58.105 INFO:teuthology.orchestra.run.smithi044.stdout: "optimal_score": 0.68999999761581421, 2024-04-20T01:47:58.105 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_acting": 2.75, 2024-04-20T01:47:58.105 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_stable": 2.75, 2024-04-20T01:47:58.105 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity_weighted": 1, 2024-04-20T01:47:58.105 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity": 1, 2024-04-20T01:47:58.105 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity_weighted": 1 2024-04-20T01:47:58.105 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.106 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.106 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.106 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 3, 2024-04-20T01:47:58.106 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": "unique_pool_0", 2024-04-20T01:47:58.106 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-04-20T01:47:30.734962+0000", 2024-04-20T01:47:58.106 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 8193, 2024-04-20T01:47:58.106 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-04-20T01:47:58.106 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 3, 2024-04-20T01:47:58.106 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 7, 2024-04-20T01:47:58.106 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 5, 2024-04-20T01:47:58.106 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 1, 2024-04-20T01:47:58.106 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-04-20T01:47:58.107 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-04-20T01:47:58.107 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-04-20T01:47:58.107 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-20T01:47:58.107 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-04-20T01:47:58.107 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-04-20T01:47:58.107 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 16, 2024-04-20T01:47:58.107 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 16, 2024-04-20T01:47:58.107 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 16, 2024-04-20T01:47:58.107 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 16, 2024-04-20T01:47:58.107 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 16, 2024-04-20T01:47:58.107 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-04-20T01:47:58.107 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-04-20T01:47:58.107 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-04-20T01:47:58.107 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-04-20T01:47:58.108 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-04-20T01:47:58.108 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-04-20T01:47:58.108 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-04-20T01:47:58.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.108 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "56", 2024-04-20T01:47:58.108 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-04-20T01:47:58.108 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-20T01:47:58.108 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-04-20T01:47:58.108 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-04-20T01:47:58.108 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-04-20T01:47:58.110 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 24, 2024-04-20T01:47:58.110 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 56, 2024-04-20T01:47:58.110 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-04-20T01:47:58.110 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-04-20T01:47:58.110 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-04-20T01:47:58.110 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-04-20T01:47:58.110 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-04-20T01:47:58.110 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-04-20T01:47:58.110 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-04-20T01:47:58.110 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-04-20T01:47:58.110 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-04-20T01:47:58.111 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-04-20T01:47:58.111 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-04-20T01:47:58.111 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-20T01:47:58.111 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-20T01:47:58.111 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-20T01:47:58.111 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-04-20T01:47:58.111 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-04-20T01:47:58.111 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "shecprofile", 2024-04-20T01:47:58.111 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-04-20T01:47:58.111 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-04-20T01:47:58.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.111 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-04-20T01:47:58.111 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-04-20T01:47:58.111 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-04-20T01:47:58.112 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-04-20T01:47:58.112 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-04-20T01:47:58.112 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-04-20T01:47:58.112 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-04-20T01:47:58.112 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-04-20T01:47:58.112 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 16384, 2024-04-20T01:47:58.112 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-04-20T01:47:58.112 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-04-20T01:47:58.112 INFO:teuthology.orchestra.run.smithi044.stdout: "options": {}, 2024-04-20T01:47:58.112 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-04-20T01:47:58.112 INFO:teuthology.orchestra.run.smithi044.stdout: "rados": {} 2024-04-20T01:47:58.112 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.112 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.112 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:47:58.113 INFO:teuthology.orchestra.run.smithi044.stdout: "osds": [ 2024-04-20T01:47:58.113 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.113 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 0, 2024-04-20T01:47:58.113 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "3903b88f-ff2b-4252-9480-92b701b1e59b", 2024-04-20T01:47:58.113 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:58.113 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:58.113 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:58.113 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:58.113 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:58.113 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:58.113 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:58.113 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:58.113 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:58.113 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:58.114 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:58.114 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.114 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.114 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6824", 2024-04-20T01:47:58.114 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:47:58.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.114 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.114 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.114 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6825", 2024-04-20T01:47:58.114 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:47:58.114 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.114 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.114 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.115 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:58.115 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.115 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.115 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6826", 2024-04-20T01:47:58.115 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:47:58.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.115 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.115 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.115 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6827", 2024-04-20T01:47:58.115 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:47:58.115 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.115 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.116 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:58.116 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.116 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.116 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6830", 2024-04-20T01:47:58.116 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:47:58.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.116 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.116 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.116 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6831", 2024-04-20T01:47:58.116 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:47:58.116 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.116 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.116 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.117 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:58.117 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.117 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.117 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6828", 2024-04-20T01:47:58.117 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:47:58.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.117 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.117 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.117 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6829", 2024-04-20T01:47:58.117 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:47:58.117 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.117 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.118 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6825/43518", 2024-04-20T01:47:58.118 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6827/43518", 2024-04-20T01:47:58.118 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6831/43518", 2024-04-20T01:47:58.118 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6829/43518", 2024-04-20T01:47:58.118 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:58.119 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:58.119 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:58.119 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.120 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.120 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.120 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 1, 2024-04-20T01:47:58.120 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "c5ff0175-b4f0-40a5-86c7-7a0a351bf070", 2024-04-20T01:47:58.120 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:58.120 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:58.120 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:58.120 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:58.120 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:58.120 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:58.121 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:58.121 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 0, 2024-04-20T01:47:58.121 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:58.121 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:58.121 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:58.121 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.121 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.121 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6816", 2024-04-20T01:47:58.121 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:47:58.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.121 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.121 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6817", 2024-04-20T01:47:58.121 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:47:58.122 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.122 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.122 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:58.122 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.122 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.122 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6818", 2024-04-20T01:47:58.122 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:47:58.122 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.122 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.122 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.122 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6819", 2024-04-20T01:47:58.122 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:47:58.122 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.123 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.123 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:58.123 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.123 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.123 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6822", 2024-04-20T01:47:58.123 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:47:58.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.123 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.123 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.123 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6823", 2024-04-20T01:47:58.123 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:47:58.123 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.123 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.124 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:58.124 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.124 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.124 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6820", 2024-04-20T01:47:58.124 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:47:58.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.124 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.124 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.124 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6821", 2024-04-20T01:47:58.124 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:47:58.124 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.124 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.124 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.124 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.121:6817/74361", 2024-04-20T01:47:58.125 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.121:6819/74361", 2024-04-20T01:47:58.125 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.121:6823/74361", 2024-04-20T01:47:58.125 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.121:6821/74361", 2024-04-20T01:47:58.125 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:58.125 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:58.125 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:58.125 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.125 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.125 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 2, 2024-04-20T01:47:58.125 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "7af325c1-36b5-420c-a2a9-44c00a002977", 2024-04-20T01:47:58.125 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:58.125 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:58.125 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:58.125 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:58.126 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:58.126 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:58.126 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:58.126 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:58.126 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:58.126 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:58.126 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:58.126 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.126 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.126 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6809", 2024-04-20T01:47:58.126 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:47:58.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.126 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.127 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.127 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6811", 2024-04-20T01:47:58.127 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:47:58.127 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.127 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.127 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:58.127 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.127 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.127 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6814", 2024-04-20T01:47:58.127 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:47:58.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.127 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.127 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.127 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6816", 2024-04-20T01:47:58.128 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:47:58.128 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.128 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.128 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:58.128 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.128 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.128 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6822", 2024-04-20T01:47:58.128 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:47:58.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.128 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.128 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.128 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6823", 2024-04-20T01:47:58.128 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:47:58.129 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.129 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.129 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:58.129 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.129 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.129 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6819", 2024-04-20T01:47:58.129 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:47:58.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.129 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.129 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.129 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6821", 2024-04-20T01:47:58.129 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:47:58.129 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.130 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.130 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.123:6811/43108", 2024-04-20T01:47:58.130 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.123:6816/43108", 2024-04-20T01:47:58.130 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.123:6823/43108", 2024-04-20T01:47:58.130 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.123:6821/43108", 2024-04-20T01:47:58.130 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:58.130 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:58.130 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:58.130 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.130 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.130 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 3, 2024-04-20T01:47:58.130 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "99334f43-83ea-4c3f-b91d-ae72c8361b8c", 2024-04-20T01:47:58.131 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:58.131 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:58.131 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:58.131 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:58.131 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:58.131 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:58.131 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:58.131 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:58.131 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:58.131 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:58.131 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:58.131 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.131 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.131 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.132 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6824", 2024-04-20T01:47:58.132 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:47:58.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.132 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.132 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6825", 2024-04-20T01:47:58.132 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:47:58.132 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.132 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.132 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:58.132 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.132 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.132 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.133 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6826", 2024-04-20T01:47:58.133 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:47:58.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.133 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.133 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6827", 2024-04-20T01:47:58.133 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:47:58.133 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.133 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.133 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:58.133 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.133 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.134 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6830", 2024-04-20T01:47:58.134 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:47:58.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.134 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.134 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6831", 2024-04-20T01:47:58.134 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:47:58.134 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.134 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.134 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:58.134 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.134 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.134 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.134 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6828", 2024-04-20T01:47:58.135 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:47:58.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.135 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.135 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.135 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6829", 2024-04-20T01:47:58.135 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:47:58.135 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.135 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.135 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.165:6825/74545", 2024-04-20T01:47:58.135 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.165:6827/74545", 2024-04-20T01:47:58.135 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.165:6831/74545", 2024-04-20T01:47:58.135 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.165:6829/74545", 2024-04-20T01:47:58.135 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:58.135 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:58.136 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:58.136 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.136 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.136 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 4, 2024-04-20T01:47:58.136 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "8764d799-af67-4e6c-889f-9fdd91201cc2", 2024-04-20T01:47:58.136 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:58.136 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:58.136 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:58.136 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:58.136 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:58.136 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:58.136 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:58.136 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:58.136 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:58.137 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:58.137 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:58.137 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.137 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.137 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6809", 2024-04-20T01:47:58.137 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:47:58.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.137 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.137 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.137 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6811", 2024-04-20T01:47:58.137 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:47:58.137 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.137 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.138 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:58.138 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.138 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.138 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6813", 2024-04-20T01:47:58.138 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:47:58.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.138 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.138 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6815", 2024-04-20T01:47:58.138 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:47:58.138 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.138 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.139 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:58.139 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.139 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.139 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6821", 2024-04-20T01:47:58.139 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:47:58.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.139 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.139 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.139 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6823", 2024-04-20T01:47:58.139 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:47:58.139 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.139 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.139 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:58.140 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.140 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.140 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6817", 2024-04-20T01:47:58.140 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:47:58.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.140 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.140 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6819", 2024-04-20T01:47:58.140 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:47:58.140 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.140 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.140 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6811/43517", 2024-04-20T01:47:58.140 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6815/43517", 2024-04-20T01:47:58.141 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6823/43517", 2024-04-20T01:47:58.141 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6819/43517", 2024-04-20T01:47:58.141 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:58.141 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:58.141 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:58.141 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.141 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 5, 2024-04-20T01:47:58.141 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "2556b2fa-4dcc-479c-a379-6f9fe96e9c77", 2024-04-20T01:47:58.141 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:58.141 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:58.141 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:58.141 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:58.141 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:58.142 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:58.142 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:58.142 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:58.142 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:58.142 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:58.142 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:58.142 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.142 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.142 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6800", 2024-04-20T01:47:58.142 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:47:58.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.142 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.142 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6801", 2024-04-20T01:47:58.143 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:47:58.143 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.143 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.143 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:58.143 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.143 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.143 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6802", 2024-04-20T01:47:58.143 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:47:58.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.143 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.143 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.143 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6803", 2024-04-20T01:47:58.143 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:47:58.144 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.144 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.144 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:58.144 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.144 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.144 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6806", 2024-04-20T01:47:58.144 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:47:58.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.144 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.144 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6807", 2024-04-20T01:47:58.144 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:47:58.145 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.145 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.145 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:58.145 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.145 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.145 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6804", 2024-04-20T01:47:58.145 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:47:58.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.145 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.145 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.145 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6805", 2024-04-20T01:47:58.145 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:47:58.145 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.145 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.146 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.121:6801/74354", 2024-04-20T01:47:58.146 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.121:6803/74354", 2024-04-20T01:47:58.146 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.121:6807/74354", 2024-04-20T01:47:58.146 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.121:6805/74354", 2024-04-20T01:47:58.146 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:58.146 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:58.146 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:58.146 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.146 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 6, 2024-04-20T01:47:58.146 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "baeaae3a-fff4-4ccc-a08a-ff32db4531ba", 2024-04-20T01:47:58.146 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:58.147 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:58.147 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:58.147 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:58.147 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:58.147 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:58.147 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:58.147 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:58.147 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:58.147 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:58.147 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:58.147 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.147 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.147 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.147 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6800", 2024-04-20T01:47:58.148 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:47:58.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.148 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.148 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6801", 2024-04-20T01:47:58.148 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:47:58.148 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.148 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.148 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:58.148 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.148 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.148 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.148 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6802", 2024-04-20T01:47:58.148 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:47:58.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.149 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.149 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6803", 2024-04-20T01:47:58.149 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:47:58.149 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.149 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.149 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:58.149 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.149 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.149 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6806", 2024-04-20T01:47:58.149 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:47:58.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.150 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.150 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6807", 2024-04-20T01:47:58.150 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:47:58.150 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.150 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.150 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:58.150 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.150 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.150 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6804", 2024-04-20T01:47:58.150 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:47:58.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.150 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.151 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.151 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6805", 2024-04-20T01:47:58.151 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:47:58.151 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.151 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.151 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.123:6801/43107", 2024-04-20T01:47:58.151 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.123:6803/43107", 2024-04-20T01:47:58.151 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.123:6807/43107", 2024-04-20T01:47:58.151 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.123:6805/43107", 2024-04-20T01:47:58.151 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:58.151 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:58.151 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:58.151 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.152 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 7, 2024-04-20T01:47:58.152 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "0019f005-3e62-4d9d-b334-f1d347a484ce", 2024-04-20T01:47:58.152 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:58.152 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:58.152 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:58.152 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:58.152 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:58.152 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:58.152 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:58.152 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:58.152 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:58.152 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:58.152 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:58.152 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.153 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.153 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6800", 2024-04-20T01:47:58.153 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:47:58.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.153 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.153 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6801", 2024-04-20T01:47:58.153 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:47:58.153 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.153 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.153 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:58.153 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.154 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.154 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6802", 2024-04-20T01:47:58.154 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:47:58.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.154 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.154 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6803", 2024-04-20T01:47:58.154 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:47:58.154 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.154 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.154 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:58.154 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.155 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.155 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6806", 2024-04-20T01:47:58.155 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:47:58.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.155 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.155 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6807", 2024-04-20T01:47:58.155 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:47:58.155 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.155 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.155 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:58.155 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.155 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.155 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6804", 2024-04-20T01:47:58.156 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:47:58.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.156 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.156 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6805", 2024-04-20T01:47:58.156 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:47:58.156 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.156 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.156 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.165:6801/74543", 2024-04-20T01:47:58.156 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.165:6803/74543", 2024-04-20T01:47:58.156 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.165:6807/74543", 2024-04-20T01:47:58.156 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.165:6805/74543", 2024-04-20T01:47:58.156 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:58.156 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:58.157 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:58.157 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.157 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 8, 2024-04-20T01:47:58.157 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "6f5200a3-8980-4241-a6fc-ef604fa612b2", 2024-04-20T01:47:58.157 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:58.157 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:58.157 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:58.157 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:58.157 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:58.157 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:58.157 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:58.157 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 51, 2024-04-20T01:47:58.157 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:58.158 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:58.158 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:58.158 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.158 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.158 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6808", 2024-04-20T01:47:58.158 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:47:58.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.158 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.158 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6810", 2024-04-20T01:47:58.158 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:47:58.158 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.158 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.159 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:58.159 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.159 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.159 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6812", 2024-04-20T01:47:58.159 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:47:58.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.159 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.159 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6814", 2024-04-20T01:47:58.159 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:47:58.159 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.159 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.159 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:58.160 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.160 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.160 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6820", 2024-04-20T01:47:58.160 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:47:58.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.160 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.160 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6822", 2024-04-20T01:47:58.160 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:47:58.160 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.160 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.160 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:58.160 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.161 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.161 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6816", 2024-04-20T01:47:58.161 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:47:58.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.161 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.161 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6818", 2024-04-20T01:47:58.161 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:47:58.161 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.161 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.161 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6810/43515", 2024-04-20T01:47:58.161 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6814/43515", 2024-04-20T01:47:58.161 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6822/43515", 2024-04-20T01:47:58.161 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6818/43515", 2024-04-20T01:47:58.162 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:58.162 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:58.162 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:58.162 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.162 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 9, 2024-04-20T01:47:58.162 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "29cf9aaa-3a99-4605-92f4-64f1f6a1728d", 2024-04-20T01:47:58.162 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:58.162 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:58.162 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:58.162 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:58.162 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:58.162 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:58.163 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:58.163 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 0, 2024-04-20T01:47:58.163 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:58.163 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:58.163 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:58.163 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.163 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.163 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6824", 2024-04-20T01:47:58.163 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:47:58.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.163 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.163 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6825", 2024-04-20T01:47:58.163 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:47:58.164 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.164 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.164 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:58.164 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.164 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.164 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6826", 2024-04-20T01:47:58.164 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:47:58.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.164 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.164 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6827", 2024-04-20T01:47:58.164 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:47:58.164 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.165 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.165 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:58.165 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.165 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.165 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6830", 2024-04-20T01:47:58.165 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:47:58.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.165 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.165 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6831", 2024-04-20T01:47:58.165 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:47:58.165 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.166 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.166 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:58.166 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.166 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.166 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6828", 2024-04-20T01:47:58.166 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:47:58.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.166 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.166 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6829", 2024-04-20T01:47:58.166 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:47:58.166 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.167 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.167 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.121:6825/74379", 2024-04-20T01:47:58.167 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.121:6827/74379", 2024-04-20T01:47:58.167 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.121:6831/74379", 2024-04-20T01:47:58.167 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.121:6829/74379", 2024-04-20T01:47:58.167 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:58.167 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:58.167 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:58.167 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.167 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 10, 2024-04-20T01:47:58.167 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "705906c4-1514-4754-ac40-45226a837180", 2024-04-20T01:47:58.167 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:58.168 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:58.168 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:58.168 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:58.168 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:58.168 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:58.168 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:58.168 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:58.168 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:58.168 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:58.168 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:58.168 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.168 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.168 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6808", 2024-04-20T01:47:58.168 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:47:58.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.169 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.169 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6810", 2024-04-20T01:47:58.169 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:47:58.169 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.169 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.169 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:58.169 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.169 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.169 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6812", 2024-04-20T01:47:58.169 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:47:58.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.170 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.170 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6813", 2024-04-20T01:47:58.170 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:47:58.170 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.170 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.170 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:58.170 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.170 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.170 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6818", 2024-04-20T01:47:58.170 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:47:58.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.171 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.171 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6820", 2024-04-20T01:47:58.171 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:47:58.171 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.171 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.171 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:58.171 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.171 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.171 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6815", 2024-04-20T01:47:58.171 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:47:58.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.172 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.172 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6817", 2024-04-20T01:47:58.172 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:47:58.172 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.172 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.172 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.123:6810/43109", 2024-04-20T01:47:58.172 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.123:6813/43109", 2024-04-20T01:47:58.172 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.123:6820/43109", 2024-04-20T01:47:58.172 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.123:6817/43109", 2024-04-20T01:47:58.172 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:58.172 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:58.172 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:58.172 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.173 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 11, 2024-04-20T01:47:58.173 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "b45f2645-72d1-4d72-89e9-2212f5aa074c", 2024-04-20T01:47:58.173 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:58.173 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:58.173 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:58.173 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:58.173 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:58.173 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:58.173 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:58.173 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:58.173 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:58.173 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:58.173 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:58.173 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.174 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.174 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6808", 2024-04-20T01:47:58.174 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:47:58.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.174 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.174 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6809", 2024-04-20T01:47:58.174 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:47:58.174 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.174 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.174 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:58.174 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.175 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.175 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6810", 2024-04-20T01:47:58.175 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:47:58.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.175 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.175 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6811", 2024-04-20T01:47:58.175 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:47:58.175 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.175 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.175 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:58.175 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.176 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.176 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6814", 2024-04-20T01:47:58.176 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:47:58.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.176 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.176 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6815", 2024-04-20T01:47:58.176 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:47:58.176 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.176 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.176 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:58.176 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.176 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.177 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6812", 2024-04-20T01:47:58.177 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:47:58.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.177 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.177 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6813", 2024-04-20T01:47:58.177 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:47:58.177 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.177 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.177 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.165:6809/74544", 2024-04-20T01:47:58.177 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.165:6811/74544", 2024-04-20T01:47:58.177 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.165:6815/74544", 2024-04-20T01:47:58.177 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.165:6813/74544", 2024-04-20T01:47:58.177 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:58.178 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:58.178 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:58.178 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.178 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 12, 2024-04-20T01:47:58.178 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "3ea410b9-a368-467b-a179-2e3aba562d63", 2024-04-20T01:47:58.178 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:58.178 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:58.178 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:58.178 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:58.178 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:58.178 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:58.178 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:58.179 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 0, 2024-04-20T01:47:58.179 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:58.179 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:58.179 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:58.179 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.179 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.179 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6800", 2024-04-20T01:47:58.179 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:47:58.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.179 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.179 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6801", 2024-04-20T01:47:58.179 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:47:58.179 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.179 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.180 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:58.180 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.180 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.180 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6802", 2024-04-20T01:47:58.180 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:47:58.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.180 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.180 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6803", 2024-04-20T01:47:58.180 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:47:58.180 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.180 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.181 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:58.181 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.181 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.181 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6806", 2024-04-20T01:47:58.181 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:47:58.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.181 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.181 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6807", 2024-04-20T01:47:58.181 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:47:58.181 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.181 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.181 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:58.182 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.182 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.182 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6804", 2024-04-20T01:47:58.182 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:47:58.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.182 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.182 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6805", 2024-04-20T01:47:58.182 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:47:58.182 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.182 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.182 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6801/43516", 2024-04-20T01:47:58.183 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6803/43516", 2024-04-20T01:47:58.183 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6807/43516", 2024-04-20T01:47:58.183 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6805/43516", 2024-04-20T01:47:58.183 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:58.183 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:58.183 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:58.183 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.183 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 13, 2024-04-20T01:47:58.183 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "15a5154b-e7cb-4b57-8468-dcb048d93f63", 2024-04-20T01:47:58.183 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:58.183 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:58.183 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:58.183 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:58.184 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:58.184 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:58.184 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 22, 2024-04-20T01:47:58.184 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:58.184 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:58.184 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:58.184 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:58.184 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.184 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.184 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6808", 2024-04-20T01:47:58.184 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:47:58.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.184 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.185 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6809", 2024-04-20T01:47:58.185 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:47:58.185 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.185 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.185 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:58.185 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.185 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.185 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6810", 2024-04-20T01:47:58.185 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:47:58.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.185 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.185 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6811", 2024-04-20T01:47:58.186 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:47:58.186 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.186 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.186 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:58.186 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.186 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.186 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6814", 2024-04-20T01:47:58.186 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:47:58.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.186 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.186 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6815", 2024-04-20T01:47:58.187 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:47:58.187 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.187 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.187 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:58.187 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.187 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.187 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6812", 2024-04-20T01:47:58.187 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:47:58.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.187 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.187 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6813", 2024-04-20T01:47:58.187 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:47:58.188 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.188 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.188 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.121:6809/74355", 2024-04-20T01:47:58.188 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.121:6811/74355", 2024-04-20T01:47:58.188 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.121:6815/74355", 2024-04-20T01:47:58.188 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.121:6813/74355", 2024-04-20T01:47:58.188 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:58.188 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:58.188 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:58.188 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.188 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 14, 2024-04-20T01:47:58.188 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "23b53da2-f848-4e76-9632-4d9effd5b430", 2024-04-20T01:47:58.189 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:58.189 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:58.189 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:58.189 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:58.189 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:58.189 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:58.189 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:58.189 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:47:58.189 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:58.189 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:58.189 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:58.189 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.189 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.189 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6824", 2024-04-20T01:47:58.190 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:47:58.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.190 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.190 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6825", 2024-04-20T01:47:58.190 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:47:58.190 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.190 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.190 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:58.190 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.190 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.190 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6826", 2024-04-20T01:47:58.191 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:47:58.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.191 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.191 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6827", 2024-04-20T01:47:58.191 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:47:58.191 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.191 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.191 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:58.191 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.191 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.191 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6830", 2024-04-20T01:47:58.191 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:47:58.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.192 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.192 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6831", 2024-04-20T01:47:58.192 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:47:58.192 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.192 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.192 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:58.192 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.192 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.192 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6828", 2024-04-20T01:47:58.192 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:47:58.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.193 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.193 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6829", 2024-04-20T01:47:58.193 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:47:58.193 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.193 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.193 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.123:6825/43110", 2024-04-20T01:47:58.193 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.123:6827/43110", 2024-04-20T01:47:58.193 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.123:6831/43110", 2024-04-20T01:47:58.193 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.123:6829/43110", 2024-04-20T01:47:58.193 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:58.193 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:58.193 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:58.193 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.194 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 15, 2024-04-20T01:47:58.194 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "546efc0f-82a3-47d2-aeec-2d849eb1faa1", 2024-04-20T01:47:58.194 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:47:58.194 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:47:58.194 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:47:58.194 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:47:58.194 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:47:58.194 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:47:58.194 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:47:58.194 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 0, 2024-04-20T01:47:58.194 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:47:58.194 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:47:58.194 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:47:58.194 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.195 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.195 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6816", 2024-04-20T01:47:58.195 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:47:58.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.195 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.195 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6817", 2024-04-20T01:47:58.195 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:47:58.195 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.195 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.195 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:47:58.195 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.196 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.196 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6818", 2024-04-20T01:47:58.196 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:47:58.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.196 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.196 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6819", 2024-04-20T01:47:58.196 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:47:58.196 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.196 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.196 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:47:58.196 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.196 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.197 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6822", 2024-04-20T01:47:58.197 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:47:58.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.197 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.197 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6823", 2024-04-20T01:47:58.197 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:47:58.197 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.197 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.197 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:47:58.197 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:47:58.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.197 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:47:58.197 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6820", 2024-04-20T01:47:58.198 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:47:58.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.198 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:47:58.198 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6821", 2024-04-20T01:47:58.198 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:47:58.198 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.198 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.198 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.165:6817/74562", 2024-04-20T01:47:58.198 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.165:6819/74562", 2024-04-20T01:47:58.198 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.165:6823/74562", 2024-04-20T01:47:58.198 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.165:6821/74562", 2024-04-20T01:47:58.198 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:47:58.199 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:47:58.199 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:47:58.199 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.199 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.199 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:47:58.199 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_xinfo": [ 2024-04-20T01:47:58.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.199 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 0, 2024-04-20T01:47:58.199 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:58.199 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:58.199 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:58.199 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:58.199 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:58.199 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.124928+0000", 2024-04-20T01:47:58.199 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:58.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.200 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 1, 2024-04-20T01:47:58.200 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:58.200 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:58.200 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:58.200 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:58.200 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:58.200 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.093711+0000", 2024-04-20T01:47:58.200 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:58.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.200 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 2, 2024-04-20T01:47:58.200 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:58.200 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.008326+0000", 2024-04-20T01:47:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 3, 2024-04-20T01:47:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.085397+0000", 2024-04-20T01:47:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:58.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.202 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 4, 2024-04-20T01:47:58.202 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:58.202 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:58.202 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:58.202 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:58.202 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:58.202 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.117767+0000", 2024-04-20T01:47:58.202 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:58.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.202 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 5, 2024-04-20T01:47:58.202 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:58.202 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:58.203 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:58.203 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:58.203 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:58.203 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:12.999445+0000", 2024-04-20T01:47:58.203 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:58.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.203 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 6, 2024-04-20T01:47:58.203 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:58.203 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:58.203 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:58.203 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:58.203 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:58.203 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.032367+0000", 2024-04-20T01:47:58.203 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:58.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.204 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 7, 2024-04-20T01:47:58.204 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:58.204 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:58.204 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:58.204 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:58.204 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:58.204 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.058776+0000", 2024-04-20T01:47:58.204 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:58.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.204 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 8, 2024-04-20T01:47:58.204 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:58.204 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:58.205 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:58.205 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:58.205 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:58.205 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.113013+0000", 2024-04-20T01:47:58.205 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:58.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.205 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 9, 2024-04-20T01:47:58.205 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:58.205 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:58.205 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:58.205 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:58.205 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:58.205 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.110714+0000", 2024-04-20T01:47:58.205 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:58.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.206 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 10, 2024-04-20T01:47:58.206 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:58.206 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:58.206 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:58.206 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:58.206 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:58.206 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.092533+0000", 2024-04-20T01:47:58.206 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:58.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.206 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 11, 2024-04-20T01:47:58.206 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.083135+0000", 2024-04-20T01:47:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 12, 2024-04-20T01:47:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.094847+0000", 2024-04-20T01:47:58.207 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:58.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.208 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 13, 2024-04-20T01:47:58.208 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:58.208 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:58.208 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:58.208 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:58.208 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:58.208 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.059856+0000", 2024-04-20T01:47:58.208 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:58.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.208 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 14, 2024-04-20T01:47:58.208 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:58.208 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:58.209 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:58.209 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:58.209 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:58.209 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.128995+0000", 2024-04-20T01:47:58.209 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:58.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.209 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 15, 2024-04-20T01:47:58.209 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:47:58.209 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:47:58.209 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:47:58.209 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:47:58.209 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:47:58.209 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.061883+0000", 2024-04-20T01:47:58.209 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:47:58.210 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.210 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:47:58.210 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap": [], 2024-04-20T01:47:58.210 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap_items": [ 2024-04-20T01:47:58.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.210 INFO:teuthology.orchestra.run.smithi044.stdout: "pgid": "3.4", 2024-04-20T01:47:58.210 INFO:teuthology.orchestra.run.smithi044.stdout: "mappings": [ 2024-04-20T01:47:58.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.210 INFO:teuthology.orchestra.run.smithi044.stdout: "from": 0, 2024-04-20T01:47:58.210 INFO:teuthology.orchestra.run.smithi044.stdout: "to": 10 2024-04-20T01:47:58.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.210 INFO:teuthology.orchestra.run.smithi044.stdout: "from": 15, 2024-04-20T01:47:58.210 INFO:teuthology.orchestra.run.smithi044.stdout: "to": 6 2024-04-20T01:47:58.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.211 INFO:teuthology.orchestra.run.smithi044.stdout: "from": 13, 2024-04-20T01:47:58.211 INFO:teuthology.orchestra.run.smithi044.stdout: "to": 7 2024-04-20T01:47:58.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.211 INFO:teuthology.orchestra.run.smithi044.stdout: "from": 4, 2024-04-20T01:47:58.211 INFO:teuthology.orchestra.run.smithi044.stdout: "to": 12 2024-04-20T01:47:58.211 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.211 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.211 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.211 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:47:58.211 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap_primaries": [], 2024-04-20T01:47:58.211 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_temp": [], 2024-04-20T01:47:58.211 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_temp": [], 2024-04-20T01:47:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: "blocklist": {}, 2024-04-20T01:47:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: "range_blocklist": {}, 2024-04-20T01:47:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profiles": { 2024-04-20T01:47:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: "default": { 2024-04-20T01:47:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-failure-domain": "osd", 2024-04-20T01:47:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: "k": "2", 2024-04-20T01:47:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: "m": "1", 2024-04-20T01:47:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: "plugin": "jerasure", 2024-04-20T01:47:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: "technique": "reed_sol_van" 2024-04-20T01:47:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: "shecprofile": { 2024-04-20T01:47:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: "c": "2", 2024-04-20T01:47:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-device-class": "", 2024-04-20T01:47:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-failure-domain": "osd", 2024-04-20T01:47:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-root": "default", 2024-04-20T01:47:58.212 INFO:teuthology.orchestra.run.smithi044.stdout: "k": "4", 2024-04-20T01:47:58.213 INFO:teuthology.orchestra.run.smithi044.stdout: "m": "3", 2024-04-20T01:47:58.213 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "shecprofile", 2024-04-20T01:47:58.213 INFO:teuthology.orchestra.run.smithi044.stdout: "plugin": "shec", 2024-04-20T01:47:58.213 INFO:teuthology.orchestra.run.smithi044.stdout: "technique": "multiple" 2024-04-20T01:47:58.213 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.213 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps_queue": [ 2024-04-20T01:47:58.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.213 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 3, 2024-04-20T01:47:58.213 INFO:teuthology.orchestra.run.smithi044.stdout: "snaps": [ 2024-04-20T01:47:58.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.213 INFO:teuthology.orchestra.run.smithi044.stdout: "begin": 10, 2024-04-20T01:47:58.213 INFO:teuthology.orchestra.run.smithi044.stdout: "length": 1 2024-04-20T01:47:58.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.214 INFO:teuthology.orchestra.run.smithi044.stdout: "begin": 12, 2024-04-20T01:47:58.214 INFO:teuthology.orchestra.run.smithi044.stdout: "length": 2 2024-04-20T01:47:58.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.214 INFO:teuthology.orchestra.run.smithi044.stdout: "begin": 17, 2024-04-20T01:47:58.214 INFO:teuthology.orchestra.run.smithi044.stdout: "length": 1 2024-04-20T01:47:58.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.214 INFO:teuthology.orchestra.run.smithi044.stdout: "begin": 19, 2024-04-20T01:47:58.214 INFO:teuthology.orchestra.run.smithi044.stdout: "length": 1 2024-04-20T01:47:58.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:47:58.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:47:58.214 INFO:teuthology.orchestra.run.smithi044.stdout: "begin": 21, 2024-04-20T01:47:58.214 INFO:teuthology.orchestra.run.smithi044.stdout: "length": 1 2024-04-20T01:47:58.215 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.215 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:47:58.215 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.215 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:47:58.215 INFO:teuthology.orchestra.run.smithi044.stdout: "new_removed_snaps": [], 2024-04-20T01:47:58.215 INFO:teuthology.orchestra.run.smithi044.stdout: "new_purged_snaps": [], 2024-04-20T01:47:58.215 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_node_flags": {}, 2024-04-20T01:47:58.215 INFO:teuthology.orchestra.run.smithi044.stdout: "device_class_flags": {}, 2024-04-20T01:47:58.215 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": { 2024-04-20T01:47:58.215 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode_enabled": false, 2024-04-20T01:47:58.215 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_bucket_count": 0, 2024-04-20T01:47:58.215 INFO:teuthology.orchestra.run.smithi044.stdout: "degraded_stretch_mode": 0, 2024-04-20T01:47:58.215 INFO:teuthology.orchestra.run.smithi044.stdout: "recovering_stretch_mode": 0, 2024-04-20T01:47:58.215 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode_bucket": 0 2024-04-20T01:47:58.215 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:47:58.215 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:47:58.218 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:58.218 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:58.219 INFO:tasks.thrashosds.thrasher:j is {'epoch': 56, 'fsid': '9c2f64c1-aa35-4337-9f5c-081b9e3a2938', 'created': '2024-04-20T01:47:02.934241+0000', 'modified': '2024-04-20T01:47:57.714018+0000', 'last_up_change': '2024-04-20T01:47:15.986754+0000', 'last_in_change': '2024-04-20T01:47:07.894719+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T01:47:15.195181+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 15.380000114440918, 'score_stable': 15.380000114440918, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-04-20T01:47:18.448004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 3.990000009536743, 'score_stable': 3.990000009536743, 'optimal_score': 0.6899999976158142, 'raw_score_acting': 2.75, 'raw_score_stable': 2.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-04-20T01:47:30.734962+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 7, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 24, 'snap_epoch': 56, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'shecprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '3903b88f-ff2b-4252-9480-92b701b1e59b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6824', 'nonce': 43518}, {'type': 'v1', 'addr': '172.21.15.44:6825', 'nonce': 43518}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6826', 'nonce': 43518}, {'type': 'v1', 'addr': '172.21.15.44:6827', 'nonce': 43518}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6830', 'nonce': 43518}, {'type': 'v1', 'addr': '172.21.15.44:6831', 'nonce': 43518}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6828', 'nonce': 43518}, {'type': 'v1', 'addr': '172.21.15.44:6829', 'nonce': 43518}]}, 'public_addr': '172.21.15.44:6825/43518', 'cluster_addr': '172.21.15.44:6827/43518', 'heartbeat_back_addr': '172.21.15.44:6831/43518', 'heartbeat_front_addr': '172.21.15.44:6829/43518', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c5ff0175-b4f0-40a5-86c7-7a0a351bf070', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6816', 'nonce': 74361}, {'type': 'v1', 'addr': '172.21.15.121:6817', 'nonce': 74361}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6818', 'nonce': 74361}, {'type': 'v1', 'addr': '172.21.15.121:6819', 'nonce': 74361}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6822', 'nonce': 74361}, {'type': 'v1', 'addr': '172.21.15.121:6823', 'nonce': 74361}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6820', 'nonce': 74361}, {'type': 'v1', 'addr': '172.21.15.121:6821', 'nonce': 74361}]}, 'public_addr': '172.21.15.121:6817/74361', 'cluster_addr': '172.21.15.121:6819/74361', 'heartbeat_back_addr': '172.21.15.121:6823/74361', 'heartbeat_front_addr': '172.21.15.121:6821/74361', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '7af325c1-36b5-420c-a2a9-44c00a002977', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6809', 'nonce': 43108}, {'type': 'v1', 'addr': '172.21.15.123:6811', 'nonce': 43108}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6814', 'nonce': 43108}, {'type': 'v1', 'addr': '172.21.15.123:6816', 'nonce': 43108}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6822', 'nonce': 43108}, {'type': 'v1', 'addr': '172.21.15.123:6823', 'nonce': 43108}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6819', 'nonce': 43108}, {'type': 'v1', 'addr': '172.21.15.123:6821', 'nonce': 43108}]}, 'public_addr': '172.21.15.123:6811/43108', 'cluster_addr': '172.21.15.123:6816/43108', 'heartbeat_back_addr': '172.21.15.123:6823/43108', 'heartbeat_front_addr': '172.21.15.123:6821/43108', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '99334f43-83ea-4c3f-b91d-ae72c8361b8c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6824', 'nonce': 74545}, {'type': 'v1', 'addr': '172.21.15.165:6825', 'nonce': 74545}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6826', 'nonce': 74545}, {'type': 'v1', 'addr': '172.21.15.165:6827', 'nonce': 74545}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6830', 'nonce': 74545}, {'type': 'v1', 'addr': '172.21.15.165:6831', 'nonce': 74545}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6828', 'nonce': 74545}, {'type': 'v1', 'addr': '172.21.15.165:6829', 'nonce': 74545}]}, 'public_addr': '172.21.15.165:6825/74545', 'cluster_addr': '172.21.15.165:6827/74545', 'heartbeat_back_addr': '172.21.15.165:6831/74545', 'heartbeat_front_addr': '172.21.15.165:6829/74545', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8764d799-af67-4e6c-889f-9fdd91201cc2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6809', 'nonce': 43517}, {'type': 'v1', 'addr': '172.21.15.44:6811', 'nonce': 43517}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6813', 'nonce': 43517}, {'type': 'v1', 'addr': '172.21.15.44:6815', 'nonce': 43517}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6821', 'nonce': 43517}, {'type': 'v1', 'addr': '172.21.15.44:6823', 'nonce': 43517}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6817', 'nonce': 43517}, {'type': 'v1', 'addr': '172.21.15.44:6819', 'nonce': 43517}]}, 'public_addr': '172.21.15.44:6811/43517', 'cluster_addr': '172.21.15.44:6815/43517', 'heartbeat_back_addr': '172.21.15.44:6823/43517', 'heartbeat_front_addr': '172.21.15.44:6819/43517', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '2556b2fa-4dcc-479c-a379-6f9fe96e9c77', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6800', 'nonce': 74354}, {'type': 'v1', 'addr': '172.21.15.121:6801', 'nonce': 74354}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6802', 'nonce': 74354}, {'type': 'v1', 'addr': '172.21.15.121:6803', 'nonce': 74354}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6806', 'nonce': 74354}, {'type': 'v1', 'addr': '172.21.15.121:6807', 'nonce': 74354}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6804', 'nonce': 74354}, {'type': 'v1', 'addr': '172.21.15.121:6805', 'nonce': 74354}]}, 'public_addr': '172.21.15.121:6801/74354', 'cluster_addr': '172.21.15.121:6803/74354', 'heartbeat_back_addr': '172.21.15.121:6807/74354', 'heartbeat_front_addr': '172.21.15.121:6805/74354', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'baeaae3a-fff4-4ccc-a08a-ff32db4531ba', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6800', 'nonce': 43107}, {'type': 'v1', 'addr': '172.21.15.123:6801', 'nonce': 43107}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6802', 'nonce': 43107}, {'type': 'v1', 'addr': '172.21.15.123:6803', 'nonce': 43107}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6806', 'nonce': 43107}, {'type': 'v1', 'addr': '172.21.15.123:6807', 'nonce': 43107}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6804', 'nonce': 43107}, {'type': 'v1', 'addr': '172.21.15.123:6805', 'nonce': 43107}]}, 'public_addr': '172.21.15.123:6801/43107', 'cluster_addr': '172.21.15.123:6803/43107', 'heartbeat_back_addr': '172.21.15.123:6807/43107', 'heartbeat_front_addr': '172.21.15.123:6805/43107', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '0019f005-3e62-4d9d-b334-f1d347a484ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6800', 'nonce': 74543}, {'type': 'v1', 'addr': '172.21.15.165:6801', 'nonce': 74543}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6802', 'nonce': 74543}, {'type': 'v1', 'addr': '172.21.15.165:6803', 'nonce': 74543}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6806', 'nonce': 74543}, {'type': 'v1', 'addr': '172.21.15.165:6807', 'nonce': 74543}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6804', 'nonce': 74543}, {'type': 'v1', 'addr': '172.21.15.165:6805', 'nonce': 74543}]}, 'public_addr': '172.21.15.165:6801/74543', 'cluster_addr': '172.21.15.165:6803/74543', 'heartbeat_back_addr': '172.21.15.165:6807/74543', 'heartbeat_front_addr': '172.21.15.165:6805/74543', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '6f5200a3-8980-4241-a6fc-ef604fa612b2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6808', 'nonce': 43515}, {'type': 'v1', 'addr': '172.21.15.44:6810', 'nonce': 43515}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6812', 'nonce': 43515}, {'type': 'v1', 'addr': '172.21.15.44:6814', 'nonce': 43515}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6820', 'nonce': 43515}, {'type': 'v1', 'addr': '172.21.15.44:6822', 'nonce': 43515}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6816', 'nonce': 43515}, {'type': 'v1', 'addr': '172.21.15.44:6818', 'nonce': 43515}]}, 'public_addr': '172.21.15.44:6810/43515', 'cluster_addr': '172.21.15.44:6814/43515', 'heartbeat_back_addr': '172.21.15.44:6822/43515', 'heartbeat_front_addr': '172.21.15.44:6818/43515', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '29cf9aaa-3a99-4605-92f4-64f1f6a1728d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6824', 'nonce': 74379}, {'type': 'v1', 'addr': '172.21.15.121:6825', 'nonce': 74379}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6826', 'nonce': 74379}, {'type': 'v1', 'addr': '172.21.15.121:6827', 'nonce': 74379}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6830', 'nonce': 74379}, {'type': 'v1', 'addr': '172.21.15.121:6831', 'nonce': 74379}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6828', 'nonce': 74379}, {'type': 'v1', 'addr': '172.21.15.121:6829', 'nonce': 74379}]}, 'public_addr': '172.21.15.121:6825/74379', 'cluster_addr': '172.21.15.121:6827/74379', 'heartbeat_back_addr': '172.21.15.121:6831/74379', 'heartbeat_front_addr': '172.21.15.121:6829/74379', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '705906c4-1514-4754-ac40-45226a837180', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6808', 'nonce': 43109}, {'type': 'v1', 'addr': '172.21.15.123:6810', 'nonce': 43109}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6812', 'nonce': 43109}, {'type': 'v1', 'addr': '172.21.15.123:6813', 'nonce': 43109}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6818', 'nonce': 43109}, {'type': 'v1', 'addr': '172.21.15.123:6820', 'nonce': 43109}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6815', 'nonce': 43109}, {'type': 'v1', 'addr': '172.21.15.123:6817', 'nonce': 43109}]}, 'public_addr': '172.21.15.123:6810/43109', 'cluster_addr': '172.21.15.123:6813/43109', 'heartbeat_back_addr': '172.21.15.123:6820/43109', 'heartbeat_front_addr': '172.21.15.123:6817/43109', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': 'b45f2645-72d1-4d72-89e9-2212f5aa074c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6808', 'nonce': 74544}, {'type': 'v1', 'addr': '172.21.15.165:6809', 'nonce': 74544}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6810', 'nonce': 74544}, {'type': 'v1', 'addr': '172.21.15.165:6811', 'nonce': 74544}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6814', 'nonce': 74544}, {'type': 'v1', 'addr': '172.21.15.165:6815', 'nonce': 74544}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6812', 'nonce': 74544}, {'type': 'v1', 'addr': '172.21.15.165:6813', 'nonce': 74544}]}, 'public_addr': '172.21.15.165:6809/74544', 'cluster_addr': '172.21.15.165:6811/74544', 'heartbeat_back_addr': '172.21.15.165:6815/74544', 'heartbeat_front_addr': '172.21.15.165:6813/74544', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': '3ea410b9-a368-467b-a179-2e3aba562d63', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6800', 'nonce': 43516}, {'type': 'v1', 'addr': '172.21.15.44:6801', 'nonce': 43516}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6802', 'nonce': 43516}, {'type': 'v1', 'addr': '172.21.15.44:6803', 'nonce': 43516}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6806', 'nonce': 43516}, {'type': 'v1', 'addr': '172.21.15.44:6807', 'nonce': 43516}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6804', 'nonce': 43516}, {'type': 'v1', 'addr': '172.21.15.44:6805', 'nonce': 43516}]}, 'public_addr': '172.21.15.44:6801/43516', 'cluster_addr': '172.21.15.44:6803/43516', 'heartbeat_back_addr': '172.21.15.44:6807/43516', 'heartbeat_front_addr': '172.21.15.44:6805/43516', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': '15a5154b-e7cb-4b57-8468-dcb048d93f63', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6808', 'nonce': 74355}, {'type': 'v1', 'addr': '172.21.15.121:6809', 'nonce': 74355}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6810', 'nonce': 74355}, {'type': 'v1', 'addr': '172.21.15.121:6811', 'nonce': 74355}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6814', 'nonce': 74355}, {'type': 'v1', 'addr': '172.21.15.121:6815', 'nonce': 74355}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6812', 'nonce': 74355}, {'type': 'v1', 'addr': '172.21.15.121:6813', 'nonce': 74355}]}, 'public_addr': '172.21.15.121:6809/74355', 'cluster_addr': '172.21.15.121:6811/74355', 'heartbeat_back_addr': '172.21.15.121:6815/74355', 'heartbeat_front_addr': '172.21.15.121:6813/74355', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '23b53da2-f848-4e76-9632-4d9effd5b430', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6824', 'nonce': 43110}, {'type': 'v1', 'addr': '172.21.15.123:6825', 'nonce': 43110}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6826', 'nonce': 43110}, {'type': 'v1', 'addr': '172.21.15.123:6827', 'nonce': 43110}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6830', 'nonce': 43110}, {'type': 'v1', 'addr': '172.21.15.123:6831', 'nonce': 43110}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6828', 'nonce': 43110}, {'type': 'v1', 'addr': '172.21.15.123:6829', 'nonce': 43110}]}, 'public_addr': '172.21.15.123:6825/43110', 'cluster_addr': '172.21.15.123:6827/43110', 'heartbeat_back_addr': '172.21.15.123:6831/43110', 'heartbeat_front_addr': '172.21.15.123:6829/43110', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': '546efc0f-82a3-47d2-aeec-2d849eb1faa1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6816', 'nonce': 74562}, {'type': 'v1', 'addr': '172.21.15.165:6817', 'nonce': 74562}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6818', 'nonce': 74562}, {'type': 'v1', 'addr': '172.21.15.165:6819', 'nonce': 74562}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6822', 'nonce': 74562}, {'type': 'v1', 'addr': '172.21.15.165:6823', 'nonce': 74562}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6820', 'nonce': 74562}, {'type': 'v1', 'addr': '172.21.15.165:6821', 'nonce': 74562}]}, 'public_addr': '172.21.15.165:6817/74562', 'cluster_addr': '172.21.15.165:6819/74562', 'heartbeat_back_addr': '172.21.15.165:6823/74562', 'heartbeat_front_addr': '172.21.15.165:6821/74562', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.124928+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.093711+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.008326+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.085397+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.117767+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:12.999445+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.032367+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.058776+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.113013+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.110714+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.092533+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.083135+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.094847+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.059856+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.128995+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.061883+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.4', 'mappings': [{'from': 0, 'to': 10}, {'from': 15, 'to': 6}, {'from': 13, 'to': 7}, {'from': 4, 'to': 12}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'shecprofile': {'c': '2', 'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'k': '4', 'm': '3', 'name': 'shecprofile', 'plugin': 'shec', 'technique': 'multiple'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 10, 'length': 1}, {'begin': 12, 'length': 2}, {'begin': 17, 'length': 1}, {'begin': 19, 'length': 1}, {'begin': 21, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-20T01:47:58.219 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.4 2024-04-20T01:47:58.219 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap-items 3.4 2024-04-20T01:47:58.219 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:47:58.231+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:47:58.220 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-20T01:47:58.318 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:47:58.332+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:47:58.347 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:58.347 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:58.357 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-20T01:47:58.419 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:58.433+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:47:58.519 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:47:58.534+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:47:58.525 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:58.525 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:58.535 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-20T01:47:58.620 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:58.635+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:47:58.664 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:58.664 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:58.674 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-20T01:47:58.709 INFO:tasks.rados.rados.0.smithi165.stdout:199: done (0 left) 2024-04-20T01:47:58.709 INFO:tasks.rados.rados.0.smithi165.stdout:200: snap_create 2024-04-20T01:47:58.715 INFO:teuthology.orchestra.run.smithi044.stderr:clear 3.4 pg_upmap_items mapping 2024-04-20T01:47:58.721 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-20T01:47:58.721 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:47:58.735+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:47:58.803 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:58.804 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:58.813 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-20T01:47:58.822 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:58.837+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:58.830 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-20T01:47:58.923 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:58.937+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:47:58.941 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:58.942 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:58.951 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-20T01:47:58.995 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:47:59.016 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:47:59.023 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:47:59.038+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:47:59.037 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:47:59.058 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:47:59.078 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:47:59.079 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:59.079 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:59.090 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-20T01:47:59.099 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:47:59.120 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:47:59.124 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:47:59.139+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:47:59.142 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:47:59.162 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:47:59.183 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:47:59.203 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:47:59.217 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:47:59.217 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:47:59.224 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:47:59.226 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:47:59.240+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:47:59.227 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-20T01:47:59.244 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:47:59.265 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:47:59.285 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:47:59.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:47:59.300+0000 7f2d9a855700 0 -- 172.21.15.44:0/984132389 >> [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] conn(0x7f2d94221680 msgr2=0x7f2d94223a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-20T01:47:59.326 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:47:59.341+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:47:59.396 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:47:59.397 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-20T01:47:59.397 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-20T01:47:59.397 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-20T01:47:59.397 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.397 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.5s3 32/30 ECSubWrite(tid=1, reqid=client.4643.0:11, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:59.397 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.754397+0000", 2024-04-20T01:47:59.397 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 24.655786113000001, 2024-04-20T01:47:59.397 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01012041, 2024-04-20T01:47:59.397 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:59.397 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:59.397 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:59.397 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.398 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:59.398 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754397+0000", 2024-04-20T01:47:59.398 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.398 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.398 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:59.398 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754397+0000", 2024-04-20T01:47:59.398 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.398 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.398 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:59.398 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754396+0000", 2024-04-20T01:47:59.398 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:59.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.398 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.399 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:59.399 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754423+0000", 2024-04-20T01:47:59.399 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7032000000000001e-05 2024-04-20T01:47:59.399 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.399 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.399 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:59.399 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754438+0000", 2024-04-20T01:47:59.399 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4946999999999999e-05 2024-04-20T01:47:59.399 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.399 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.399 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:59.399 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754447+0000", 2024-04-20T01:47:59.399 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8550000000000008e-06 2024-04-20T01:47:59.399 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.399 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.400 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.400 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754496+0000", 2024-04-20T01:47:59.400 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9018999999999999e-05 2024-04-20T01:47:59.400 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.400 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.400 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:59.400 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754506+0000", 2024-04-20T01:47:59.400 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7079999999999992e-06 2024-04-20T01:47:59.400 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.400 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.400 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:59.400 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764493+0000", 2024-04-20T01:47:59.400 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0099871460000000006 2024-04-20T01:47:59.400 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.400 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.401 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:59.401 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764517+0000", 2024-04-20T01:47:59.401 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4467000000000001e-05 2024-04-20T01:47:59.401 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.401 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.401 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.401 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.401 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.401 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.9s4 32/30 ECSubWrite(tid=2, reqid=client.4643.0:7, at_version=32'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:59.401 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.755700+0000", 2024-04-20T01:47:59.401 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 24.654482731999998, 2024-04-20T01:47:59.401 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0088510410000000005, 2024-04-20T01:47:59.401 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:59.401 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:59.401 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:59.402 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.402 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:59.402 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755700+0000", 2024-04-20T01:47:59.402 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.402 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.402 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.402 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:59.402 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755700+0000", 2024-04-20T01:47:59.402 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.402 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.402 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.402 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:59.402 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755700+0000", 2024-04-20T01:47:59.402 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:59.402 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.403 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.403 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:59.403 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755735+0000", 2024-04-20T01:47:59.403 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5524000000000001e-05 2024-04-20T01:47:59.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.403 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.403 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:59.403 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755750+0000", 2024-04-20T01:47:59.403 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4609e-05 2024-04-20T01:47:59.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.403 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.403 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:59.403 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755759+0000", 2024-04-20T01:47:59.403 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3700000000000001e-06 2024-04-20T01:47:59.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.404 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.404 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.404 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755799+0000", 2024-04-20T01:47:59.404 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9560000000000001e-05 2024-04-20T01:47:59.404 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.404 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.404 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:59.404 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755804+0000", 2024-04-20T01:47:59.404 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3149999999999999e-06 2024-04-20T01:47:59.404 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.404 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.404 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:59.404 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764532+0000", 2024-04-20T01:47:59.404 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087277770000000008 2024-04-20T01:47:59.404 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.405 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.405 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:59.405 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764551+0000", 2024-04-20T01:47:59.405 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9582e-05 2024-04-20T01:47:59.405 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.405 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.405 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.405 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.405 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.405 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.9s4 32/30 ECSubWrite(tid=3, reqid=client.4643.0:8, at_version=32'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:59.405 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.755948+0000", 2024-04-20T01:47:59.405 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 24.654235556, 2024-04-20T01:47:59.405 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0086164850000000001, 2024-04-20T01:47:59.405 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:59.405 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:59.405 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:59.406 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.406 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:59.406 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755948+0000", 2024-04-20T01:47:59.406 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.406 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.406 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:59.406 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755948+0000", 2024-04-20T01:47:59.406 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.406 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.406 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:59.406 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755947+0000", 2024-04-20T01:47:59.406 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:59.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.407 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.407 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:59.407 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755963+0000", 2024-04-20T01:47:59.407 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6013000000000001e-05 2024-04-20T01:47:59.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.407 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.407 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:59.407 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755975+0000", 2024-04-20T01:47:59.407 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2347000000000001e-05 2024-04-20T01:47:59.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.407 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.407 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:59.407 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755985+0000", 2024-04-20T01:47:59.407 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8730000000000002e-06 2024-04-20T01:47:59.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.408 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.408 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.408 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.756022+0000", 2024-04-20T01:47:59.408 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7039000000000002e-05 2024-04-20T01:47:59.408 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.408 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.408 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:59.408 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.756027+0000", 2024-04-20T01:47:59.408 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.006e-06 2024-04-20T01:47:59.408 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.408 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.408 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:59.408 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764554+0000", 2024-04-20T01:47:59.408 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0085269609999999996 2024-04-20T01:47:59.408 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.408 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.409 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:59.409 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764564+0000", 2024-04-20T01:47:59.409 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8740000000000007e-06 2024-04-20T01:47:59.409 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.409 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.409 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.409 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.409 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.409 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s3 33/30 ECSubWrite(tid=157, reqid=client.4643.0:16, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:47:59.409 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.757106+0000", 2024-04-20T01:47:59.409 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 24.653076873, 2024-04-20T01:47:59.409 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0086461160000000006, 2024-04-20T01:47:59.409 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:59.409 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:59.409 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:59.410 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.410 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:59.410 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757106+0000", 2024-04-20T01:47:59.410 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.410 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.410 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.410 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:59.410 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757106+0000", 2024-04-20T01:47:59.410 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.410 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.410 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.410 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:59.410 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757106+0000", 2024-04-20T01:47:59.410 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:59.410 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.411 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.411 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:59.411 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757173+0000", 2024-04-20T01:47:59.411 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6734000000000002e-05 2024-04-20T01:47:59.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.411 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.411 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:59.411 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757184+0000", 2024-04-20T01:47:59.411 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0896999999999999e-05 2024-04-20T01:47:59.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.411 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.411 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:59.411 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757192+0000", 2024-04-20T01:47:59.411 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4460000000000001e-06 2024-04-20T01:47:59.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.412 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.412 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.412 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757248+0000", 2024-04-20T01:47:59.412 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5949999999999998e-05 2024-04-20T01:47:59.412 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.412 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.412 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:59.412 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757257+0000", 2024-04-20T01:47:59.412 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.935e-06 2024-04-20T01:47:59.412 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.412 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.412 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:59.412 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.765715+0000", 2024-04-20T01:47:59.412 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0084578169999999994 2024-04-20T01:47:59.412 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.413 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.413 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:59.413 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.765752+0000", 2024-04-20T01:47:59.413 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7608000000000003e-05 2024-04-20T01:47:59.413 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.413 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.413 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.413 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.413 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.413 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s3 33/30 ECSubWrite(tid=168, reqid=client.4643.0:67, at_version=33'3, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:47:59.413 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.784794+0000", 2024-04-20T01:47:59.413 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 24.625389038000002, 2024-04-20T01:47:59.413 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011714966, 2024-04-20T01:47:59.413 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:59.413 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:59.413 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:59.414 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.414 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:59.414 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784794+0000", 2024-04-20T01:47:59.414 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.414 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.414 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.414 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:59.414 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784794+0000", 2024-04-20T01:47:59.414 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.414 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.414 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.414 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:59.414 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784794+0000", 2024-04-20T01:47:59.414 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:59.414 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.415 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.415 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:59.415 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784957+0000", 2024-04-20T01:47:59.415 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016303800000000001 2024-04-20T01:47:59.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.415 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.415 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:59.415 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784970+0000", 2024-04-20T01:47:59.415 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3305000000000001e-05 2024-04-20T01:47:59.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.415 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.415 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:59.415 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784979+0000", 2024-04-20T01:47:59.415 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7129999999999994e-06 2024-04-20T01:47:59.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.416 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.416 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.416 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785046+0000", 2024-04-20T01:47:59.416 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7108999999999997e-05 2024-04-20T01:47:59.416 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.416 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.416 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:59.416 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785055+0000", 2024-04-20T01:47:59.416 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.234e-06 2024-04-20T01:47:59.416 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.417 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.417 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:59.417 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796482+0000", 2024-04-20T01:47:59.417 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011427196000000001 2024-04-20T01:47:59.417 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.417 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.417 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:59.417 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796509+0000", 2024-04-20T01:47:59.417 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6999e-05 2024-04-20T01:47:59.417 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.417 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.417 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.417 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.418 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.418 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s3 33/30 ECSubWrite(tid=169, reqid=client.4643.0:68, at_version=33'4, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:47:59.418 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.785553+0000", 2024-04-20T01:47:59.418 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 24.624630200999999, 2024-04-20T01:47:59.418 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010979604, 2024-04-20T01:47:59.418 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:59.418 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:59.418 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:59.418 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.418 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:59.418 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785553+0000", 2024-04-20T01:47:59.419 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.419 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.419 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.419 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:59.419 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785553+0000", 2024-04-20T01:47:59.419 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.419 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.419 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.419 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:59.419 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785552+0000", 2024-04-20T01:47:59.419 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:59.419 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.420 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.420 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:59.420 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785579+0000", 2024-04-20T01:47:59.420 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6492999999999999e-05 2024-04-20T01:47:59.420 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.420 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.420 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:59.420 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785589+0000", 2024-04-20T01:47:59.420 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0353000000000001e-05 2024-04-20T01:47:59.420 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.420 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.420 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:59.420 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785598+0000", 2024-04-20T01:47:59.421 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8130000000000002e-06 2024-04-20T01:47:59.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.421 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.421 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.421 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788372+0000", 2024-04-20T01:47:59.421 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027740410000000001 2024-04-20T01:47:59.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.421 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.421 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:59.421 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788381+0000", 2024-04-20T01:47:59.421 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5930000000000006e-06 2024-04-20T01:47:59.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.421 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.422 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:59.422 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796518+0000", 2024-04-20T01:47:59.422 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0081374710000000003 2024-04-20T01:47:59.422 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.422 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.422 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:59.422 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796533+0000", 2024-04-20T01:47:59.422 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.451e-05 2024-04-20T01:47:59.422 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.423 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.423 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.423 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.423 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.423 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s4 33/30 ECSubWrite(tid=10, reqid=client.4643.0:70, at_version=33'7, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:59.423 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.787329+0000", 2024-04-20T01:47:59.423 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 24.622854651000001, 2024-04-20T01:47:59.423 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092166409999999994, 2024-04-20T01:47:59.423 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:59.423 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:59.423 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:59.423 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.423 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:59.424 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787329+0000", 2024-04-20T01:47:59.424 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.424 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.424 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:59.424 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787329+0000", 2024-04-20T01:47:59.424 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.424 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.424 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:59.424 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787328+0000", 2024-04-20T01:47:59.424 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:59.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.425 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.425 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:59.425 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787448+0000", 2024-04-20T01:47:59.425 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011992199999999999 2024-04-20T01:47:59.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.425 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.425 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:59.425 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787458+0000", 2024-04-20T01:47:59.425 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0037000000000001e-05 2024-04-20T01:47:59.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.425 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.425 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:59.426 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787466+0000", 2024-04-20T01:47:59.426 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6270000000000006e-06 2024-04-20T01:47:59.426 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.426 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.426 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.426 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787517+0000", 2024-04-20T01:47:59.426 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0890000000000002e-05 2024-04-20T01:47:59.426 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.426 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.426 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:59.426 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787528+0000", 2024-04-20T01:47:59.427 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0363000000000001e-05 2024-04-20T01:47:59.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.427 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:59.427 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796522+0000", 2024-04-20T01:47:59.427 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0089942619999999994 2024-04-20T01:47:59.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.427 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:59.427 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796545+0000", 2024-04-20T01:47:59.427 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3213999999999998e-05 2024-04-20T01:47:59.427 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.427 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.428 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.428 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s4 33/30 ECSubWrite(tid=11, reqid=client.4643.0:71, at_version=33'8, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:47:59.428 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.788214+0000", 2024-04-20T01:47:59.428 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 24.62196917, 2024-04-20T01:47:59.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083448129999999995, 2024-04-20T01:47:59.428 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:59.428 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:59.428 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:59.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:59.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788214+0000", 2024-04-20T01:47:59.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:59.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788214+0000", 2024-04-20T01:47:59.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:59.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788214+0000", 2024-04-20T01:47:59.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:59.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.430 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:59.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788224+0000", 2024-04-20T01:47:59.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0013000000000001e-05 2024-04-20T01:47:59.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.430 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:59.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788231+0000", 2024-04-20T01:47:59.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0990000000000001e-06 2024-04-20T01:47:59.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.430 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:59.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788245+0000", 2024-04-20T01:47:59.431 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4166e-05 2024-04-20T01:47:59.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.431 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.431 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.431 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788446+0000", 2024-04-20T01:47:59.431 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00020135800000000001 2024-04-20T01:47:59.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.431 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.431 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:59.431 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788455+0000", 2024-04-20T01:47:59.431 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5099999999999998e-06 2024-04-20T01:47:59.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.432 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.432 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:59.432 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796549+0000", 2024-04-20T01:47:59.432 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080940860000000003 2024-04-20T01:47:59.432 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.432 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.432 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:59.432 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796559+0000", 2024-04-20T01:47:59.432 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9340000000000001e-06 2024-04-20T01:47:59.432 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.432 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.432 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.433 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:106 3.cs0 3:3f280175:::smithi16579500-36:head [append 0~589824,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:59.433 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.823792+0000", 2024-04-20T01:47:59.433 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 24.586391464999998, 2024-04-20T01:47:59.433 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0085904729999999995, 2024-04-20T01:47:59.433 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:59.433 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:59.433 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:47:59.433 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:47:59.433 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:59.433 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 106 2024-04-20T01:47:59.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.434 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:59.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:59.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.823792+0000", 2024-04-20T01:47:59.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:59.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.823792+0000", 2024-04-20T01:47:59.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:59.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.823790+0000", 2024-04-20T01:47:59.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:59.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:59.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824429+0000", 2024-04-20T01:47:59.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00063880600000000001 2024-04-20T01:47:59.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:59.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824439+0000", 2024-04-20T01:47:59.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2709999999999998e-06 2024-04-20T01:47:59.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:59.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824445+0000", 2024-04-20T01:47:59.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7789999999999998e-06 2024-04-20T01:47:59.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824497+0000", 2024-04-20T01:47:59.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1582000000000002e-05 2024-04-20T01:47:59.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.437 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:47:59.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824650+0000", 2024-04-20T01:47:59.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015330199999999999 2024-04-20T01:47:59.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.437 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:59.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.825666+0000", 2024-04-20T01:47:59.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010162280000000001 2024-04-20T01:47:59.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:59.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.827037+0000", 2024-04-20T01:47:59.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001370432 2024-04-20T01:47:59.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:47:59.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.832324+0000", 2024-04-20T01:47:59.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052870970000000002 2024-04-20T01:47:59.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.439 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:59.439 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.832382+0000", 2024-04-20T01:47:59.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8275e-05 2024-04-20T01:47:59.439 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.439 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.439 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.439 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:107 3.cs0 3:3f280175:::smithi16579500-36:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:59.439 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.824453+0000", 2024-04-20T01:47:59.439 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 24.585729990000001, 2024-04-20T01:47:59.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0085790480000000006, 2024-04-20T01:47:59.440 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:59.440 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:59.440 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:47:59.440 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:47:59.440 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:59.440 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 107 2024-04-20T01:47:59.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.440 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:59.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:59.440 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824453+0000", 2024-04-20T01:47:59.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:59.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824453+0000", 2024-04-20T01:47:59.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:59.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824453+0000", 2024-04-20T01:47:59.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:59.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:59.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824457+0000", 2024-04-20T01:47:59.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9160000000000003e-06 2024-04-20T01:47:59.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:59.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824464+0000", 2024-04-20T01:47:59.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4569999999999999e-06 2024-04-20T01:47:59.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:59.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824469+0000", 2024-04-20T01:47:59.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5229999999999997e-06 2024-04-20T01:47:59.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.443 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.826264+0000", 2024-04-20T01:47:59.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001795004 2024-04-20T01:47:59.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.443 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:47:59.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.826336+0000", 2024-04-20T01:47:59.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2331999999999998e-05 2024-04-20T01:47:59.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:59.444 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.826671+0000", 2024-04-20T01:47:59.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00033562300000000001 2024-04-20T01:47:59.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:59.444 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.827561+0000", 2024-04-20T01:47:59.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00088987500000000004 2024-04-20T01:47:59.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:47:59.444 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.832965+0000", 2024-04-20T01:47:59.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054034390000000003 2024-04-20T01:47:59.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:59.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.833032+0000", 2024-04-20T01:47:59.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7470000000000003e-05 2024-04-20T01:47:59.445 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.445 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.445 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.445 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:108 3.cs0 3:3f280175:::smithi16579500-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:47:59.446 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.824485+0000", 2024-04-20T01:47:59.446 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 24.58569863, 2024-04-20T01:47:59.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0099600550000000006, 2024-04-20T01:47:59.446 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:59.446 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:47:59.446 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:47:59.446 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:47:59.446 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:59.446 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 108 2024-04-20T01:47:59.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.446 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:59.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:59.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824485+0000", 2024-04-20T01:47:59.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:59.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824485+0000", 2024-04-20T01:47:59.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:59.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824484+0000", 2024-04-20T01:47:59.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:59.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.448 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:59.448 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824487+0000", 2024-04-20T01:47:59.448 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4889999999999998e-06 2024-04-20T01:47:59.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.448 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:59.448 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824493+0000", 2024-04-20T01:47:59.448 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1709999999999999e-06 2024-04-20T01:47:59.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:59.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824498+0000", 2024-04-20T01:47:59.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1569999999999999e-06 2024-04-20T01:47:59.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.826844+0000", 2024-04-20T01:47:59.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023460249999999998 2024-04-20T01:47:59.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for rw locks", 2024-04-20T01:47:59.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.826938+0000", 2024-04-20T01:47:59.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3547000000000004e-05 2024-04-20T01:47:59.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.450 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.833332+0000", 2024-04-20T01:47:59.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006394846 2024-04-20T01:47:59.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.450 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:47:59.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.833419+0000", 2024-04-20T01:47:59.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6304999999999994e-05 2024-04-20T01:47:59.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:59.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.834445+0000", 2024-04-20T01:47:59.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001025946 2024-04-20T01:47:59.451 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.451 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.451 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.451 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s4 42/30 ECSubWrite(tid=229, reqid=client.4643.0:302, at_version=42'29, trim_to=0'0, roll_forward_to=42'27))", 2024-04-20T01:47:59.451 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.829628+0000", 2024-04-20T01:47:59.451 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.580555482999999, 2024-04-20T01:47:59.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0078318369999999995, 2024-04-20T01:47:59.452 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:59.452 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:59.452 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:59.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.452 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:59.452 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.829628+0000", 2024-04-20T01:47:59.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.452 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:59.452 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.829628+0000", 2024-04-20T01:47:59.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.453 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.453 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:59.453 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.829627+0000", 2024-04-20T01:47:59.453 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:59.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.453 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.453 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:59.453 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.829954+0000", 2024-04-20T01:47:59.453 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032663000000000001 2024-04-20T01:47:59.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.453 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.453 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:59.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.829972+0000", 2024-04-20T01:47:59.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8649999999999999e-05 2024-04-20T01:47:59.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:59.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.829982+0000", 2024-04-20T01:47:59.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4450000000000002e-06 2024-04-20T01:47:59.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830037+0000", 2024-04-20T01:47:59.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5189000000000003e-05 2024-04-20T01:47:59.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.455 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:59.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830047+0000", 2024-04-20T01:47:59.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8160000000000005e-06 2024-04-20T01:47:59.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.455 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:59.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.837433+0000", 2024-04-20T01:47:59.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0073860899999999997 2024-04-20T01:47:59.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.456 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:59.456 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.837460+0000", 2024-04-20T01:47:59.456 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6821e-05 2024-04-20T01:47:59.456 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.456 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.456 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.456 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.456 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:325 3.cs0 3:3f280175:::smithi16579500-36:head [append 0~425984] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:47:59.456 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.949234+0000", 2024-04-20T01:47:59.456 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.460949447999999, 2024-04-20T01:47:59.456 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092546179999999992, 2024-04-20T01:47:59.457 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:59.457 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:59.457 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:47:59.457 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:47:59.457 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:59.457 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 325 2024-04-20T01:47:59.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.457 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:59.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.457 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:59.457 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.949234+0000", 2024-04-20T01:47:59.457 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.458 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:59.458 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.949234+0000", 2024-04-20T01:47:59.458 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.458 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.458 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:59.458 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.949232+0000", 2024-04-20T01:47:59.458 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:47:59.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.458 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.458 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:59.459 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.949737+0000", 2024-04-20T01:47:59.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00050513299999999995 2024-04-20T01:47:59.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.459 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:59.459 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.949751+0000", 2024-04-20T01:47:59.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3966e-05 2024-04-20T01:47:59.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.459 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:59.459 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.949776+0000", 2024-04-20T01:47:59.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4946000000000001e-05 2024-04-20T01:47:59.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.949831+0000", 2024-04-20T01:47:59.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4591e-05 2024-04-20T01:47:59.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:47:59.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.949953+0000", 2024-04-20T01:47:59.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012193600000000001 2024-04-20T01:47:59.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:59.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.951143+0000", 2024-04-20T01:47:59.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001190387 2024-04-20T01:47:59.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:59.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.952463+0000", 2024-04-20T01:47:59.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013197879999999999 2024-04-20T01:47:59.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:47:59.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.958405+0000", 2024-04-20T01:47:59.462 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005941956 2024-04-20T01:47:59.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.462 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.462 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:59.462 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.958488+0000", 2024-04-20T01:47:59.462 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3463999999999999e-05 2024-04-20T01:47:59.462 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.462 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.462 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.462 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.462 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:328 3.cs0 3:3f280175:::smithi16579500-36:head [setxattr _header (56) in=7b] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:47:59.463 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.953789+0000", 2024-04-20T01:47:59.463 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.456393787, 2024-04-20T01:47:59.463 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0088952739999999999, 2024-04-20T01:47:59.463 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:59.463 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:47:59.463 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:47:59.463 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:47:59.463 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:59.463 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 328 2024-04-20T01:47:59.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.463 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:59.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.464 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:59.464 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953789+0000", 2024-04-20T01:47:59.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.464 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:59.464 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953789+0000", 2024-04-20T01:47:59.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.464 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:59.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953789+0000", 2024-04-20T01:47:59.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:59.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:59.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953793+0000", 2024-04-20T01:47:59.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3460000000000004e-06 2024-04-20T01:47:59.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:59.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953797+0000", 2024-04-20T01:47:59.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7570000000000002e-06 2024-04-20T01:47:59.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:59.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953800+0000", 2024-04-20T01:47:59.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0809999999999998e-06 2024-04-20T01:47:59.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957330+0000", 2024-04-20T01:47:59.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035297430000000001 2024-04-20T01:47:59.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.467 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:47:59.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957432+0000", 2024-04-20T01:47:59.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010186799999999999 2024-04-20T01:47:59.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.467 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:59.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957890+0000", 2024-04-20T01:47:59.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045805100000000001 2024-04-20T01:47:59.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.467 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:59.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.962457+0000", 2024-04-20T01:47:59.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045676720000000001 2024-04-20T01:47:59.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.468 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:47:59.468 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.962618+0000", 2024-04-20T01:47:59.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016040600000000001 2024-04-20T01:47:59.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.468 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:59.468 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.962685+0000", 2024-04-20T01:47:59.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6853999999999994e-05 2024-04-20T01:47:59.468 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.468 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.469 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.469 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:329 3.cs0 3:3f280175:::smithi16579500-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:47:59.469 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.953803+0000", 2024-04-20T01:47:59.469 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.456380529, 2024-04-20T01:47:59.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011991458999999999, 2024-04-20T01:47:59.469 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:59.469 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:47:59.469 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:47:59.469 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:47:59.469 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:47:59.469 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 329 2024-04-20T01:47:59.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.470 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:59.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:59.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953803+0000", 2024-04-20T01:47:59.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:59.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953803+0000", 2024-04-20T01:47:59.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:59.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953802+0000", 2024-04-20T01:47:59.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:59.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:59.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953805+0000", 2024-04-20T01:47:59.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.553e-06 2024-04-20T01:47:59.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:59.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953809+0000", 2024-04-20T01:47:59.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.715e-06 2024-04-20T01:47:59.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.472 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:59.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953812+0000", 2024-04-20T01:47:59.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0529999999999998e-06 2024-04-20T01:47:59.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.472 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.958685+0000", 2024-04-20T01:47:59.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048727739999999999 2024-04-20T01:47:59.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.473 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for rw locks", 2024-04-20T01:47:59.473 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.958748+0000", 2024-04-20T01:47:59.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3748e-05 2024-04-20T01:47:59.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.473 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.473 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.962966+0000", 2024-04-20T01:47:59.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042174300000000003 2024-04-20T01:47:59.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:47:59.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.963059+0000", 2024-04-20T01:47:59.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3437999999999995e-05 2024-04-20T01:47:59.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:59.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.965794+0000", 2024-04-20T01:47:59.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027349990000000001 2024-04-20T01:47:59.474 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.474 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.474 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.475 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s4 48/30 ECSubWrite(tid=253, reqid=osd.13.0:253, at_version=48'44, trim_to=0'0, roll_forward_to=47'40))", 2024-04-20T01:47:59.475 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.711403+0000", 2024-04-20T01:47:59.475 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.6987799119999991, 2024-04-20T01:47:59.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0093226750000000008, 2024-04-20T01:47:59.475 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:59.475 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:59.475 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:59.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:59.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711403+0000", 2024-04-20T01:47:59.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:59.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711403+0000", 2024-04-20T01:47:59.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:59.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711403+0000", 2024-04-20T01:47:59.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:59.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:59.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711422+0000", 2024-04-20T01:47:59.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9680000000000001e-05 2024-04-20T01:47:59.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:59.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711432+0000", 2024-04-20T01:47:59.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6330000000000008e-06 2024-04-20T01:47:59.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:59.478 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711440+0000", 2024-04-20T01:47:59.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1699999999999997e-06 2024-04-20T01:47:59.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.478 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.478 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711486+0000", 2024-04-20T01:47:59.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6118000000000001e-05 2024-04-20T01:47:59.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.478 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:59.478 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711496+0000", 2024-04-20T01:47:59.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9890000000000004e-06 2024-04-20T01:47:59.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.479 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.479 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:59.479 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.720699+0000", 2024-04-20T01:47:59.479 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092022619999999992 2024-04-20T01:47:59.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.479 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.479 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:59.479 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.720726+0000", 2024-04-20T01:47:59.479 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7378000000000001e-05 2024-04-20T01:47:59.479 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.479 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.479 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.480 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.480 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s4 48/30 ECSubWrite(tid=44, reqid=osd.5.0:44, at_version=48'33, trim_to=0'0, roll_forward_to=47'31))", 2024-04-20T01:47:59.480 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.716826+0000", 2024-04-20T01:47:59.480 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.6933572130000005, 2024-04-20T01:47:59.480 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0082773139999999992, 2024-04-20T01:47:59.480 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:59.480 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:59.480 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:59.480 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.480 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:59.480 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716826+0000", 2024-04-20T01:47:59.481 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.481 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.481 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:59.481 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716826+0000", 2024-04-20T01:47:59.481 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.481 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.481 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:59.481 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716825+0000", 2024-04-20T01:47:59.481 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:59.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.481 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.482 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:59.482 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716843+0000", 2024-04-20T01:47:59.482 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7390000000000001e-05 2024-04-20T01:47:59.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.482 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.482 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:59.482 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716858+0000", 2024-04-20T01:47:59.482 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5009e-05 2024-04-20T01:47:59.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.482 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.482 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:59.483 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716867+0000", 2024-04-20T01:47:59.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8589999999999994e-06 2024-04-20T01:47:59.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.483 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.483 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716929+0000", 2024-04-20T01:47:59.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2701999999999997e-05 2024-04-20T01:47:59.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.483 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:59.483 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716936+0000", 2024-04-20T01:47:59.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5309999999999998e-06 2024-04-20T01:47:59.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.484 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.484 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:59.484 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.725078+0000", 2024-04-20T01:47:59.484 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0081417729999999997 2024-04-20T01:47:59.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.484 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.484 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:59.484 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.725103+0000", 2024-04-20T01:47:59.484 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.561e-05 2024-04-20T01:47:59.484 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.484 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.484 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.485 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.as3 48/30 ECSubWrite(tid=33, reqid=osd.6.0:33, at_version=48'14, trim_to=0'0, roll_forward_to=47'12))", 2024-04-20T01:47:59.485 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.717236+0000", 2024-04-20T01:47:59.485 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.6929472820000004, 2024-04-20T01:47:59.485 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0078967310000000006, 2024-04-20T01:47:59.485 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:59.485 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:59.485 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:59.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.485 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:59.485 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717236+0000", 2024-04-20T01:47:59.485 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:59.486 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717236+0000", 2024-04-20T01:47:59.486 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:59.486 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717235+0000", 2024-04-20T01:47:59.486 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:47:59.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:59.487 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717253+0000", 2024-04-20T01:47:59.487 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7399000000000001e-05 2024-04-20T01:47:59.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.487 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.487 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:59.487 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717264+0000", 2024-04-20T01:47:59.487 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1158999999999999e-05 2024-04-20T01:47:59.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.487 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.487 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:59.487 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717278+0000", 2024-04-20T01:47:59.487 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4477999999999999e-05 2024-04-20T01:47:59.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.488 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.488 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717319+0000", 2024-04-20T01:47:59.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0519999999999998e-05 2024-04-20T01:47:59.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.488 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:59.488 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717325+0000", 2024-04-20T01:47:59.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1229999999999997e-06 2024-04-20T01:47:59.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.488 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:59.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.725107+0000", 2024-04-20T01:47:59.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077819209999999998 2024-04-20T01:47:59.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:59.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.725133+0000", 2024-04-20T01:47:59.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5707e-05 2024-04-20T01:47:59.489 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.489 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.489 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.490 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s4 55/30 ECSubWrite(tid=47, reqid=client.4643.0:462, at_version=55'36, trim_to=0'0, roll_forward_to=48'33))", 2024-04-20T01:47:59.490 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:56.740180+0000", 2024-04-20T01:47:59.490 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 2.6700034640000001, 2024-04-20T01:47:59.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0088424850000000006, 2024-04-20T01:47:59.490 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:59.490 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:59.490 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:59.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.490 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:59.490 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740180+0000", 2024-04-20T01:47:59.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:59.491 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740180+0000", 2024-04-20T01:47:59.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:59.491 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740179+0000", 2024-04-20T01:47:59.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:59.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:59.491 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740203+0000", 2024-04-20T01:47:59.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4267999999999999e-05 2024-04-20T01:47:59.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.492 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:59.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740222+0000", 2024-04-20T01:47:59.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9032e-05 2024-04-20T01:47:59.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.492 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:59.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740233+0000", 2024-04-20T01:47:59.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1381e-05 2024-04-20T01:47:59.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740307+0000", 2024-04-20T01:47:59.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3559000000000005e-05 2024-04-20T01:47:59.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:59.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740320+0000", 2024-04-20T01:47:59.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3339000000000001e-05 2024-04-20T01:47:59.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:59.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.748994+0000", 2024-04-20T01:47:59.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0086742150000000007 2024-04-20T01:47:59.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:59.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.749022+0000", 2024-04-20T01:47:59.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7746000000000002e-05 2024-04-20T01:47:59.494 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.494 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.494 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.494 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s4 55/30 ECSubWrite(tid=48, reqid=client.4643.0:463, at_version=55'37, trim_to=0'0, roll_forward_to=48'33))", 2024-04-20T01:47:59.495 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:56.740880+0000", 2024-04-20T01:47:59.495 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 2.6693030420000001, 2024-04-20T01:47:59.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0081551660000000002, 2024-04-20T01:47:59.495 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:47:59.495 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:47:59.495 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:47:59.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:47:59.495 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740880+0000", 2024-04-20T01:47:59.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:47:59.496 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740880+0000", 2024-04-20T01:47:59.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:47:59.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.496 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:47:59.496 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740879+0000", 2024-04-20T01:47:59.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:47:59.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.496 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:47:59.496 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740910+0000", 2024-04-20T01:47:59.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0546999999999997e-05 2024-04-20T01:47:59.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.497 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:47:59.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740926+0000", 2024-04-20T01:47:59.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6433e-05 2024-04-20T01:47:59.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.497 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:47:59.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740936+0000", 2024-04-20T01:47:59.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.024e-05 2024-04-20T01:47:59.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:47:59.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.742457+0000", 2024-04-20T01:47:59.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001520948 2024-04-20T01:47:59.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:47:59.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.742464+0000", 2024-04-20T01:47:59.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9970000000000001e-06 2024-04-20T01:47:59.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:47:59.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.749027+0000", 2024-04-20T01:47:59.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065627109999999997 2024-04-20T01:47:59.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:47:59.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:47:59.499 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:47:59.499 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.749035+0000", 2024-04-20T01:47:59.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1689999999999992e-06 2024-04-20T01:47:59.499 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.499 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.499 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.499 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:47:59.499 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:47:59.499 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:47:59.501 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-20T01:47:59.502 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:47:59.515+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:47:59.508 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:47:59.510 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:47:59.510 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:47:59.510 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:47:59.510 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:47:59.510 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:47:59.510 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:47:59.511 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:47:59.511 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:47:59.511 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:47:59.511 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:47:59.511 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:47:59.511 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:47:59.511 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:47:59.511 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:47:59.511 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:47:59.511 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:47:59.602 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:47:59.617+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:47:59.643 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:47:59.643 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:47:59.643 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-04-20T01:47:59.643 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:47:59.653 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-20T01:47:59.703 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:47:59.717+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:47:59.712 INFO:tasks.rados.rados.0.smithi165.stdout:200: done (0 left) 2024-04-20T01:47:59.712 INFO:tasks.rados.rados.0.smithi165.stdout:201: read oid 15 snap 13 2024-04-20T01:47:59.713 INFO:tasks.rados.rados.0.smithi165.stdout:201: expect (ObjNum 14 snap 0 seq_num 14) 2024-04-20T01:47:59.713 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 19 current snap is 15 2024-04-20T01:47:59.713 INFO:tasks.rados.rados.0.smithi165.stdout:202: seq_num 93 ranges {3768320=802816,4571136=638976,5210112=131072} 2024-04-20T01:47:59.718 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is unset 2024-04-20T01:47:59.737 INFO:tasks.rados.rados.0.smithi165.stdout:202: writing smithi16579500-19 from 3768320 to 4571136 tid 1 2024-04-20T01:47:59.741 INFO:tasks.rados.rados.0.smithi165.stdout:202: writing smithi16579500-19 from 4571136 to 5210112 tid 2 2024-04-20T01:47:59.742 INFO:tasks.rados.rados.0.smithi165.stdout:202: writing smithi16579500-19 from 5210112 to 5341184 tid 3 2024-04-20T01:47:59.742 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 14 current snap is 15 2024-04-20T01:47:59.742 INFO:tasks.rados.rados.0.smithi165.stdout:203: seq_num 94 ranges {131072=425984,557056=786432,1343488=376832} 2024-04-20T01:47:59.745 INFO:tasks.rados.rados.0.smithi165.stdout:203: writing smithi16579500-14 from 131072 to 557056 tid 1 2024-04-20T01:47:59.749 INFO:tasks.rados.rados.0.smithi165.stdout:203: writing smithi16579500-14 from 557056 to 1343488 tid 2 2024-04-20T01:47:59.752 INFO:tasks.rados.rados.0.smithi165.stdout:203: writing smithi16579500-14 from 1343488 to 1720320 tid 3 2024-04-20T01:47:59.752 INFO:tasks.rados.rados.0.smithi165.stdout:204: delete oid 25 current snap is 15 2024-04-20T01:47:59.756 INFO:tasks.rados.rados.0.smithi165.stdout:203: finishing write tid 1 to smithi16579500-14 2024-04-20T01:47:59.756 INFO:tasks.rados.rados.0.smithi165.stdout:203: finishing write tid 2 to smithi16579500-14 2024-04-20T01:47:59.758 INFO:tasks.rados.rados.0.smithi165.stdout:204: done (3 left) 2024-04-20T01:47:59.758 INFO:tasks.rados.rados.0.smithi165.stdout:205: copy_from oid 21 from oid 37 current snap is 15 2024-04-20T01:47:59.758 INFO:tasks.rados.rados.0.smithi165.stdout:206: snap_create 2024-04-20T01:47:59.764 INFO:tasks.rados.rados.0.smithi165.stdout:203: finishing write tid 3 to smithi16579500-14 2024-04-20T01:47:59.764 INFO:tasks.rados.rados.0.smithi165.stdout:203: finishing write tid 4 to smithi16579500-14 2024-04-20T01:47:59.765 INFO:tasks.rados.rados.0.smithi165.stdout:203: finishing write tid 5 to smithi16579500-14 2024-04-20T01:47:59.765 INFO:tasks.rados.rados.0.smithi165.stdout:203: oid 14 updating version 0 to 40 2024-04-20T01:47:59.766 INFO:tasks.rados.rados.0.smithi165.stdout:203: oid 14 version 40 is already newer than 39 2024-04-20T01:47:59.766 INFO:tasks.rados.rados.0.smithi165.stdout:203: oid 14 updating version 40 to 41 2024-04-20T01:47:59.766 INFO:tasks.rados.rados.0.smithi165.stdout:203: oid 14 updating version 41 to 42 2024-04-20T01:47:59.766 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 14 v 42 (ObjNum 94 snap 15 seq_num 94) dirty exists 2024-04-20T01:47:59.766 INFO:tasks.rados.rados.0.smithi165.stdout:203: left oid 14 (ObjNum 94 snap 15 seq_num 94) 2024-04-20T01:47:59.771 INFO:tasks.rados.rados.0.smithi165.stdout:205: finishing copy_from racing read to smithi16579500-21 2024-04-20T01:47:59.772 INFO:tasks.rados.rados.0.smithi165.stdout:205: finishing copy_from to smithi16579500-21 2024-04-20T01:47:59.772 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 21 v 27 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-04-20T01:47:59.783 INFO:tasks.rados.rados.0.smithi165.stdout:201: expect (ObjNum 14 snap 0 seq_num 14) 2024-04-20T01:47:59.802 INFO:tasks.rados.rados.0.smithi165.stdout:202: finishing write tid 1 to smithi16579500-19 2024-04-20T01:47:59.804 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:47:59.818+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:47:59.805 INFO:tasks.rados.rados.0.smithi165.stdout:202: finishing write tid 2 to smithi16579500-19 2024-04-20T01:47:59.808 INFO:tasks.rados.rados.0.smithi165.stdout:202: finishing write tid 3 to smithi16579500-19 2024-04-20T01:47:59.810 INFO:tasks.rados.rados.0.smithi165.stdout:202: finishing write tid 4 to smithi16579500-19 2024-04-20T01:47:59.816 INFO:tasks.rados.rados.0.smithi165.stdout:202: finishing write tid 5 to smithi16579500-19 2024-04-20T01:47:59.816 INFO:tasks.rados.rados.0.smithi165.stdout:202: oid 19 updating version 0 to 79 2024-04-20T01:47:59.816 INFO:tasks.rados.rados.0.smithi165.stdout:202: oid 19 updating version 79 to 80 2024-04-20T01:47:59.816 INFO:tasks.rados.rados.0.smithi165.stdout:202: oid 19 version 80 is already newer than 78 2024-04-20T01:47:59.816 INFO:tasks.rados.rados.0.smithi165.stdout:202: oid 19 updating version 80 to 81 2024-04-20T01:47:59.816 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 19 v 81 (ObjNum 93 snap 15 seq_num 93) dirty exists 2024-04-20T01:47:59.816 INFO:tasks.rados.rados.0.smithi165.stdout:202: left oid 19 (ObjNum 93 snap 15 seq_num 93) 2024-04-20T01:47:59.836 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:47:59.836 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:47:59.836 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:47:59.837 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:47:59.837 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:47:59.848 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-20T01:47:59.904 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:47:59.919+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:00.005 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:00.020+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:00.020 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:48:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:48:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:48:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s5 36/30 ECSubWrite(tid=40, reqid=client.4643.0:215, at_version=36'28, trim_to=0'0, roll_forward_to=36'26))", 2024-04-20T01:48:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:37.808713+0000", 2024-04-20T01:48:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 22.224393395, 2024-04-20T01:48:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059407280000000002, 2024-04-20T01:48:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.808713+0000", 2024-04-20T01:48:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.808713+0000", 2024-04-20T01:48:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.808712+0000", 2024-04-20T01:48:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.808741+0000", 2024-04-20T01:48:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9128999999999998e-05 2024-04-20T01:48:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.808753+0000", 2024-04-20T01:48:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1314999999999999e-05 2024-04-20T01:48:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.808759+0000", 2024-04-20T01:48:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7560000000000002e-06 2024-04-20T01:48:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.808810+0000", 2024-04-20T01:48:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0291999999999999e-05 2024-04-20T01:48:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.808820+0000", 2024-04-20T01:48:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7739999999999999e-06 2024-04-20T01:48:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.814625+0000", 2024-04-20T01:48:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0058058550000000004 2024-04-20T01:48:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.814654+0000", 2024-04-20T01:48:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8212999999999999e-05 2024-04-20T01:48:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:234 3.0s0 3:0c1ed9f1:::smithi16579500-46:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.787453+0000", 2024-04-20T01:48:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 21.245653554, 2024-04-20T01:48:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016795529, 2024-04-20T01:48:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 234 2024-04-20T01:48:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787453+0000", 2024-04-20T01:48:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787453+0000", 2024-04-20T01:48:00.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787451+0000", 2024-04-20T01:48:00.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:00.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787462+0000", 2024-04-20T01:48:00.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1083e-05 2024-04-20T01:48:00.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787483+0000", 2024-04-20T01:48:00.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1656999999999999e-05 2024-04-20T01:48:00.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787503+0000", 2024-04-20T01:48:00.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9623000000000002e-05 2024-04-20T01:48:00.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787595+0000", 2024-04-20T01:48:00.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2417000000000006e-05 2024-04-20T01:48:00.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:00.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787604+0000", 2024-04-20T01:48:00.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.952e-06 2024-04-20T01:48:00.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.791584+0000", 2024-04-20T01:48:00.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039792250000000003 2024-04-20T01:48:00.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:00.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.791699+0000", 2024-04-20T01:48:00.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115692 2024-04-20T01:48:00.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.792530+0000", 2024-04-20T01:48:00.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083077900000000002 2024-04-20T01:48:00.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.798777+0000", 2024-04-20T01:48:00.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062472379999999996 2024-04-20T01:48:00.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:00.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.804175+0000", 2024-04-20T01:48:00.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005397158 2024-04-20T01:48:00.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.804248+0000", 2024-04-20T01:48:00.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3682000000000004e-05 2024-04-20T01:48:00.030 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.030 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.030 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.030 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:235 3.0s0 3:0c1ed9f1:::smithi16579500-46:head [rollback 5] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:00.030 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.787517+0000", 2024-04-20T01:48:00.030 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 21.245589724999999, 2024-04-20T01:48:00.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019202061999999999, 2024-04-20T01:48:00.030 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.031 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:00.031 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:00.031 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:00.031 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:00.031 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 235 2024-04-20T01:48:00.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.031 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787517+0000", 2024-04-20T01:48:00.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787517+0000", 2024-04-20T01:48:00.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787516+0000", 2024-04-20T01:48:00.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:00.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787521+0000", 2024-04-20T01:48:00.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2599999999999996e-06 2024-04-20T01:48:00.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787534+0000", 2024-04-20T01:48:00.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2717e-05 2024-04-20T01:48:00.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787538+0000", 2024-04-20T01:48:00.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3540000000000002e-06 2024-04-20T01:48:00.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787673+0000", 2024-04-20T01:48:00.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013480799999999999 2024-04-20T01:48:00.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:00.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787678+0000", 2024-04-20T01:48:00.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5229999999999997e-06 2024-04-20T01:48:00.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793075+0000", 2024-04-20T01:48:00.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053977549999999997 2024-04-20T01:48:00.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:00.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793190+0000", 2024-04-20T01:48:00.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114775 2024-04-20T01:48:00.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793760+0000", 2024-04-20T01:48:00.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00056945599999999998 2024-04-20T01:48:00.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799198+0000", 2024-04-20T01:48:00.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0054385589999999999 2024-04-20T01:48:00.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:00.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806653+0000", 2024-04-20T01:48:00.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0074547739999999999 2024-04-20T01:48:00.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806719+0000", 2024-04-20T01:48:00.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5732e-05 2024-04-20T01:48:00.036 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.036 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.036 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.036 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:236 3.0s0 3:0c1ed9f1:::smithi16579500-46:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:00.036 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:48:00.036 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 21.245530111000001, 2024-04-20T01:48:00.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019273109, 2024-04-20T01:48:00.036 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.037 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:00.037 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:00.037 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:00.037 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:00.037 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 236 2024-04-20T01:48:00.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.037 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:48:00.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:48:00.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:48:00.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:00.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787579+0000", 2024-04-20T01:48:00.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6380000000000002e-06 2024-04-20T01:48:00.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787586+0000", 2024-04-20T01:48:00.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.81e-06 2024-04-20T01:48:00.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787592+0000", 2024-04-20T01:48:00.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5180000000000004e-06 2024-04-20T01:48:00.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787705+0000", 2024-04-20T01:48:00.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011387000000000001 2024-04-20T01:48:00.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:00.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787712+0000", 2024-04-20T01:48:00.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6229999999999999e-06 2024-04-20T01:48:00.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794765+0000", 2024-04-20T01:48:00.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0070530519999999998 2024-04-20T01:48:00.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:00.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794873+0000", 2024-04-20T01:48:00.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107861 2024-04-20T01:48:00.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795365+0000", 2024-04-20T01:48:00.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049192300000000001 2024-04-20T01:48:00.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799201+0000", 2024-04-20T01:48:00.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003836454 2024-04-20T01:48:00.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:00.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806808+0000", 2024-04-20T01:48:00.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076070119999999998 2024-04-20T01:48:00.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806849+0000", 2024-04-20T01:48:00.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0852999999999997e-05 2024-04-20T01:48:00.042 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.042 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.042 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.042 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:301 3.0s0 3:06dd530f:::smithi16579500-30:head [delete] snapc a=[a] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:00.042 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:43.808446+0000", 2024-04-20T01:48:00.042 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 16.22466047, 2024-04-20T01:48:00.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0088321900000000002, 2024-04-20T01:48:00.042 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.042 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:00.042 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:00.042 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:00.042 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:00.043 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 301 2024-04-20T01:48:00.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.043 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.808446+0000", 2024-04-20T01:48:00.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.808446+0000", 2024-04-20T01:48:00.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.808444+0000", 2024-04-20T01:48:00.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:00.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.808453+0000", 2024-04-20T01:48:00.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4010000000000003e-06 2024-04-20T01:48:00.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.808472+0000", 2024-04-20T01:48:00.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8943000000000001e-05 2024-04-20T01:48:00.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.808483+0000", 2024-04-20T01:48:00.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1029999999999999e-05 2024-04-20T01:48:00.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.808542+0000", 2024-04-20T01:48:00.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8699000000000002e-05 2024-04-20T01:48:00.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:00.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.808547+0000", 2024-04-20T01:48:00.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0350000000000001e-06 2024-04-20T01:48:00.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.812087+0000", 2024-04-20T01:48:00.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003539805 2024-04-20T01:48:00.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:00.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.812209+0000", 2024-04-20T01:48:00.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012280099999999999 2024-04-20T01:48:00.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.812979+0000", 2024-04-20T01:48:00.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00076978599999999995 2024-04-20T01:48:00.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.813899+0000", 2024-04-20T01:48:00.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00092017100000000001 2024-04-20T01:48:00.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:00.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.817202+0000", 2024-04-20T01:48:00.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003302508 2024-04-20T01:48:00.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.817278+0000", 2024-04-20T01:48:00.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6055000000000003e-05 2024-04-20T01:48:00.048 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.048 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.048 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.048 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs3 47/30 ECSubWrite(tid=39, reqid=osd.14.0:39, at_version=47'33, trim_to=0'0, roll_forward_to=46'29))", 2024-04-20T01:48:00.048 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.712895+0000", 2024-04-20T01:48:00.048 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.320211618, 2024-04-20T01:48:00.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093698280000000002, 2024-04-20T01:48:00.048 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.048 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.048 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.712895+0000", 2024-04-20T01:48:00.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.712895+0000", 2024-04-20T01:48:00.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.712894+0000", 2024-04-20T01:48:00.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:00.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.712936+0000", 2024-04-20T01:48:00.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2069e-05 2024-04-20T01:48:00.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.712949+0000", 2024-04-20T01:48:00.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2996000000000001e-05 2024-04-20T01:48:00.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.712960+0000", 2024-04-20T01:48:00.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1192000000000001e-05 2024-04-20T01:48:00.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.716318+0000", 2024-04-20T01:48:00.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003358345 2024-04-20T01:48:00.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.716329+0000", 2024-04-20T01:48:00.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0284e-05 2024-04-20T01:48:00.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722236+0000", 2024-04-20T01:48:00.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059077909999999999 2024-04-20T01:48:00.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722264+0000", 2024-04-20T01:48:00.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8061000000000001e-05 2024-04-20T01:48:00.052 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.052 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.052 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.052 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.1s5 47/30 ECSubWrite(tid=112, reqid=client.4643.0:385, at_version=47'12, trim_to=0'0, roll_forward_to=41'10))", 2024-04-20T01:48:00.052 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.720057+0000", 2024-04-20T01:48:00.052 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.313049040999999, 2024-04-20T01:48:00.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.006115934, 2024-04-20T01:48:00.052 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.052 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.052 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.720057+0000", 2024-04-20T01:48:00.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.720057+0000", 2024-04-20T01:48:00.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.720056+0000", 2024-04-20T01:48:00.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:00.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.720123+0000", 2024-04-20T01:48:00.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6523e-05 2024-04-20T01:48:00.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.720137+0000", 2024-04-20T01:48:00.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4175999999999999e-05 2024-04-20T01:48:00.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.720147+0000", 2024-04-20T01:48:00.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0180000000000001e-05 2024-04-20T01:48:00.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.720190+0000", 2024-04-20T01:48:00.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2460999999999999e-05 2024-04-20T01:48:00.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.720196+0000", 2024-04-20T01:48:00.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8079999999999999e-06 2024-04-20T01:48:00.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.726147+0000", 2024-04-20T01:48:00.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059501859999999997 2024-04-20T01:48:00.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.726173+0000", 2024-04-20T01:48:00.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6511000000000001e-05 2024-04-20T01:48:00.056 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.056 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.056 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.056 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s5 47/30 ECSubWrite(tid=80, reqid=osd.8.0:80, at_version=47'65, trim_to=0'0, roll_forward_to=44'61))", 2024-04-20T01:48:00.056 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.720204+0000", 2024-04-20T01:48:00.056 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.312901811, 2024-04-20T01:48:00.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0058777120000000002, 2024-04-20T01:48:00.056 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.056 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.056 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.720204+0000", 2024-04-20T01:48:00.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.720204+0000", 2024-04-20T01:48:00.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.720204+0000", 2024-04-20T01:48:00.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:00.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.720224+0000", 2024-04-20T01:48:00.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9899e-05 2024-04-20T01:48:00.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.720232+0000", 2024-04-20T01:48:00.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2640000000000008e-06 2024-04-20T01:48:00.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.720237+0000", 2024-04-20T01:48:00.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1540000000000001e-06 2024-04-20T01:48:00.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.721389+0000", 2024-04-20T01:48:00.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001151614 2024-04-20T01:48:00.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.721405+0000", 2024-04-20T01:48:00.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.577e-05 2024-04-20T01:48:00.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.726054+0000", 2024-04-20T01:48:00.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046496580000000001 2024-04-20T01:48:00.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.726082+0000", 2024-04-20T01:48:00.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7770999999999999e-05 2024-04-20T01:48:00.061 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.061 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.061 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.061 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s1 47/30 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=47'35, trim_to=0'0, roll_forward_to=44'31))", 2024-04-20T01:48:00.061 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.723047+0000", 2024-04-20T01:48:00.061 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.310059452000001, 2024-04-20T01:48:00.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0061917459999999997, 2024-04-20T01:48:00.062 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.062 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.062 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723047+0000", 2024-04-20T01:48:00.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723047+0000", 2024-04-20T01:48:00.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723046+0000", 2024-04-20T01:48:00.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:00.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723083+0000", 2024-04-20T01:48:00.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6851999999999999e-05 2024-04-20T01:48:00.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723100+0000", 2024-04-20T01:48:00.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6867e-05 2024-04-20T01:48:00.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723109+0000", 2024-04-20T01:48:00.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3710000000000006e-06 2024-04-20T01:48:00.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723843+0000", 2024-04-20T01:48:00.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00073401100000000002 2024-04-20T01:48:00.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723850+0000", 2024-04-20T01:48:00.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3259999999999998e-06 2024-04-20T01:48:00.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.729211+0000", 2024-04-20T01:48:00.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053608049999999997 2024-04-20T01:48:00.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.729239+0000", 2024-04-20T01:48:00.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7461e-05 2024-04-20T01:48:00.065 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.065 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.065 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.066 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s3 48/30 ECSubWrite(tid=44, reqid=osd.5.0:44, at_version=48'33, trim_to=0'0, roll_forward_to=47'31))", 2024-04-20T01:48:00.066 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.715979+0000", 2024-04-20T01:48:00.066 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.317126869999999, 2024-04-20T01:48:00.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095668509999999995, 2024-04-20T01:48:00.066 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.066 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.066 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.715979+0000", 2024-04-20T01:48:00.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.715979+0000", 2024-04-20T01:48:00.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.715979+0000", 2024-04-20T01:48:00.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:00.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716004+0000", 2024-04-20T01:48:00.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4918000000000001e-05 2024-04-20T01:48:00.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716017+0000", 2024-04-20T01:48:00.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3108999999999999e-05 2024-04-20T01:48:00.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716026+0000", 2024-04-20T01:48:00.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3470000000000005e-06 2024-04-20T01:48:00.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716096+0000", 2024-04-20T01:48:00.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9337000000000004e-05 2024-04-20T01:48:00.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716107+0000", 2024-04-20T01:48:00.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1667e-05 2024-04-20T01:48:00.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.725516+0000", 2024-04-20T01:48:00.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094081879999999996 2024-04-20T01:48:00.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.725546+0000", 2024-04-20T01:48:00.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0611999999999998e-05 2024-04-20T01:48:00.069 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.070 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.070 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.070 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs3 48/30 ECSubWrite(tid=41, reqid=osd.14.0:41, at_version=48'35, trim_to=0'0, roll_forward_to=47'33))", 2024-04-20T01:48:00.070 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.716195+0000", 2024-04-20T01:48:00.070 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.316910935999999, 2024-04-20T01:48:00.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094591420000000002, 2024-04-20T01:48:00.070 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.070 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.070 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716195+0000", 2024-04-20T01:48:00.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716195+0000", 2024-04-20T01:48:00.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716195+0000", 2024-04-20T01:48:00.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:00.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716230+0000", 2024-04-20T01:48:00.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4530000000000003e-05 2024-04-20T01:48:00.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716237+0000", 2024-04-20T01:48:00.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7069999999999992e-06 2024-04-20T01:48:00.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716244+0000", 2024-04-20T01:48:00.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4189999999999999e-06 2024-04-20T01:48:00.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716295+0000", 2024-04-20T01:48:00.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1379999999999999e-05 2024-04-20T01:48:00.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716306+0000", 2024-04-20T01:48:00.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1109e-05 2024-04-20T01:48:00.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.725623+0000", 2024-04-20T01:48:00.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093171590000000002 2024-04-20T01:48:00.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.725654+0000", 2024-04-20T01:48:00.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1019999999999998e-05 2024-04-20T01:48:00.074 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.074 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.074 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.074 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.as4 48/30 ECSubWrite(tid=33, reqid=osd.6.0:33, at_version=48'14, trim_to=0'0, roll_forward_to=47'12))", 2024-04-20T01:48:00.074 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.716747+0000", 2024-04-20T01:48:00.074 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.316359743, 2024-04-20T01:48:00.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0088926330000000005, 2024-04-20T01:48:00.074 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.074 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.074 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716747+0000", 2024-04-20T01:48:00.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716747+0000", 2024-04-20T01:48:00.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716746+0000", 2024-04-20T01:48:00.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:00.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716765+0000", 2024-04-20T01:48:00.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8810000000000001e-05 2024-04-20T01:48:00.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716775+0000", 2024-04-20T01:48:00.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8660000000000001e-06 2024-04-20T01:48:00.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716782+0000", 2024-04-20T01:48:00.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2420000000000003e-06 2024-04-20T01:48:00.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716836+0000", 2024-04-20T01:48:00.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4197000000000003e-05 2024-04-20T01:48:00.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716848+0000", 2024-04-20T01:48:00.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1093e-05 2024-04-20T01:48:00.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.725610+0000", 2024-04-20T01:48:00.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008762766 2024-04-20T01:48:00.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.725639+0000", 2024-04-20T01:48:00.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8810999999999999e-05 2024-04-20T01:48:00.078 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.078 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.078 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.078 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.1s5 48/30 ECSubWrite(tid=119, reqid=osd.4.0:119, at_version=48'14, trim_to=0'0, roll_forward_to=47'12))", 2024-04-20T01:48:00.078 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.717069+0000", 2024-04-20T01:48:00.078 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.316036883000001, 2024-04-20T01:48:00.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086177379999999998, 2024-04-20T01:48:00.078 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.078 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.079 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.717069+0000", 2024-04-20T01:48:00.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.717069+0000", 2024-04-20T01:48:00.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.717069+0000", 2024-04-20T01:48:00.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:00.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.717098+0000", 2024-04-20T01:48:00.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.879e-05 2024-04-20T01:48:00.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.717110+0000", 2024-04-20T01:48:00.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1613999999999999e-05 2024-04-20T01:48:00.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.717119+0000", 2024-04-20T01:48:00.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7929999999999992e-06 2024-04-20T01:48:00.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.717198+0000", 2024-04-20T01:48:00.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8422000000000002e-05 2024-04-20T01:48:00.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.717208+0000", 2024-04-20T01:48:00.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0552e-05 2024-04-20T01:48:00.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.725670+0000", 2024-04-20T01:48:00.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008461596 2024-04-20T01:48:00.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.725687+0000", 2024-04-20T01:48:00.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7189000000000001e-05 2024-04-20T01:48:00.082 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.083 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.083 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.083 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s1 48/30 ECSubWrite(tid=255, reqid=osd.13.0:253, at_version=0'0, trim_to=0'0, roll_forward_to=48'44))", 2024-04-20T01:48:00.083 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.721163+0000", 2024-04-20T01:48:00.083 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.311942802000001, 2024-04-20T01:48:00.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0071784350000000004, 2024-04-20T01:48:00.083 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.083 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.083 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.721163+0000", 2024-04-20T01:48:00.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.721163+0000", 2024-04-20T01:48:00.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.721163+0000", 2024-04-20T01:48:00.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:00.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.721177+0000", 2024-04-20T01:48:00.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4463e-05 2024-04-20T01:48:00.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.721189+0000", 2024-04-20T01:48:00.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2079000000000001e-05 2024-04-20T01:48:00.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.721196+0000", 2024-04-20T01:48:00.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1810000000000004e-06 2024-04-20T01:48:00.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.721251+0000", 2024-04-20T01:48:00.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4061999999999998e-05 2024-04-20T01:48:00.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.721262+0000", 2024-04-20T01:48:00.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1579999999999999e-05 2024-04-20T01:48:00.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.728298+0000", 2024-04-20T01:48:00.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0070358840000000001 2024-04-20T01:48:00.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.728342+0000", 2024-04-20T01:48:00.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3906999999999998e-05 2024-04-20T01:48:00.087 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.087 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.087 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.087 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:443 3.0s0 3:06dd530f:::smithi16579500-30:head [append 0~720896] snapc 15=[15,14,13,b] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:00.087 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.714463+0000", 2024-04-20T01:48:00.087 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.318643003, 2024-04-20T01:48:00.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014549549, 2024-04-20T01:48:00.088 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.088 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:00.088 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:00.088 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:00.088 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:00.088 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 443 2024-04-20T01:48:00.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.088 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714463+0000", 2024-04-20T01:48:00.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714463+0000", 2024-04-20T01:48:00.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714461+0000", 2024-04-20T01:48:00.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:00.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714916+0000", 2024-04-20T01:48:00.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00045487499999999998 2024-04-20T01:48:00.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714934+0000", 2024-04-20T01:48:00.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7586000000000001e-05 2024-04-20T01:48:00.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714962+0000", 2024-04-20T01:48:00.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7976000000000001e-05 2024-04-20T01:48:00.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.715010+0000", 2024-04-20T01:48:00.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8535999999999997e-05 2024-04-20T01:48:00.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:00.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.715016+0000", 2024-04-20T01:48:00.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8309999999999997e-06 2024-04-20T01:48:00.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717469+0000", 2024-04-20T01:48:00.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024527529999999998 2024-04-20T01:48:00.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:00.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717568+0000", 2024-04-20T01:48:00.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8708000000000004e-05 2024-04-20T01:48:00.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718717+0000", 2024-04-20T01:48:00.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0011493510000000001 2024-04-20T01:48:00.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724443+0000", 2024-04-20T01:48:00.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057254280000000003 2024-04-20T01:48:00.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:00.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.728929+0000", 2024-04-20T01:48:00.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044864919999999999 2024-04-20T01:48:00.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.729013+0000", 2024-04-20T01:48:00.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3770999999999995e-05 2024-04-20T01:48:00.093 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.093 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.093 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.093 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:444 3.0s0 3:06dd530f:::smithi16579500-30:head [append 0~442368] snapc 15=[15,14,13,b] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:00.093 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.716986+0000", 2024-04-20T01:48:00.093 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.3161207389999996, 2024-04-20T01:48:00.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014745457999999999, 2024-04-20T01:48:00.094 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.094 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:00.094 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:00.094 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:00.094 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:00.094 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 444 2024-04-20T01:48:00.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.094 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.716986+0000", 2024-04-20T01:48:00.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.716986+0000", 2024-04-20T01:48:00.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.716984+0000", 2024-04-20T01:48:00.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:00.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717309+0000", 2024-04-20T01:48:00.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032479500000000001 2024-04-20T01:48:00.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717321+0000", 2024-04-20T01:48:00.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2177e-05 2024-04-20T01:48:00.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717330+0000", 2024-04-20T01:48:00.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0399999999999998e-06 2024-04-20T01:48:00.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.720236+0000", 2024-04-20T01:48:00.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002905992 2024-04-20T01:48:00.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:00.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.720367+0000", 2024-04-20T01:48:00.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000130488 2024-04-20T01:48:00.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721328+0000", 2024-04-20T01:48:00.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00096118699999999996 2024-04-20T01:48:00.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.729125+0000", 2024-04-20T01:48:00.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077972429999999997 2024-04-20T01:48:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731687+0000", 2024-04-20T01:48:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00256201 2024-04-20T01:48:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731731+0000", 2024-04-20T01:48:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4005999999999998e-05 2024-04-20T01:48:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:445 3.0s0 3:06dd530f:::smithi16579500-30:head [append 0~327680] snapc 15=[15,14,13,b] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.718952+0000", 2024-04-20T01:48:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.3141540750000003, 2024-04-20T01:48:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012930084, 2024-04-20T01:48:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 445 2024-04-20T01:48:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718952+0000", 2024-04-20T01:48:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718952+0000", 2024-04-20T01:48:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718950+0000", 2024-04-20T01:48:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719225+0000", 2024-04-20T01:48:00.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00027498300000000002 2024-04-20T01:48:00.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719241+0000", 2024-04-20T01:48:00.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6036999999999999e-05 2024-04-20T01:48:00.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719251+0000", 2024-04-20T01:48:00.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.027e-05 2024-04-20T01:48:00.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722012+0000", 2024-04-20T01:48:00.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027604309999999998 2024-04-20T01:48:00.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:00.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722121+0000", 2024-04-20T01:48:00.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109207 2024-04-20T01:48:00.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722804+0000", 2024-04-20T01:48:00.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00068250299999999999 2024-04-20T01:48:00.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.729128+0000", 2024-04-20T01:48:00.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0063243079999999998 2024-04-20T01:48:00.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:00.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731817+0000", 2024-04-20T01:48:00.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0026888239999999998 2024-04-20T01:48:00.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731882+0000", 2024-04-20T01:48:00.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5499000000000004e-05 2024-04-20T01:48:00.103 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.103 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.104 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.104 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:446 3.0s0 3:06dd530f:::smithi16579500-30:head [setxattr _header (57) in=7b] snapc 15=[15,14,13,b] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:00.104 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.719261+0000", 2024-04-20T01:48:00.104 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.3138457380000004, 2024-04-20T01:48:00.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012992831, 2024-04-20T01:48:00.104 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.104 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:00.104 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:00.104 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:00.104 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:00.104 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 446 2024-04-20T01:48:00.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.104 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719261+0000", 2024-04-20T01:48:00.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719261+0000", 2024-04-20T01:48:00.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719260+0000", 2024-04-20T01:48:00.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:00.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719264+0000", 2024-04-20T01:48:00.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6179999999999997e-06 2024-04-20T01:48:00.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719271+0000", 2024-04-20T01:48:00.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9949999999999999e-06 2024-04-20T01:48:00.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719277+0000", 2024-04-20T01:48:00.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6999999999999996e-06 2024-04-20T01:48:00.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723396+0000", 2024-04-20T01:48:00.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0041186599999999997 2024-04-20T01:48:00.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:00.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723458+0000", 2024-04-20T01:48:00.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2786000000000005e-05 2024-04-20T01:48:00.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723867+0000", 2024-04-20T01:48:00.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040805300000000002 2024-04-20T01:48:00.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731913+0000", 2024-04-20T01:48:00.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080461330000000005 2024-04-20T01:48:00.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:00.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731951+0000", 2024-04-20T01:48:00.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8707000000000001e-05 2024-04-20T01:48:00.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.732253+0000", 2024-04-20T01:48:00.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030198999999999997 2024-04-20T01:48:00.109 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.109 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.109 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.109 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:447 3.0s0 3:06dd530f:::smithi16579500-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:00.109 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.719282+0000", 2024-04-20T01:48:00.109 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.3138241710000003, 2024-04-20T01:48:00.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01483317, 2024-04-20T01:48:00.109 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.109 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:00.109 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:00.109 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:00.109 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:00.110 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 447 2024-04-20T01:48:00.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.110 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719282+0000", 2024-04-20T01:48:00.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719282+0000", 2024-04-20T01:48:00.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719282+0000", 2024-04-20T01:48:00.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:00.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719294+0000", 2024-04-20T01:48:00.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2747e-05 2024-04-20T01:48:00.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719302+0000", 2024-04-20T01:48:00.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0960000000000001e-06 2024-04-20T01:48:00.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719308+0000", 2024-04-20T01:48:00.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.066e-06 2024-04-20T01:48:00.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724217+0000", 2024-04-20T01:48:00.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0049083549999999997 2024-04-20T01:48:00.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:00.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724333+0000", 2024-04-20T01:48:00.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011662099999999999 2024-04-20T01:48:00.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.732307+0000", 2024-04-20T01:48:00.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0079733059999999994 2024-04-20T01:48:00.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:00.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.732409+0000", 2024-04-20T01:48:00.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010193599999999999 2024-04-20T01:48:00.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.734115+0000", 2024-04-20T01:48:00.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0017065520000000001 2024-04-20T01:48:00.113 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.113 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.113 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.114 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s3 58/30 ECSubWrite(tid=52, reqid=client.4643.0:489, at_version=58'41, trim_to=0'0, roll_forward_to=55'37))", 2024-04-20T01:48:00.114 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:59.769563+0000", 2024-04-20T01:48:00.114 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.26354337999999999, 2024-04-20T01:48:00.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062852130000000004, 2024-04-20T01:48:00.114 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.114 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.114 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769563+0000", 2024-04-20T01:48:00.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769563+0000", 2024-04-20T01:48:00.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769562+0000", 2024-04-20T01:48:00.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:00.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769663+0000", 2024-04-20T01:48:00.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100729 2024-04-20T01:48:00.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769676+0000", 2024-04-20T01:48:00.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3336000000000001e-05 2024-04-20T01:48:00.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769685+0000", 2024-04-20T01:48:00.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8929999999999994e-06 2024-04-20T01:48:00.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769732+0000", 2024-04-20T01:48:00.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6975999999999999e-05 2024-04-20T01:48:00.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769742+0000", 2024-04-20T01:48:00.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6350000000000001e-06 2024-04-20T01:48:00.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.775826+0000", 2024-04-20T01:48:00.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0060840709999999999 2024-04-20T01:48:00.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.775848+0000", 2024-04-20T01:48:00.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2121e-05 2024-04-20T01:48:00.117 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.117 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.117 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.117 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.118 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.118 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:00.119 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-20T01:48:00.120 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:00.134+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:00.220 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:00.235+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:00.288 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:00.288 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:00.288 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-04-20T01:48:00.288 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:00.298 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-20T01:48:00.321 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:48:00.335+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:48:00.422 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:00.437+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:00.426 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:00.426 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:00.426 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:48:00.426 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:48:00.426 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:00.437 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-20T01:48:00.523 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:00.538+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:00.568 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:00.568 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:48:00.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:48:00.568 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:48:00.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.569 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(osd.8.21:1 3.9s0 3:94399c02:::smithi16579500-3:head [assert-version v4,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:48:00.569 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:37.799515+0000", 2024-04-20T01:48:00.569 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 22.782093659000001, 2024-04-20T01:48:00.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075519259999999996, 2024-04-20T01:48:00.569 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.569 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:00.569 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799515+0000", 2024-04-20T01:48:00.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799515+0000", 2024-04-20T01:48:00.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799513+0000", 2024-04-20T01:48:00.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:00.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799523+0000", 2024-04-20T01:48:00.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4020000000000008e-06 2024-04-20T01:48:00.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799536+0000", 2024-04-20T01:48:00.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3448999999999999e-05 2024-04-20T01:48:00.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799555+0000", 2024-04-20T01:48:00.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9148e-05 2024-04-20T01:48:00.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799607+0000", 2024-04-20T01:48:00.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1898999999999999e-05 2024-04-20T01:48:00.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:00.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.799615+0000", 2024-04-20T01:48:00.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1170000000000004e-06 2024-04-20T01:48:00.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.801155+0000", 2024-04-20T01:48:00.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001539852 2024-04-20T01:48:00.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:00.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.801260+0000", 2024-04-20T01:48:00.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000105079 2024-04-20T01:48:00.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:37.807067+0000", 2024-04-20T01:48:00.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0058068490000000002 2024-04-20T01:48:00.573 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.573 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.573 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.573 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:289 3.9s0 3:94399c02:::smithi16579500-3:head [copy-from ver 6] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:00.573 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:43.807269+0000", 2024-04-20T01:48:00.573 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 16.774339589, 2024-04-20T01:48:00.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016170202000000002, 2024-04-20T01:48:00.574 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.574 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:00.574 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:00.574 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:00.574 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:00.574 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 289 2024-04-20T01:48:00.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.574 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807269+0000", 2024-04-20T01:48:00.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807269+0000", 2024-04-20T01:48:00.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807267+0000", 2024-04-20T01:48:00.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:00.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807280+0000", 2024-04-20T01:48:00.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3227999999999999e-05 2024-04-20T01:48:00.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807303+0000", 2024-04-20T01:48:00.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2991000000000001e-05 2024-04-20T01:48:00.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807318+0000", 2024-04-20T01:48:00.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4321999999999999e-05 2024-04-20T01:48:00.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807385+0000", 2024-04-20T01:48:00.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7403000000000005e-05 2024-04-20T01:48:00.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:00.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807394+0000", 2024-04-20T01:48:00.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7010000000000002e-06 2024-04-20T01:48:00.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.809246+0000", 2024-04-20T01:48:00.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0018521 2024-04-20T01:48:00.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:00.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.809314+0000", 2024-04-20T01:48:00.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7818999999999996e-05 2024-04-20T01:48:00.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.817661+0000", 2024-04-20T01:48:00.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0083471710000000005 2024-04-20T01:48:00.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.819085+0000", 2024-04-20T01:48:00.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0014239739999999999 2024-04-20T01:48:00.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:00.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.823350+0000", 2024-04-20T01:48:00.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042652990000000002 2024-04-20T01:48:00.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.823439+0000", 2024-04-20T01:48:00.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9424000000000003e-05 2024-04-20T01:48:00.579 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.579 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.579 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.579 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:290 3.9s0 3:94399c02:::smithi16579500-3:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:00.579 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:43.807329+0000", 2024-04-20T01:48:00.579 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 16.774279667999998, 2024-04-20T01:48:00.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011459461000000001, 2024-04-20T01:48:00.579 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.579 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:00.579 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:00.579 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:00.579 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:00.580 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 290 2024-04-20T01:48:00.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.580 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807329+0000", 2024-04-20T01:48:00.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807329+0000", 2024-04-20T01:48:00.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807329+0000", 2024-04-20T01:48:00.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:00.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807332+0000", 2024-04-20T01:48:00.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.349e-06 2024-04-20T01:48:00.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807343+0000", 2024-04-20T01:48:00.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1270999999999999e-05 2024-04-20T01:48:00.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807347+0000", 2024-04-20T01:48:00.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-04-20T01:48:00.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807453+0000", 2024-04-20T01:48:00.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106162 2024-04-20T01:48:00.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:00.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807460+0000", 2024-04-20T01:48:00.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9809999999999997e-06 2024-04-20T01:48:00.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.809441+0000", 2024-04-20T01:48:00.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019806060000000002 2024-04-20T01:48:00.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:00.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.809484+0000", 2024-04-20T01:48:00.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3019000000000003e-05 2024-04-20T01:48:00.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.818614+0000", 2024-04-20T01:48:00.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091302699999999994 2024-04-20T01:48:00.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:00.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.818713+0000", 2024-04-20T01:48:00.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8776000000000004e-05 2024-04-20T01:48:00.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.818789+0000", 2024-04-20T01:48:00.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5944000000000003e-05 2024-04-20T01:48:00.584 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.584 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.584 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.584 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 47/30 ECSubWrite(tid=38, reqid=osd.14.0:38, at_version=47'31, trim_to=0'0, roll_forward_to=46'29))", 2024-04-20T01:48:00.584 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.711869+0000", 2024-04-20T01:48:00.585 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.86973978, 2024-04-20T01:48:00.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016896644999999998, 2024-04-20T01:48:00.585 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.585 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.585 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711869+0000", 2024-04-20T01:48:00.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711869+0000", 2024-04-20T01:48:00.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711868+0000", 2024-04-20T01:48:00.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:00.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711907+0000", 2024-04-20T01:48:00.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8828000000000002e-05 2024-04-20T01:48:00.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711927+0000", 2024-04-20T01:48:00.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9656e-05 2024-04-20T01:48:00.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711937+0000", 2024-04-20T01:48:00.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0277e-05 2024-04-20T01:48:00.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711991+0000", 2024-04-20T01:48:00.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3980000000000002e-05 2024-04-20T01:48:00.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.712002+0000", 2024-04-20T01:48:00.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719e-05 2024-04-20T01:48:00.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.728747+0000", 2024-04-20T01:48:00.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016745466000000001 2024-04-20T01:48:00.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.728766+0000", 2024-04-20T01:48:00.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8607999999999999e-05 2024-04-20T01:48:00.588 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.588 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.588 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.588 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 47/30 ECSubWrite(tid=39, reqid=osd.14.0:39, at_version=47'33, trim_to=0'0, roll_forward_to=46'29))", 2024-04-20T01:48:00.589 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.713021+0000", 2024-04-20T01:48:00.589 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.868587572999999, 2024-04-20T01:48:00.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020945089, 2024-04-20T01:48:00.589 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.589 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.589 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713021+0000", 2024-04-20T01:48:00.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713021+0000", 2024-04-20T01:48:00.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713020+0000", 2024-04-20T01:48:00.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:00.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713046+0000", 2024-04-20T01:48:00.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5554e-05 2024-04-20T01:48:00.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713064+0000", 2024-04-20T01:48:00.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7635999999999999e-05 2024-04-20T01:48:00.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713074+0000", 2024-04-20T01:48:00.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9529999999999994e-06 2024-04-20T01:48:00.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713140+0000", 2024-04-20T01:48:00.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6235999999999994e-05 2024-04-20T01:48:00.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713155+0000", 2024-04-20T01:48:00.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5647000000000001e-05 2024-04-20T01:48:00.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733953+0000", 2024-04-20T01:48:00.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020797936 2024-04-20T01:48:00.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733966+0000", 2024-04-20T01:48:00.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3067999999999999e-05 2024-04-20T01:48:00.592 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.592 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.592 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.593 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s1 47/30 ECSubWrite(tid=30, reqid=osd.3.0:30, at_version=47'19, trim_to=0'0, roll_forward_to=44'17))", 2024-04-20T01:48:00.593 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.713663+0000", 2024-04-20T01:48:00.593 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.867945813, 2024-04-20T01:48:00.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0202712, 2024-04-20T01:48:00.593 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.593 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.593 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713663+0000", 2024-04-20T01:48:00.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713663+0000", 2024-04-20T01:48:00.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713662+0000", 2024-04-20T01:48:00.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:00.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713693+0000", 2024-04-20T01:48:00.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1031999999999997e-05 2024-04-20T01:48:00.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713712+0000", 2024-04-20T01:48:00.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8615999999999999e-05 2024-04-20T01:48:00.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713726+0000", 2024-04-20T01:48:00.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4477e-05 2024-04-20T01:48:00.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713779+0000", 2024-04-20T01:48:00.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3090999999999997e-05 2024-04-20T01:48:00.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713791+0000", 2024-04-20T01:48:00.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1921e-05 2024-04-20T01:48:00.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733906+0000", 2024-04-20T01:48:00.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020114522999999999 2024-04-20T01:48:00.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733934+0000", 2024-04-20T01:48:00.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8357999999999999e-05 2024-04-20T01:48:00.596 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.596 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.596 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.597 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 47/30 ECSubWrite(tid=39, reqid=osd.5.0:39, at_version=47'27, trim_to=0'0, roll_forward_to=46'25))", 2024-04-20T01:48:00.597 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.718432+0000", 2024-04-20T01:48:00.597 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.863177156000001, 2024-04-20T01:48:00.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015468671, 2024-04-20T01:48:00.597 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.597 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.597 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718432+0000", 2024-04-20T01:48:00.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718432+0000", 2024-04-20T01:48:00.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718431+0000", 2024-04-20T01:48:00.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:00.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718455+0000", 2024-04-20T01:48:00.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3085e-05 2024-04-20T01:48:00.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718471+0000", 2024-04-20T01:48:00.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6172999999999999e-05 2024-04-20T01:48:00.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718480+0000", 2024-04-20T01:48:00.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5719999999999992e-06 2024-04-20T01:48:00.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718592+0000", 2024-04-20T01:48:00.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111517 2024-04-20T01:48:00.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718602+0000", 2024-04-20T01:48:00.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.064e-05 2024-04-20T01:48:00.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733869+0000", 2024-04-20T01:48:00.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015266798 2024-04-20T01:48:00.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733900+0000", 2024-04-20T01:48:00.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.116e-05 2024-04-20T01:48:00.600 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.600 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.600 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.601 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 47/30 ECSubWrite(tid=40, reqid=osd.5.0:40, at_version=47'29, trim_to=0'0, roll_forward_to=46'25))", 2024-04-20T01:48:00.601 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.719277+0000", 2024-04-20T01:48:00.601 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.86233146, 2024-04-20T01:48:00.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017279452000000001, 2024-04-20T01:48:00.601 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.601 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.601 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719277+0000", 2024-04-20T01:48:00.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719277+0000", 2024-04-20T01:48:00.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719277+0000", 2024-04-20T01:48:00.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:00.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719306+0000", 2024-04-20T01:48:00.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9771999999999999e-05 2024-04-20T01:48:00.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719322+0000", 2024-04-20T01:48:00.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6035000000000001e-05 2024-04-20T01:48:00.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719331+0000", 2024-04-20T01:48:00.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1130000000000007e-06 2024-04-20T01:48:00.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.728810+0000", 2024-04-20T01:48:00.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094789090000000006 2024-04-20T01:48:00.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.728819+0000", 2024-04-20T01:48:00.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8890000000000008e-06 2024-04-20T01:48:00.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.736530+0000", 2024-04-20T01:48:00.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077106129999999998 2024-04-20T01:48:00.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.736557+0000", 2024-04-20T01:48:00.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6974999999999998e-05 2024-04-20T01:48:00.604 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.604 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.605 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.605 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 47/30 ECSubWrite(tid=240, reqid=osd.13.0:240, at_version=47'33, trim_to=0'0, roll_forward_to=44'31))", 2024-04-20T01:48:00.605 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.722777+0000", 2024-04-20T01:48:00.605 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.858831746, 2024-04-20T01:48:00.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008649894, 2024-04-20T01:48:00.605 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.605 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.605 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722777+0000", 2024-04-20T01:48:00.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722777+0000", 2024-04-20T01:48:00.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722777+0000", 2024-04-20T01:48:00.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:00.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722805+0000", 2024-04-20T01:48:00.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7881e-05 2024-04-20T01:48:00.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722820+0000", 2024-04-20T01:48:00.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5387e-05 2024-04-20T01:48:00.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722831+0000", 2024-04-20T01:48:00.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813e-05 2024-04-20T01:48:00.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722915+0000", 2024-04-20T01:48:00.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3578000000000005e-05 2024-04-20T01:48:00.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722929+0000", 2024-04-20T01:48:00.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4256999999999999e-05 2024-04-20T01:48:00.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.731402+0000", 2024-04-20T01:48:00.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008472785 2024-04-20T01:48:00.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.731427+0000", 2024-04-20T01:48:00.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5355e-05 2024-04-20T01:48:00.609 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.610 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.610 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.610 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 47/30 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=47'35, trim_to=0'0, roll_forward_to=44'31))", 2024-04-20T01:48:00.610 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.723066+0000", 2024-04-20T01:48:00.610 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.858542999999999, 2024-04-20T01:48:00.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010874728, 2024-04-20T01:48:00.610 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.610 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.610 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723066+0000", 2024-04-20T01:48:00.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723066+0000", 2024-04-20T01:48:00.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723065+0000", 2024-04-20T01:48:00.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:00.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723089+0000", 2024-04-20T01:48:00.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3529000000000001e-05 2024-04-20T01:48:00.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723101+0000", 2024-04-20T01:48:00.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2143e-05 2024-04-20T01:48:00.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723110+0000", 2024-04-20T01:48:00.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9809999999999992e-06 2024-04-20T01:48:00.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.725642+0000", 2024-04-20T01:48:00.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002532337 2024-04-20T01:48:00.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.725655+0000", 2024-04-20T01:48:00.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2707e-05 2024-04-20T01:48:00.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733911+0000", 2024-04-20T01:48:00.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082559369999999997 2024-04-20T01:48:00.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733941+0000", 2024-04-20T01:48:00.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9798000000000002e-05 2024-04-20T01:48:00.614 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.614 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.614 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.614 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:396 3.9s0 3:9a2fbd15:::smithi16579500-1:head [copy-from ver 32] snapc d=[c,b] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-20T01:48:00.614 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.746088+0000", 2024-04-20T01:48:00.614 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.835520821999999, 2024-04-20T01:48:00.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01355685, 2024-04-20T01:48:00.614 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.614 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:00.614 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:00.615 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:00.615 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:00.615 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 396 2024-04-20T01:48:00.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.615 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746088+0000", 2024-04-20T01:48:00.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746088+0000", 2024-04-20T01:48:00.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746086+0000", 2024-04-20T01:48:00.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:00.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746094+0000", 2024-04-20T01:48:00.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.272e-06 2024-04-20T01:48:00.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746109+0000", 2024-04-20T01:48:00.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5563999999999999e-05 2024-04-20T01:48:00.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746131+0000", 2024-04-20T01:48:00.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1515999999999999e-05 2024-04-20T01:48:00.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746178+0000", 2024-04-20T01:48:00.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7472000000000002e-05 2024-04-20T01:48:00.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:00.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746261+0000", 2024-04-20T01:48:00.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2412999999999999e-05 2024-04-20T01:48:00.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.752312+0000", 2024-04-20T01:48:00.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0060515120000000002 2024-04-20T01:48:00.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.754299+0000", 2024-04-20T01:48:00.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019870690000000002 2024-04-20T01:48:00.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:00.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.759594+0000", 2024-04-20T01:48:00.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0052944070000000001 2024-04-20T01:48:00.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.759645+0000", 2024-04-20T01:48:00.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1236000000000002e-05 2024-04-20T01:48:00.620 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.620 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.620 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.620 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:397 3.9s0 3:9a2fbd15:::smithi16579500-1:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-04-20T01:48:00.620 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.746146+0000", 2024-04-20T01:48:00.620 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.835463281999999, 2024-04-20T01:48:00.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0073986349999999998, 2024-04-20T01:48:00.621 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.621 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:00.621 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:00.621 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:00.621 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:00.621 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 397 2024-04-20T01:48:00.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.621 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746146+0000", 2024-04-20T01:48:00.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746146+0000", 2024-04-20T01:48:00.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746145+0000", 2024-04-20T01:48:00.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:00.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746149+0000", 2024-04-20T01:48:00.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1130000000000003e-06 2024-04-20T01:48:00.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746158+0000", 2024-04-20T01:48:00.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1770000000000004e-06 2024-04-20T01:48:00.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746165+0000", 2024-04-20T01:48:00.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9979999999999997e-06 2024-04-20T01:48:00.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746395+0000", 2024-04-20T01:48:00.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000229457 2024-04-20T01:48:00.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:00.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746430+0000", 2024-04-20T01:48:00.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5215000000000001e-05 2024-04-20T01:48:00.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.753309+0000", 2024-04-20T01:48:00.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0068791850000000003 2024-04-20T01:48:00.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:00.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.753422+0000", 2024-04-20T01:48:00.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011273900000000001 2024-04-20T01:48:00.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.753544+0000", 2024-04-20T01:48:00.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0001225 2024-04-20T01:48:00.626 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.626 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.626 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.626 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 48/30 ECSubWrite(tid=44, reqid=osd.5.0:44, at_version=48'33, trim_to=0'0, roll_forward_to=47'31))", 2024-04-20T01:48:00.626 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.716311+0000", 2024-04-20T01:48:00.626 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.865297417000001, 2024-04-20T01:48:00.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093701210000000004, 2024-04-20T01:48:00.627 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.627 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.627 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716311+0000", 2024-04-20T01:48:00.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716311+0000", 2024-04-20T01:48:00.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716311+0000", 2024-04-20T01:48:00.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:00.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716336+0000", 2024-04-20T01:48:00.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5216e-05 2024-04-20T01:48:00.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716348+0000", 2024-04-20T01:48:00.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1369000000000001e-05 2024-04-20T01:48:00.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716358+0000", 2024-04-20T01:48:00.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9019999999999994e-06 2024-04-20T01:48:00.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716410+0000", 2024-04-20T01:48:00.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1974000000000001e-05 2024-04-20T01:48:00.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716419+0000", 2024-04-20T01:48:00.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4439999999999997e-06 2024-04-20T01:48:00.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.725652+0000", 2024-04-20T01:48:00.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092323909999999995 2024-04-20T01:48:00.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.725682+0000", 2024-04-20T01:48:00.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0049999999999999e-05 2024-04-20T01:48:00.631 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.631 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.631 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.631 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 48/30 ECSubWrite(tid=256, reqid=client.4643.0:403, at_version=48'46, trim_to=0'0, roll_forward_to=48'44))", 2024-04-20T01:48:00.631 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.789372+0000", 2024-04-20T01:48:00.631 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.792236840999999, 2024-04-20T01:48:00.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086922490000000008, 2024-04-20T01:48:00.632 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.632 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.632 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789372+0000", 2024-04-20T01:48:00.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789372+0000", 2024-04-20T01:48:00.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.633 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789371+0000", 2024-04-20T01:48:00.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:00.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.633 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789968+0000", 2024-04-20T01:48:00.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00059678300000000004 2024-04-20T01:48:00.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.634 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.634 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789979+0000", 2024-04-20T01:48:00.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1499e-05 2024-04-20T01:48:00.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.634 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.634 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789986+0000", 2024-04-20T01:48:00.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2179999999999996e-06 2024-04-20T01:48:00.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.634 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.790029+0000", 2024-04-20T01:48:00.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2876000000000001e-05 2024-04-20T01:48:00.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.790036+0000", 2024-04-20T01:48:00.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7870000000000002e-06 2024-04-20T01:48:00.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.798040+0000", 2024-04-20T01:48:00.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080038720000000004 2024-04-20T01:48:00.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.798064+0000", 2024-04-20T01:48:00.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4037999999999999e-05 2024-04-20T01:48:00.636 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.636 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.636 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.636 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 48/30 ECSubWrite(tid=257, reqid=client.4643.0:404, at_version=48'47, trim_to=0'0, roll_forward_to=48'44))", 2024-04-20T01:48:00.636 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.791622+0000", 2024-04-20T01:48:00.636 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.789986855, 2024-04-20T01:48:00.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0088453630000000002, 2024-04-20T01:48:00.637 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.637 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.637 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.791622+0000", 2024-04-20T01:48:00.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.791622+0000", 2024-04-20T01:48:00.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.791621+0000", 2024-04-20T01:48:00.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:00.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792215+0000", 2024-04-20T01:48:00.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000593689 2024-04-20T01:48:00.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792227+0000", 2024-04-20T01:48:00.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2037e-05 2024-04-20T01:48:00.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792236+0000", 2024-04-20T01:48:00.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8240000000000006e-06 2024-04-20T01:48:00.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792538+0000", 2024-04-20T01:48:00.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000301668 2024-04-20T01:48:00.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792549+0000", 2024-04-20T01:48:00.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0915e-05 2024-04-20T01:48:00.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.800443+0000", 2024-04-20T01:48:00.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0078945300000000003 2024-04-20T01:48:00.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.800467+0000", 2024-04-20T01:48:00.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4309000000000001e-05 2024-04-20T01:48:00.641 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.641 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.641 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.641 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 48/30 ECSubWrite(tid=258, reqid=client.4643.0:405, at_version=48'48, trim_to=0'0, roll_forward_to=48'44))", 2024-04-20T01:48:00.642 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.792706+0000", 2024-04-20T01:48:00.642 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.7889026, 2024-04-20T01:48:00.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.007791498, 2024-04-20T01:48:00.642 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.642 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.642 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792706+0000", 2024-04-20T01:48:00.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.643 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792706+0000", 2024-04-20T01:48:00.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.643 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792706+0000", 2024-04-20T01:48:00.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:00.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792834+0000", 2024-04-20T01:48:00.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012846600000000001 2024-04-20T01:48:00.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792846+0000", 2024-04-20T01:48:00.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2018000000000001e-05 2024-04-20T01:48:00.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792855+0000", 2024-04-20T01:48:00.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6149999999999997e-06 2024-04-20T01:48:00.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.793183+0000", 2024-04-20T01:48:00.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000327752 2024-04-20T01:48:00.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.793191+0000", 2024-04-20T01:48:00.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8009999999999993e-06 2024-04-20T01:48:00.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.800472+0000", 2024-04-20T01:48:00.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0072811059999999999 2024-04-20T01:48:00.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.800498+0000", 2024-04-20T01:48:00.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5320000000000002e-05 2024-04-20T01:48:00.646 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.646 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.646 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.647 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s1 53/30 ECSubWrite(tid=33, reqid=client.4643.0:444, at_version=53'22, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:00.647 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.721415+0000", 2024-04-20T01:48:00.647 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.8601940780000001, 2024-04-20T01:48:00.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097484879999999996, 2024-04-20T01:48:00.647 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.647 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.647 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721415+0000", 2024-04-20T01:48:00.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721415+0000", 2024-04-20T01:48:00.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721413+0000", 2024-04-20T01:48:00.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:00.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721522+0000", 2024-04-20T01:48:00.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108445 2024-04-20T01:48:00.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721540+0000", 2024-04-20T01:48:00.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8128e-05 2024-04-20T01:48:00.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721555+0000", 2024-04-20T01:48:00.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5228999999999999e-05 2024-04-20T01:48:00.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721639+0000", 2024-04-20T01:48:00.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3803999999999999e-05 2024-04-20T01:48:00.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721650+0000", 2024-04-20T01:48:00.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0702e-05 2024-04-20T01:48:00.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731131+0000", 2024-04-20T01:48:00.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094816929999999994 2024-04-20T01:48:00.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731163+0000", 2024-04-20T01:48:00.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1946999999999998e-05 2024-04-20T01:48:00.651 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.651 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.651 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.651 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s1 53/30 ECSubWrite(tid=34, reqid=client.4643.0:445, at_version=53'23, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:00.651 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.722886+0000", 2024-04-20T01:48:00.652 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.8587233770000005, 2024-04-20T01:48:00.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008294961, 2024-04-20T01:48:00.652 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.652 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.652 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722886+0000", 2024-04-20T01:48:00.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722886+0000", 2024-04-20T01:48:00.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722885+0000", 2024-04-20T01:48:00.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:00.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722985+0000", 2024-04-20T01:48:00.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100473 2024-04-20T01:48:00.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723011+0000", 2024-04-20T01:48:00.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5755e-05 2024-04-20T01:48:00.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723021+0000", 2024-04-20T01:48:00.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6900000000000004e-06 2024-04-20T01:48:00.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.655 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.655 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723071+0000", 2024-04-20T01:48:00.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.024e-05 2024-04-20T01:48:00.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.655 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.655 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723083+0000", 2024-04-20T01:48:00.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1858999999999999e-05 2024-04-20T01:48:00.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.655 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731169+0000", 2024-04-20T01:48:00.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080867419999999992 2024-04-20T01:48:00.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731180+0000", 2024-04-20T01:48:00.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1097000000000001e-05 2024-04-20T01:48:00.656 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.656 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.656 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.657 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 58/30 ECSubWrite(tid=52, reqid=client.4643.0:489, at_version=58'41, trim_to=0'0, roll_forward_to=55'37))", 2024-04-20T01:48:00.657 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:59.769343+0000", 2024-04-20T01:48:00.657 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.81226573999999996, 2024-04-20T01:48:00.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094380780000000008, 2024-04-20T01:48:00.657 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.657 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.657 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769343+0000", 2024-04-20T01:48:00.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769343+0000", 2024-04-20T01:48:00.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769343+0000", 2024-04-20T01:48:00.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:00.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769462+0000", 2024-04-20T01:48:00.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011929 2024-04-20T01:48:00.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.659 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.659 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769474+0000", 2024-04-20T01:48:00.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1857e-05 2024-04-20T01:48:00.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769482+0000", 2024-04-20T01:48:00.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0809999999999994e-06 2024-04-20T01:48:00.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769531+0000", 2024-04-20T01:48:00.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9168000000000001e-05 2024-04-20T01:48:00.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769538+0000", 2024-04-20T01:48:00.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5930000000000002e-06 2024-04-20T01:48:00.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.778749+0000", 2024-04-20T01:48:00.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092115670000000004 2024-04-20T01:48:00.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.778781+0000", 2024-04-20T01:48:00.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2140000000000001e-05 2024-04-20T01:48:00.661 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.661 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.661 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.662 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 58/30 ECSubWrite(tid=53, reqid=client.4643.0:490, at_version=58'42, trim_to=0'0, roll_forward_to=55'37))", 2024-04-20T01:48:00.662 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:59.770576+0000", 2024-04-20T01:48:00.662 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.811033063, 2024-04-20T01:48:00.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082383630000000003, 2024-04-20T01:48:00.662 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:00.662 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:00.662 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:00.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:00.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.770576+0000", 2024-04-20T01:48:00.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:00.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.770576+0000", 2024-04-20T01:48:00.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:00.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:00.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.770575+0000", 2024-04-20T01:48:00.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:00.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:00.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.770595+0000", 2024-04-20T01:48:00.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9700000000000001e-05 2024-04-20T01:48:00.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:00.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.770607+0000", 2024-04-20T01:48:00.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1907e-05 2024-04-20T01:48:00.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:00.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.770615+0000", 2024-04-20T01:48:00.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3650000000000004e-06 2024-04-20T01:48:00.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:00.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.770804+0000", 2024-04-20T01:48:00.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000189094 2024-04-20T01:48:00.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:00.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.770815+0000", 2024-04-20T01:48:00.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0351e-05 2024-04-20T01:48:00.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:00.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.778797+0000", 2024-04-20T01:48:00.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0079824960000000004 2024-04-20T01:48:00.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:00.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:00.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:00.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.778814+0000", 2024-04-20T01:48:00.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7172000000000001e-05 2024-04-20T01:48:00.666 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.666 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.666 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.666 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:00.667 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:00.667 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:00.668 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-20T01:48:00.669 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:00.683+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:00.723 INFO:tasks.rados.rados.0.smithi165.stdout:201: done (4 left) 2024-04-20T01:48:00.723 INFO:tasks.rados.rados.0.smithi165.stdout:202: done (3 left) 2024-04-20T01:48:00.724 INFO:tasks.rados.rados.0.smithi165.stdout:203: done (2 left) 2024-04-20T01:48:00.724 INFO:tasks.rados.rados.0.smithi165.stdout:205: done (1 left) 2024-04-20T01:48:00.724 INFO:tasks.rados.rados.0.smithi165.stdout:206: done (0 left) 2024-04-20T01:48:00.724 INFO:tasks.rados.rados.0.smithi165.stdout:207: read oid 5 snap 10 2024-04-20T01:48:00.724 INFO:tasks.rados.rados.0.smithi165.stdout:207: expect (ObjNum 4 snap 0 seq_num 4) 2024-04-20T01:48:00.724 INFO:tasks.rados.rados.0.smithi165.stdout:208: snap_remove snap 12 2024-04-20T01:48:00.768 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:00.782+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:00.838 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:00.838 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:00.838 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-04-20T01:48:00.838 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:00.848 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-20T01:48:00.869 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:00.883+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:00.970 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:00.984+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:01.018 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:01.018 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:01.019 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:48:01.019 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:48:01.019 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:01.029 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-20T01:48:01.071 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:01.086+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:01.159 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:01.159 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:48:01.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:48:01.159 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:48:01.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.160 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:36 3.es0 3:79bd6b86:::smithi16579500-12:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:01.160 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962173+0000", 2024-04-20T01:48:01.160 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 26.210525105999999, 2024-04-20T01:48:01.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090032729999999991, 2024-04-20T01:48:01.160 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.160 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:01.160 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:01.160 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:01.160 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:01.160 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 36 2024-04-20T01:48:01.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.160 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962173+0000", 2024-04-20T01:48:01.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962173+0000", 2024-04-20T01:48:01.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962173+0000", 2024-04-20T01:48:01.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:01.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962175+0000", 2024-04-20T01:48:01.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9269999999999999e-06 2024-04-20T01:48:01.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962179+0000", 2024-04-20T01:48:01.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0300000000000004e-06 2024-04-20T01:48:01.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962183+0000", 2024-04-20T01:48:01.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7340000000000002e-06 2024-04-20T01:48:01.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.964291+0000", 2024-04-20T01:48:01.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021081390000000002 2024-04-20T01:48:01.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:01.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.964348+0000", 2024-04-20T01:48:01.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7218e-05 2024-04-20T01:48:01.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.969516+0000", 2024-04-20T01:48:01.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0051684540000000003 2024-04-20T01:48:01.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:01.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.969602+0000", 2024-04-20T01:48:01.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5294e-05 2024-04-20T01:48:01.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.971176+0000", 2024-04-20T01:48:01.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001574719 2024-04-20T01:48:01.165 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.165 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.165 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.165 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:103 3.es0 3:78f9000f:::smithi16579500-35:head [append 0~573440,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:01.165 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962194+0000", 2024-04-20T01:48:01.165 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 26.210504645, 2024-04-20T01:48:01.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0087917459999999996, 2024-04-20T01:48:01.165 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.165 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:01.166 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:01.166 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:01.166 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:01.166 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 103 2024-04-20T01:48:01.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.166 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962194+0000", 2024-04-20T01:48:01.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962194+0000", 2024-04-20T01:48:01.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962193+0000", 2024-04-20T01:48:01.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:01.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962426+0000", 2024-04-20T01:48:01.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023287700000000001 2024-04-20T01:48:01.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962434+0000", 2024-04-20T01:48:01.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4590000000000001e-06 2024-04-20T01:48:01.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962438+0000", 2024-04-20T01:48:01.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0929999999999996e-06 2024-04-20T01:48:01.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.964426+0000", 2024-04-20T01:48:01.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019882620000000002 2024-04-20T01:48:01.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:01.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.964549+0000", 2024-04-20T01:48:01.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012331399999999999 2024-04-20T01:48:01.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.965303+0000", 2024-04-20T01:48:01.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00075378299999999995 2024-04-20T01:48:01.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.969193+0000", 2024-04-20T01:48:01.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0038896400000000002 2024-04-20T01:48:01.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:01.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970919+0000", 2024-04-20T01:48:01.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001726521 2024-04-20T01:48:01.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970985+0000", 2024-04-20T01:48:01.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6062000000000005e-05 2024-04-20T01:48:01.170 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.170 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.171 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.171 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:104 3.es0 3:78f9000f:::smithi16579500-35:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:01.171 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:48:01.171 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 26.210255799999999, 2024-04-20T01:48:01.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012120811, 2024-04-20T01:48:01.171 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.171 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:01.171 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:01.171 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:01.171 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:01.171 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 104 2024-04-20T01:48:01.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.171 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:48:01.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:48:01.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:48:01.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:01.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962445+0000", 2024-04-20T01:48:01.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3210000000000001e-06 2024-04-20T01:48:01.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962451+0000", 2024-04-20T01:48:01.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.31e-06 2024-04-20T01:48:01.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962457+0000", 2024-04-20T01:48:01.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4819999999999999e-06 2024-04-20T01:48:01.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966434+0000", 2024-04-20T01:48:01.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039772239999999997 2024-04-20T01:48:01.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:01.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966501+0000", 2024-04-20T01:48:01.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6947999999999997e-05 2024-04-20T01:48:01.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966766+0000", 2024-04-20T01:48:01.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00026468999999999999 2024-04-20T01:48:01.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970614+0000", 2024-04-20T01:48:01.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0038481330000000001 2024-04-20T01:48:01.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:01.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974486+0000", 2024-04-20T01:48:01.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003871799 2024-04-20T01:48:01.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974563+0000", 2024-04-20T01:48:01.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7416999999999994e-05 2024-04-20T01:48:01.176 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.176 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.176 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.176 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:105 3.es0 3:78f9000f:::smithi16579500-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:01.177 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962461+0000", 2024-04-20T01:48:01.177 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 26.210237589999998, 2024-04-20T01:48:01.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014134643000000001, 2024-04-20T01:48:01.177 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.177 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:01.177 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:01.177 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:01.177 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:01.177 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 105 2024-04-20T01:48:01.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.177 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962461+0000", 2024-04-20T01:48:01.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962461+0000", 2024-04-20T01:48:01.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962460+0000", 2024-04-20T01:48:01.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:01.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962462+0000", 2024-04-20T01:48:01.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.897e-06 2024-04-20T01:48:01.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962466+0000", 2024-04-20T01:48:01.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4520000000000002e-06 2024-04-20T01:48:01.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962468+0000", 2024-04-20T01:48:01.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6570000000000001e-06 2024-04-20T01:48:01.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966956+0000", 2024-04-20T01:48:01.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044879730000000001 2024-04-20T01:48:01.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:01.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.967029+0000", 2024-04-20T01:48:01.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2657999999999995e-05 2024-04-20T01:48:01.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974602+0000", 2024-04-20T01:48:01.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075729370000000001 2024-04-20T01:48:01.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:01.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974650+0000", 2024-04-20T01:48:01.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8000000000000001e-05 2024-04-20T01:48:01.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976595+0000", 2024-04-20T01:48:01.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019453789999999999 2024-04-20T01:48:01.182 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.182 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.182 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.182 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:109 3.es0 3:7f925225:::smithi16579500-37:head [append 0~606208,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:01.182 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:48:01.182 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 26.210227158999999, 2024-04-20T01:48:01.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012453573000000001, 2024-04-20T01:48:01.182 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.182 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:01.182 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:01.183 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:01.183 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:01.183 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 109 2024-04-20T01:48:01.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.183 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:48:01.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:48:01.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:48:01.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:01.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962732+0000", 2024-04-20T01:48:01.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00026073200000000001 2024-04-20T01:48:01.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962742+0000", 2024-04-20T01:48:01.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0735e-05 2024-04-20T01:48:01.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962749+0000", 2024-04-20T01:48:01.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.365e-06 2024-04-20T01:48:01.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.967197+0000", 2024-04-20T01:48:01.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044487119999999996 2024-04-20T01:48:01.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:01.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.967324+0000", 2024-04-20T01:48:01.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126575 2024-04-20T01:48:01.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968044+0000", 2024-04-20T01:48:01.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00072015399999999998 2024-04-20T01:48:01.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970795+0000", 2024-04-20T01:48:01.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027507650000000001 2024-04-20T01:48:01.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:01.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974884+0000", 2024-04-20T01:48:01.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040886469999999999 2024-04-20T01:48:01.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974925+0000", 2024-04-20T01:48:01.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1094000000000003e-05 2024-04-20T01:48:01.187 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.187 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.187 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.187 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:110 3.es0 3:7f925225:::smithi16579500-37:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:01.188 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962755+0000", 2024-04-20T01:48:01.188 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 26.209943289000002, 2024-04-20T01:48:01.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012356723999999999, 2024-04-20T01:48:01.188 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.188 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:01.188 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:01.188 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:01.188 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:01.188 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 110 2024-04-20T01:48:01.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.188 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962755+0000", 2024-04-20T01:48:01.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962755+0000", 2024-04-20T01:48:01.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962754+0000", 2024-04-20T01:48:01.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:01.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962758+0000", 2024-04-20T01:48:01.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8750000000000002e-06 2024-04-20T01:48:01.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.190 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962765+0000", 2024-04-20T01:48:01.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2370000000000003e-06 2024-04-20T01:48:01.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.190 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962770+0000", 2024-04-20T01:48:01.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9030000000000003e-06 2024-04-20T01:48:01.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.191 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.191 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968614+0000", 2024-04-20T01:48:01.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0058437619999999997 2024-04-20T01:48:01.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.191 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:01.191 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968676+0000", 2024-04-20T01:48:01.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1592000000000003e-05 2024-04-20T01:48:01.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.191 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.191 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968935+0000", 2024-04-20T01:48:01.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025953599999999999 2024-04-20T01:48:01.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970828+0000", 2024-04-20T01:48:01.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0018928510000000001 2024-04-20T01:48:01.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:01.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975036+0000", 2024-04-20T01:48:01.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042079680000000003 2024-04-20T01:48:01.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975112+0000", 2024-04-20T01:48:01.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5558999999999999e-05 2024-04-20T01:48:01.193 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.193 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.193 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.193 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:111 3.es0 3:7f925225:::smithi16579500-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:01.193 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962775+0000", 2024-04-20T01:48:01.193 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 26.209923534000001, 2024-04-20T01:48:01.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014337756, 2024-04-20T01:48:01.193 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.193 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:01.193 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:01.193 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:01.194 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:01.194 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 111 2024-04-20T01:48:01.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.194 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962775+0000", 2024-04-20T01:48:01.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962775+0000", 2024-04-20T01:48:01.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962774+0000", 2024-04-20T01:48:01.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:01.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962777+0000", 2024-04-20T01:48:01.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7839999999999999e-06 2024-04-20T01:48:01.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962784+0000", 2024-04-20T01:48:01.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9809999999999997e-06 2024-04-20T01:48:01.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962790+0000", 2024-04-20T01:48:01.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5509999999999999e-06 2024-04-20T01:48:01.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970564+0000", 2024-04-20T01:48:01.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077740680000000003 2024-04-20T01:48:01.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:01.197 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970609+0000", 2024-04-20T01:48:01.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5059999999999999e-05 2024-04-20T01:48:01.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.197 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.197 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975165+0000", 2024-04-20T01:48:01.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0045564999999999998 2024-04-20T01:48:01.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.197 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:01.197 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975263+0000", 2024-04-20T01:48:01.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7689000000000006e-05 2024-04-20T01:48:01.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.198 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977112+0000", 2024-04-20T01:48:01.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001849552 2024-04-20T01:48:01.198 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.198 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.198 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.198 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:142 3.es0 3:797661d0:::smithi16579500-48:head [append 0~425984,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:01.198 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962794+0000", 2024-04-20T01:48:01.198 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 26.209904333000001, 2024-04-20T01:48:01.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013969364, 2024-04-20T01:48:01.198 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.199 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:01.199 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:01.199 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:01.199 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:01.199 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 142 2024-04-20T01:48:01.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.199 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.199 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.199 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962794+0000", 2024-04-20T01:48:01.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.199 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962794+0000", 2024-04-20T01:48:01.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962793+0000", 2024-04-20T01:48:01.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:01.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963014+0000", 2024-04-20T01:48:01.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000221015 2024-04-20T01:48:01.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963025+0000", 2024-04-20T01:48:01.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0739e-05 2024-04-20T01:48:01.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963032+0000", 2024-04-20T01:48:01.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7580000000000003e-06 2024-04-20T01:48:01.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.971338+0000", 2024-04-20T01:48:01.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008305659 2024-04-20T01:48:01.202 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.202 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.202 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:01.202 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.971477+0000", 2024-04-20T01:48:01.202 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013920100000000001 2024-04-20T01:48:01.202 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.202 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.202 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.202 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.972169+0000", 2024-04-20T01:48:01.202 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00069257800000000001 2024-04-20T01:48:01.202 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.202 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.202 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.203 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974577+0000", 2024-04-20T01:48:01.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024072820000000002 2024-04-20T01:48:01.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.203 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:01.203 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976694+0000", 2024-04-20T01:48:01.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021172629999999999 2024-04-20T01:48:01.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.203 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.203 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976763+0000", 2024-04-20T01:48:01.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9259999999999998e-05 2024-04-20T01:48:01.203 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.203 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.204 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.204 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:143 3.es0 3:797661d0:::smithi16579500-48:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:01.204 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.963039+0000", 2024-04-20T01:48:01.204 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 26.209659578, 2024-04-20T01:48:01.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013923471999999999, 2024-04-20T01:48:01.204 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.204 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:01.204 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:01.204 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:01.204 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:01.204 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 143 2024-04-20T01:48:01.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.205 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963039+0000", 2024-04-20T01:48:01.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963039+0000", 2024-04-20T01:48:01.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963038+0000", 2024-04-20T01:48:01.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:01.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963210+0000", 2024-04-20T01:48:01.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017155800000000001 2024-04-20T01:48:01.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963217+0000", 2024-04-20T01:48:01.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7309999999999993e-06 2024-04-20T01:48:01.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.207 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.207 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963230+0000", 2024-04-20T01:48:01.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.295e-05 2024-04-20T01:48:01.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.207 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.207 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.972654+0000", 2024-04-20T01:48:01.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094237369999999997 2024-04-20T01:48:01.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.207 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:01.207 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.972720+0000", 2024-04-20T01:48:01.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5958999999999996e-05 2024-04-20T01:48:01.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.973225+0000", 2024-04-20T01:48:01.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00050500399999999998 2024-04-20T01:48:01.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974578+0000", 2024-04-20T01:48:01.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001353295 2024-04-20T01:48:01.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:01.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976919+0000", 2024-04-20T01:48:01.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023405840000000002 2024-04-20T01:48:01.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976962+0000", 2024-04-20T01:48:01.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3226000000000003e-05 2024-04-20T01:48:01.209 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.209 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.209 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.209 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:144 3.es0 3:797661d0:::smithi16579500-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:01.209 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:48:01.210 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 26.209460944, 2024-04-20T01:48:01.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014456992, 2024-04-20T01:48:01.210 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.210 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:01.210 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:01.210 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:01.210 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:01.210 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 144 2024-04-20T01:48:01.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.210 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.210 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.210 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:48:01.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.211 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:48:01.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.211 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:48:01.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:01.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963240+0000", 2024-04-20T01:48:01.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6459999999999999e-06 2024-04-20T01:48:01.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963247+0000", 2024-04-20T01:48:01.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9650000000000002e-06 2024-04-20T01:48:01.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963253+0000", 2024-04-20T01:48:01.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1900000000000003e-06 2024-04-20T01:48:01.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.973649+0000", 2024-04-20T01:48:01.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010396865999999999 2024-04-20T01:48:01.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:01.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.973745+0000", 2024-04-20T01:48:01.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5345000000000003e-05 2024-04-20T01:48:01.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974151+0000", 2024-04-20T01:48:01.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000405893 2024-04-20T01:48:01.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.214 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976606+0000", 2024-04-20T01:48:01.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024551730000000002 2024-04-20T01:48:01.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:01.214 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977622+0000", 2024-04-20T01:48:01.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0010159570000000001 2024-04-20T01:48:01.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.214 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977694+0000", 2024-04-20T01:48:01.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2469e-05 2024-04-20T01:48:01.215 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.215 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.215 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.215 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:145 3.es0 3:797661d0:::smithi16579500-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:01.215 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:48:01.215 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 26.209439122999999, 2024-04-20T01:48:01.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016339408999999999, 2024-04-20T01:48:01.215 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.215 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:01.215 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:01.215 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:01.215 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:01.216 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 145 2024-04-20T01:48:01.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.216 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:48:01.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:48:01.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:48:01.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:01.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963267+0000", 2024-04-20T01:48:01.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2490000000000001e-06 2024-04-20T01:48:01.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963274+0000", 2024-04-20T01:48:01.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7599999999999997e-06 2024-04-20T01:48:01.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963279+0000", 2024-04-20T01:48:01.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5400000000000003e-06 2024-04-20T01:48:01.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977300+0000", 2024-04-20T01:48:01.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014020608 2024-04-20T01:48:01.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:01.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977365+0000", 2024-04-20T01:48:01.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5617999999999994e-05 2024-04-20T01:48:01.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.978057+0000", 2024-04-20T01:48:01.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00069179099999999998 2024-04-20T01:48:01.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:01.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.978136+0000", 2024-04-20T01:48:01.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9132000000000001e-05 2024-04-20T01:48:01.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.979598+0000", 2024-04-20T01:48:01.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0014621599999999999 2024-04-20T01:48:01.220 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.220 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.220 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.220 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.bs4 37/30 ECSubWrite(tid=65, reqid=client.4643.0:226, at_version=37'17, trim_to=0'0, roll_forward_to=36'14))", 2024-04-20T01:48:01.220 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.790236+0000", 2024-04-20T01:48:01.220 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 22.382462519000001, 2024-04-20T01:48:01.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091261220000000004, 2024-04-20T01:48:01.220 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.220 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.220 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.220 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.220 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.790236+0000", 2024-04-20T01:48:01.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.790236+0000", 2024-04-20T01:48:01.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.790235+0000", 2024-04-20T01:48:01.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:01.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.790520+0000", 2024-04-20T01:48:01.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028522399999999999 2024-04-20T01:48:01.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.790531+0000", 2024-04-20T01:48:01.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0871e-05 2024-04-20T01:48:01.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.790540+0000", 2024-04-20T01:48:01.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1570000000000006e-06 2024-04-20T01:48:01.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.790875+0000", 2024-04-20T01:48:01.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00033489400000000002 2024-04-20T01:48:01.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.790880+0000", 2024-04-20T01:48:01.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6629999999999998e-06 2024-04-20T01:48:01.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799335+0000", 2024-04-20T01:48:01.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0084541489999999993 2024-04-20T01:48:01.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799362+0000", 2024-04-20T01:48:01.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7161e-05 2024-04-20T01:48:01.224 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.224 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.224 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.224 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:245 3.es0 3:78f9000f:::smithi16579500-35:head [copy-from ver 10] snapc 5=[2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:01.224 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785668+0000", 2024-04-20T01:48:01.224 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 21.387029937000001, 2024-04-20T01:48:01.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025155522, 2024-04-20T01:48:01.224 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.224 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:01.224 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:01.225 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:01.225 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:01.225 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 245 2024-04-20T01:48:01.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.225 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785668+0000", 2024-04-20T01:48:01.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785668+0000", 2024-04-20T01:48:01.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785666+0000", 2024-04-20T01:48:01.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:01.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785676+0000", 2024-04-20T01:48:01.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0139999999999999e-05 2024-04-20T01:48:01.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785692+0000", 2024-04-20T01:48:01.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5894999999999999e-05 2024-04-20T01:48:01.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785715+0000", 2024-04-20T01:48:01.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2654999999999999e-05 2024-04-20T01:48:01.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785771+0000", 2024-04-20T01:48:01.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6331e-05 2024-04-20T01:48:01.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:01.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785779+0000", 2024-04-20T01:48:01.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4789999999999999e-06 2024-04-20T01:48:01.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791776+0000", 2024-04-20T01:48:01.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059978840000000002 2024-04-20T01:48:01.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:01.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791905+0000", 2024-04-20T01:48:01.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000128147 2024-04-20T01:48:01.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.805603+0000", 2024-04-20T01:48:01.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013698824 2024-04-20T01:48:01.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.808612+0000", 2024-04-20T01:48:01.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003008541 2024-04-20T01:48:01.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:01.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.810754+0000", 2024-04-20T01:48:01.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021421840000000001 2024-04-20T01:48:01.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.810824+0000", 2024-04-20T01:48:01.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9627000000000003e-05 2024-04-20T01:48:01.230 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.230 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.230 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.230 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:246 3.es0 3:78f9000f:::smithi16579500-35:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:01.230 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:48:01.230 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 21.386969757999999, 2024-04-20T01:48:01.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020980167000000001, 2024-04-20T01:48:01.230 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.230 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:01.230 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:01.230 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:01.231 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:01.231 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 246 2024-04-20T01:48:01.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.231 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:48:01.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:48:01.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:48:01.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:01.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785731+0000", 2024-04-20T01:48:01.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3550000000000001e-06 2024-04-20T01:48:01.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785738+0000", 2024-04-20T01:48:01.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2080000000000003e-06 2024-04-20T01:48:01.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785747+0000", 2024-04-20T01:48:01.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.388e-06 2024-04-20T01:48:01.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785859+0000", 2024-04-20T01:48:01.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112432 2024-04-20T01:48:01.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:01.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785866+0000", 2024-04-20T01:48:01.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0890000000000002e-06 2024-04-20T01:48:01.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792104+0000", 2024-04-20T01:48:01.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062376970000000004 2024-04-20T01:48:01.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:01.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792157+0000", 2024-04-20T01:48:01.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3269000000000001e-05 2024-04-20T01:48:01.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.806521+0000", 2024-04-20T01:48:01.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014363651999999999 2024-04-20T01:48:01.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:01.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.806639+0000", 2024-04-20T01:48:01.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011755400000000001 2024-04-20T01:48:01.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.806709+0000", 2024-04-20T01:48:01.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0037999999999996e-05 2024-04-20T01:48:01.235 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.235 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.235 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.236 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:247 3.es0 3:79bd6b86:::smithi16579500-12:head [read 0~196608,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:01.236 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785760+0000", 2024-04-20T01:48:01.236 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 21.386938509, 2024-04-20T01:48:01.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011359487999999999, 2024-04-20T01:48:01.236 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.236 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:01.236 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:01.236 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:01.236 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:01.236 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 247 2024-04-20T01:48:01.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.236 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785760+0000", 2024-04-20T01:48:01.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785760+0000", 2024-04-20T01:48:01.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785759+0000", 2024-04-20T01:48:01.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:01.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785764+0000", 2024-04-20T01:48:01.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.194e-06 2024-04-20T01:48:01.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785772+0000", 2024-04-20T01:48:01.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9150000000000002e-06 2024-04-20T01:48:01.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785780+0000", 2024-04-20T01:48:01.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5639999999999999e-06 2024-04-20T01:48:01.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785905+0000", 2024-04-20T01:48:01.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012477000000000001 2024-04-20T01:48:01.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:01.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785909+0000", 2024-04-20T01:48:01.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1099999999999996e-06 2024-04-20T01:48:01.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792209+0000", 2024-04-20T01:48:01.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062998009999999998 2024-04-20T01:48:01.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.240 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:01.240 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792316+0000", 2024-04-20T01:48:01.240 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107161 2024-04-20T01:48:01.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.240 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.240 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.240 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.797119+0000", 2024-04-20T01:48:01.240 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0048032860000000004 2024-04-20T01:48:01.240 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.240 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.240 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.240 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.240 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:248 3.es0 3:79bd6b86:::smithi16579500-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:01.240 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:48:01.241 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 21.386913998000001, 2024-04-20T01:48:01.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011602988, 2024-04-20T01:48:01.241 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.241 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:01.241 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:01.241 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:01.241 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:01.241 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 248 2024-04-20T01:48:01.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.241 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:48:01.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:48:01.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:48:01.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:01.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785787+0000", 2024-04-20T01:48:01.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4079999999999998e-06 2024-04-20T01:48:01.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785791+0000", 2024-04-20T01:48:01.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9929999999999997e-06 2024-04-20T01:48:01.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785795+0000", 2024-04-20T01:48:01.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1559999999999997e-06 2024-04-20T01:48:01.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785936+0000", 2024-04-20T01:48:01.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014018 2024-04-20T01:48:01.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:01.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785939+0000", 2024-04-20T01:48:01.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-04-20T01:48:01.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794183+0000", 2024-04-20T01:48:01.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082434339999999991 2024-04-20T01:48:01.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:01.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794272+0000", 2024-04-20T01:48:01.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9246000000000006e-05 2024-04-20T01:48:01.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.797387+0000", 2024-04-20T01:48:01.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0031150269999999998 2024-04-20T01:48:01.245 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.245 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.245 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.246 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:249 3.es0 3:79bd6b86:::smithi16579500-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:01.246 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:48:01.246 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 21.386899701000001, 2024-04-20T01:48:01.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011999312, 2024-04-20T01:48:01.246 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.246 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:01.246 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:01.246 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:01.246 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:01.246 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 249 2024-04-20T01:48:01.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.246 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:48:01.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:48:01.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:48:01.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:01.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785800+0000", 2024-04-20T01:48:01.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.995e-06 2024-04-20T01:48:01.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785804+0000", 2024-04-20T01:48:01.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2990000000000001e-06 2024-04-20T01:48:01.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785807+0000", 2024-04-20T01:48:01.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.715e-06 2024-04-20T01:48:01.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785955+0000", 2024-04-20T01:48:01.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000147923 2024-04-20T01:48:01.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:01.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785958+0000", 2024-04-20T01:48:01.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8940000000000002e-06 2024-04-20T01:48:01.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794609+0000", 2024-04-20T01:48:01.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086509289999999999 2024-04-20T01:48:01.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.250 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:01.250 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.795125+0000", 2024-04-20T01:48:01.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00051564999999999998 2024-04-20T01:48:01.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.250 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.250 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.797798+0000", 2024-04-20T01:48:01.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0026731250000000002 2024-04-20T01:48:01.250 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.250 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.250 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.250 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.bs4 38/30 ECSubWrite(tid=72, reqid=client.4643.0:250, at_version=38'20, trim_to=0'0, roll_forward_to=37'18))", 2024-04-20T01:48:01.251 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.787459+0000", 2024-04-20T01:48:01.251 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 21.385239172999999, 2024-04-20T01:48:01.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080595449999999996, 2024-04-20T01:48:01.251 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.251 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.251 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.787459+0000", 2024-04-20T01:48:01.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.787459+0000", 2024-04-20T01:48:01.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.787459+0000", 2024-04-20T01:48:01.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:01.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.787477+0000", 2024-04-20T01:48:01.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8553999999999999e-05 2024-04-20T01:48:01.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.787498+0000", 2024-04-20T01:48:01.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0673999999999999e-05 2024-04-20T01:48:01.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.787508+0000", 2024-04-20T01:48:01.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0319999999999999e-05 2024-04-20T01:48:01.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.787563+0000", 2024-04-20T01:48:01.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4694999999999997e-05 2024-04-20T01:48:01.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.787570+0000", 2024-04-20T01:48:01.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2439999999999996e-06 2024-04-20T01:48:01.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.795481+0000", 2024-04-20T01:48:01.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0079110650000000001 2024-04-20T01:48:01.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.795519+0000", 2024-04-20T01:48:01.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7141000000000002e-05 2024-04-20T01:48:01.254 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.254 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.255 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.255 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.bs4 48/30 ECSubWrite(tid=120, reqid=osd.4.0:120, at_version=48'51, trim_to=0'0, roll_forward_to=47'49))", 2024-04-20T01:48:01.255 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.717256+0000", 2024-04-20T01:48:01.255 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.455442114, 2024-04-20T01:48:01.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0081088800000000006, 2024-04-20T01:48:01.255 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.255 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.255 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.255 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.717256+0000", 2024-04-20T01:48:01.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.717256+0000", 2024-04-20T01:48:01.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.717256+0000", 2024-04-20T01:48:01.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:01.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.717283+0000", 2024-04-20T01:48:01.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7423000000000002e-05 2024-04-20T01:48:01.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.717297+0000", 2024-04-20T01:48:01.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3678e-05 2024-04-20T01:48:01.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.717315+0000", 2024-04-20T01:48:01.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8649999999999999e-05 2024-04-20T01:48:01.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.717389+0000", 2024-04-20T01:48:01.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3888999999999997e-05 2024-04-20T01:48:01.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.717397+0000", 2024-04-20T01:48:01.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4959999999999999e-06 2024-04-20T01:48:01.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.725330+0000", 2024-04-20T01:48:01.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.007933084 2024-04-20T01:48:01.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.725365+0000", 2024-04-20T01:48:01.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5024999999999998e-05 2024-04-20T01:48:01.259 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.259 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.259 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.259 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.bs4 48/30 ECSubWrite(tid=121, reqid=osd.4.0:121, at_version=48'53, trim_to=0'0, roll_forward_to=47'49))", 2024-04-20T01:48:01.259 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.718211+0000", 2024-04-20T01:48:01.259 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.454487436999999, 2024-04-20T01:48:01.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010033991000000001, 2024-04-20T01:48:01.260 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.260 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.260 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.718211+0000", 2024-04-20T01:48:01.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.718211+0000", 2024-04-20T01:48:01.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.718210+0000", 2024-04-20T01:48:01.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:01.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.261 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.718234+0000", 2024-04-20T01:48:01.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3966999999999999e-05 2024-04-20T01:48:01.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.261 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.718250+0000", 2024-04-20T01:48:01.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6620999999999999e-05 2024-04-20T01:48:01.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.718261+0000", 2024-04-20T01:48:01.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0348e-05 2024-04-20T01:48:01.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.718345+0000", 2024-04-20T01:48:01.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4079999999999995e-05 2024-04-20T01:48:01.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.718355+0000", 2024-04-20T01:48:01.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0427e-05 2024-04-20T01:48:01.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.728196+0000", 2024-04-20T01:48:01.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0098408179999999994 2024-04-20T01:48:01.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.728245+0000", 2024-04-20T01:48:01.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.867e-05 2024-04-20T01:48:01.263 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.263 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.263 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.263 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.263 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.264 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:01.265 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-20T01:48:01.266 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:01.280+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:01.366 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:01.380+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:01.394 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:01.394 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:01.394 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-04-20T01:48:01.394 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:01.404 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-20T01:48:01.467 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:01.481+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:01.525 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-20T01:48:01.531 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:01.531 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:01.531 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:48:01.531 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:48:01.531 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:01.542 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-20T01:48:01.570 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:01.584+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:01.668 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:01.683+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:01.671 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:01.671 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:48:01.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:48:01.671 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:48:01.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.671 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs2 33/30 ECSubWrite(tid=4, reqid=client.4643.0:94, at_version=33'3, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:48:01.671 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.810869+0000", 2024-04-20T01:48:01.672 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 26.873835513, 2024-04-20T01:48:01.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040776120000000004, 2024-04-20T01:48:01.672 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.672 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.672 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.810869+0000", 2024-04-20T01:48:01.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.810869+0000", 2024-04-20T01:48:01.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.810869+0000", 2024-04-20T01:48:01.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:01.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.811245+0000", 2024-04-20T01:48:01.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00037595499999999997 2024-04-20T01:48:01.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.811257+0000", 2024-04-20T01:48:01.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2099000000000001e-05 2024-04-20T01:48:01.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.811266+0000", 2024-04-20T01:48:01.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8629999999999997e-06 2024-04-20T01:48:01.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.811319+0000", 2024-04-20T01:48:01.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3338000000000003e-05 2024-04-20T01:48:01.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.811331+0000", 2024-04-20T01:48:01.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1457999999999999e-05 2024-04-20T01:48:01.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.814920+0000", 2024-04-20T01:48:01.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035894189999999999 2024-04-20T01:48:01.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.814947+0000", 2024-04-20T01:48:01.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7129999999999999e-05 2024-04-20T01:48:01.675 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.675 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.676 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.676 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs2 33/30 ECSubWrite(tid=10, reqid=client.4643.0:125, at_version=0'0, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:48:01.676 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.854400+0000", 2024-04-20T01:48:01.676 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 26.830304730000002, 2024-04-20T01:48:01.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046267000000000001, 2024-04-20T01:48:01.676 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.676 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.676 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.854400+0000", 2024-04-20T01:48:01.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.854400+0000", 2024-04-20T01:48:01.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.854399+0000", 2024-04-20T01:48:01.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:01.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.854410+0000", 2024-04-20T01:48:01.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0743e-05 2024-04-20T01:48:01.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.854424+0000", 2024-04-20T01:48:01.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.399e-05 2024-04-20T01:48:01.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.854430+0000", 2024-04-20T01:48:01.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2689999999999996e-06 2024-04-20T01:48:01.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.854469+0000", 2024-04-20T01:48:01.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8732000000000001e-05 2024-04-20T01:48:01.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.854478+0000", 2024-04-20T01:48:01.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5159999999999994e-06 2024-04-20T01:48:01.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.859010+0000", 2024-04-20T01:48:01.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0045325740000000002 2024-04-20T01:48:01.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.859027+0000", 2024-04-20T01:48:01.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6762999999999999e-05 2024-04-20T01:48:01.680 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.680 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.680 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.680 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.680 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 37/30 ECSubWrite(tid=43, reqid=client.4643.0:241, at_version=37'31, trim_to=0'0, roll_forward_to=36'28))", 2024-04-20T01:48:01.680 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.793501+0000", 2024-04-20T01:48:01.680 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 22.891203966999999, 2024-04-20T01:48:01.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005847517, 2024-04-20T01:48:01.680 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.680 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.680 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.680 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.680 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793501+0000", 2024-04-20T01:48:01.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793501+0000", 2024-04-20T01:48:01.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793500+0000", 2024-04-20T01:48:01.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:01.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793525+0000", 2024-04-20T01:48:01.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4641999999999999e-05 2024-04-20T01:48:01.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.682 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.682 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793537+0000", 2024-04-20T01:48:01.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2626000000000001e-05 2024-04-20T01:48:01.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.682 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.682 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793547+0000", 2024-04-20T01:48:01.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.842e-06 2024-04-20T01:48:01.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.682 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793599+0000", 2024-04-20T01:48:01.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2301e-05 2024-04-20T01:48:01.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793607+0000", 2024-04-20T01:48:01.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9070000000000007e-06 2024-04-20T01:48:01.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799321+0000", 2024-04-20T01:48:01.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005713358 2024-04-20T01:48:01.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799349+0000", 2024-04-20T01:48:01.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7832999999999999e-05 2024-04-20T01:48:01.684 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.684 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.684 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.684 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 37/30 ECSubWrite(tid=19, reqid=client.4643.0:235, at_version=37'9, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:48:01.684 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.793816+0000", 2024-04-20T01:48:01.684 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 22.890888491999998, 2024-04-20T01:48:01.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0055550790000000001, 2024-04-20T01:48:01.684 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.684 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.684 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793816+0000", 2024-04-20T01:48:01.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793816+0000", 2024-04-20T01:48:01.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793816+0000", 2024-04-20T01:48:01.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:01.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793838+0000", 2024-04-20T01:48:01.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2075000000000001e-05 2024-04-20T01:48:01.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793849+0000", 2024-04-20T01:48:01.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1433999999999999e-05 2024-04-20T01:48:01.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793859+0000", 2024-04-20T01:48:01.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4849999999999998e-06 2024-04-20T01:48:01.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.687 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.687 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793956+0000", 2024-04-20T01:48:01.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7114000000000006e-05 2024-04-20T01:48:01.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.687 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.687 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793965+0000", 2024-04-20T01:48:01.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0820000000000005e-06 2024-04-20T01:48:01.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.687 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.687 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799344+0000", 2024-04-20T01:48:01.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053794400000000001 2024-04-20T01:48:01.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799372+0000", 2024-04-20T01:48:01.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7231000000000001e-05 2024-04-20T01:48:01.688 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.688 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.688 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.689 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 37/30 ECSubWrite(tid=44, reqid=client.4643.0:242, at_version=37'32, trim_to=0'0, roll_forward_to=36'28))", 2024-04-20T01:48:01.689 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.794496+0000", 2024-04-20T01:48:01.689 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 22.890208600000001, 2024-04-20T01:48:01.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0049252380000000002, 2024-04-20T01:48:01.689 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.689 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.689 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794496+0000", 2024-04-20T01:48:01.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794496+0000", 2024-04-20T01:48:01.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794496+0000", 2024-04-20T01:48:01.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:01.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794516+0000", 2024-04-20T01:48:01.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0475999999999998e-05 2024-04-20T01:48:01.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794530+0000", 2024-04-20T01:48:01.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3565e-05 2024-04-20T01:48:01.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794539+0000", 2024-04-20T01:48:01.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9600000000000006e-06 2024-04-20T01:48:01.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794591+0000", 2024-04-20T01:48:01.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2574999999999997e-05 2024-04-20T01:48:01.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794602+0000", 2024-04-20T01:48:01.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1338999999999999e-05 2024-04-20T01:48:01.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799353+0000", 2024-04-20T01:48:01.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047508999999999997 2024-04-20T01:48:01.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799422+0000", 2024-04-20T01:48:01.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8281999999999994e-05 2024-04-20T01:48:01.692 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.693 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.693 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.693 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 38/30 ECSubWrite(tid=23, reqid=osd.3.0:22, at_version=0'0, trim_to=0'0, roll_forward_to=38'12))", 2024-04-20T01:48:01.693 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.788077+0000", 2024-04-20T01:48:01.693 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 21.896628438, 2024-04-20T01:48:01.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057844480000000002, 2024-04-20T01:48:01.693 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.693 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.693 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788077+0000", 2024-04-20T01:48:01.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788077+0000", 2024-04-20T01:48:01.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788076+0000", 2024-04-20T01:48:01.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:01.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788088+0000", 2024-04-20T01:48:01.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2672e-05 2024-04-20T01:48:01.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788101+0000", 2024-04-20T01:48:01.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2468e-05 2024-04-20T01:48:01.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788121+0000", 2024-04-20T01:48:01.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0315000000000001e-05 2024-04-20T01:48:01.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788181+0000", 2024-04-20T01:48:01.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0257999999999998e-05 2024-04-20T01:48:01.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788194+0000", 2024-04-20T01:48:01.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2714000000000001e-05 2024-04-20T01:48:01.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.793811+0000", 2024-04-20T01:48:01.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056169649999999998 2024-04-20T01:48:01.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.793861+0000", 2024-04-20T01:48:01.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9978000000000002e-05 2024-04-20T01:48:01.697 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.697 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.697 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.698 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 38/30 ECSubWrite(tid=46, reqid=osd.8.0:46, at_version=38'34, trim_to=0'0, roll_forward_to=37'32))", 2024-04-20T01:48:01.698 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.791125+0000", 2024-04-20T01:48:01.698 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 21.893579726999999, 2024-04-20T01:48:01.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046488060000000001, 2024-04-20T01:48:01.698 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.698 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.698 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791125+0000", 2024-04-20T01:48:01.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791125+0000", 2024-04-20T01:48:01.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791125+0000", 2024-04-20T01:48:01.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:01.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791147+0000", 2024-04-20T01:48:01.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2269999999999999e-05 2024-04-20T01:48:01.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791160+0000", 2024-04-20T01:48:01.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2492e-05 2024-04-20T01:48:01.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791172+0000", 2024-04-20T01:48:01.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2079000000000001e-05 2024-04-20T01:48:01.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791235+0000", 2024-04-20T01:48:01.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2957e-05 2024-04-20T01:48:01.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791246+0000", 2024-04-20T01:48:01.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1236999999999999e-05 2024-04-20T01:48:01.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.795741+0000", 2024-04-20T01:48:01.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.004495265 2024-04-20T01:48:01.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.795774+0000", 2024-04-20T01:48:01.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2775999999999999e-05 2024-04-20T01:48:01.702 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.702 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.702 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.702 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 42/30 ECSubWrite(tid=64, reqid=client.4643.0:318, at_version=42'51, trim_to=0'0, roll_forward_to=41'48))", 2024-04-20T01:48:01.702 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:43.938535+0000", 2024-04-20T01:48:01.702 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 17.746169618, 2024-04-20T01:48:01.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043950170000000002, 2024-04-20T01:48:01.702 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.702 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.703 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.938535+0000", 2024-04-20T01:48:01.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.938535+0000", 2024-04-20T01:48:01.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.938535+0000", 2024-04-20T01:48:01.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:01.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.938673+0000", 2024-04-20T01:48:01.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013837800000000001 2024-04-20T01:48:01.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.938684+0000", 2024-04-20T01:48:01.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1233000000000001e-05 2024-04-20T01:48:01.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.938693+0000", 2024-04-20T01:48:01.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4470000000000006e-06 2024-04-20T01:48:01.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.939397+0000", 2024-04-20T01:48:01.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00070402599999999996 2024-04-20T01:48:01.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.939404+0000", 2024-04-20T01:48:01.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2309999999999999e-06 2024-04-20T01:48:01.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.942907+0000", 2024-04-20T01:48:01.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003502654 2024-04-20T01:48:01.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.942930+0000", 2024-04-20T01:48:01.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3728000000000001e-05 2024-04-20T01:48:01.706 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.706 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.706 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.706 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 48/30 ECSubWrite(tid=86, reqid=client.4643.0:400, at_version=48'72, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:01.706 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.713186+0000", 2024-04-20T01:48:01.707 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.971518989, 2024-04-20T01:48:01.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0041488510000000003, 2024-04-20T01:48:01.707 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.707 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.707 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.713186+0000", 2024-04-20T01:48:01.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.713186+0000", 2024-04-20T01:48:01.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.713185+0000", 2024-04-20T01:48:01.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:01.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.713205+0000", 2024-04-20T01:48:01.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9454e-05 2024-04-20T01:48:01.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.713216+0000", 2024-04-20T01:48:01.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1311000000000001e-05 2024-04-20T01:48:01.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.713223+0000", 2024-04-20T01:48:01.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4290000000000004e-06 2024-04-20T01:48:01.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.713283+0000", 2024-04-20T01:48:01.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9722000000000002e-05 2024-04-20T01:48:01.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.713295+0000", 2024-04-20T01:48:01.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1616e-05 2024-04-20T01:48:01.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.717303+0000", 2024-04-20T01:48:01.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040082579999999998 2024-04-20T01:48:01.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.717335+0000", 2024-04-20T01:48:01.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1918000000000002e-05 2024-04-20T01:48:01.710 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.711 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.711 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.711 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 53/30 ECSubWrite(tid=32, reqid=client.4643.0:443, at_version=53'21, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:01.711 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.718947+0000", 2024-04-20T01:48:01.711 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.9657574699999998, 2024-04-20T01:48:01.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009785129, 2024-04-20T01:48:01.711 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.711 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.711 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718947+0000", 2024-04-20T01:48:01.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718947+0000", 2024-04-20T01:48:01.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718947+0000", 2024-04-20T01:48:01.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:01.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719111+0000", 2024-04-20T01:48:01.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000164227 2024-04-20T01:48:01.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719125+0000", 2024-04-20T01:48:01.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.454e-05 2024-04-20T01:48:01.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719137+0000", 2024-04-20T01:48:01.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1783999999999999e-05 2024-04-20T01:48:01.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719206+0000", 2024-04-20T01:48:01.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9123999999999998e-05 2024-04-20T01:48:01.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719221+0000", 2024-04-20T01:48:01.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4963999999999999e-05 2024-04-20T01:48:01.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.728704+0000", 2024-04-20T01:48:01.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094822859999999995 2024-04-20T01:48:01.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.728733+0000", 2024-04-20T01:48:01.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8972e-05 2024-04-20T01:48:01.715 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.715 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.715 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.715 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 53/30 ECSubWrite(tid=33, reqid=client.4643.0:444, at_version=53'22, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:01.716 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.721427+0000", 2024-04-20T01:48:01.716 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.9632782200000003, 2024-04-20T01:48:01.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010045943999999999, 2024-04-20T01:48:01.716 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.716 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.716 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721427+0000", 2024-04-20T01:48:01.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721427+0000", 2024-04-20T01:48:01.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721424+0000", 2024-04-20T01:48:01.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999967 2024-04-20T01:48:01.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721541+0000", 2024-04-20T01:48:01.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117309 2024-04-20T01:48:01.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721560+0000", 2024-04-20T01:48:01.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8559999999999998e-05 2024-04-20T01:48:01.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721571+0000", 2024-04-20T01:48:01.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1612e-05 2024-04-20T01:48:01.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721648+0000", 2024-04-20T01:48:01.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6534999999999998e-05 2024-04-20T01:48:01.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721660+0000", 2024-04-20T01:48:01.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2396e-05 2024-04-20T01:48:01.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731447+0000", 2024-04-20T01:48:01.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097865039999999997 2024-04-20T01:48:01.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731473+0000", 2024-04-20T01:48:01.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6129999999999999e-05 2024-04-20T01:48:01.720 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.720 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.720 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.720 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 53/30 ECSubWrite(tid=34, reqid=client.4643.0:445, at_version=53'23, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:01.720 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.722888+0000", 2024-04-20T01:48:01.720 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.9618167399999997, 2024-04-20T01:48:01.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086321880000000007, 2024-04-20T01:48:01.720 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.720 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.720 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722888+0000", 2024-04-20T01:48:01.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722888+0000", 2024-04-20T01:48:01.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722887+0000", 2024-04-20T01:48:01.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:01.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722958+0000", 2024-04-20T01:48:01.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0167999999999996e-05 2024-04-20T01:48:01.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722970+0000", 2024-04-20T01:48:01.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2393e-05 2024-04-20T01:48:01.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722979+0000", 2024-04-20T01:48:01.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1509999999999993e-06 2024-04-20T01:48:01.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723935+0000", 2024-04-20T01:48:01.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00095590900000000001 2024-04-20T01:48:01.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723947+0000", 2024-04-20T01:48:01.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2405999999999999e-05 2024-04-20T01:48:01.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731503+0000", 2024-04-20T01:48:01.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075554990000000002 2024-04-20T01:48:01.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731520+0000", 2024-04-20T01:48:01.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7490999999999999e-05 2024-04-20T01:48:01.724 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.724 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.724 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.724 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 53/30 ECSubWrite(tid=35, reqid=client.4643.0:446, at_version=53'24, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:01.724 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.724462+0000", 2024-04-20T01:48:01.724 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.9602426609999997, 2024-04-20T01:48:01.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0070986110000000003, 2024-04-20T01:48:01.725 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.725 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.725 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724462+0000", 2024-04-20T01:48:01.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724462+0000", 2024-04-20T01:48:01.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724461+0000", 2024-04-20T01:48:01.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:01.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724482+0000", 2024-04-20T01:48:01.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0557999999999999e-05 2024-04-20T01:48:01.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724502+0000", 2024-04-20T01:48:01.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9704999999999999e-05 2024-04-20T01:48:01.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724511+0000", 2024-04-20T01:48:01.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8699999999999998e-06 2024-04-20T01:48:01.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724537+0000", 2024-04-20T01:48:01.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.688e-05 2024-04-20T01:48:01.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724543+0000", 2024-04-20T01:48:01.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9909999999999999e-06 2024-04-20T01:48:01.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731548+0000", 2024-04-20T01:48:01.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0070045990000000002 2024-04-20T01:48:01.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731561+0000", 2024-04-20T01:48:01.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2857000000000001e-05 2024-04-20T01:48:01.729 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.729 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.729 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.729 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs2 55/30 ECSubWrite(tid=40, reqid=client.4643.0:458, at_version=55'22, trim_to=0'0, roll_forward_to=52'21))", 2024-04-20T01:48:01.729 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:56.729618+0000", 2024-04-20T01:48:01.729 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9550867199999997, 2024-04-20T01:48:01.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0048654960000000004, 2024-04-20T01:48:01.729 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.729 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.729 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.729618+0000", 2024-04-20T01:48:01.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.729618+0000", 2024-04-20T01:48:01.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.729618+0000", 2024-04-20T01:48:01.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:01.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.729643+0000", 2024-04-20T01:48:01.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4508999999999999e-05 2024-04-20T01:48:01.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.729655+0000", 2024-04-20T01:48:01.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2072999999999999e-05 2024-04-20T01:48:01.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.729665+0000", 2024-04-20T01:48:01.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9499999999999996e-06 2024-04-20T01:48:01.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.729752+0000", 2024-04-20T01:48:01.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7134999999999998e-05 2024-04-20T01:48:01.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.729762+0000", 2024-04-20T01:48:01.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0729e-05 2024-04-20T01:48:01.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.734446+0000", 2024-04-20T01:48:01.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046834499999999996 2024-04-20T01:48:01.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.734484+0000", 2024-04-20T01:48:01.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7870999999999997e-05 2024-04-20T01:48:01.733 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.733 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.733 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.733 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs2 55/30 ECSubWrite(tid=42, reqid=osd.2.0:42, at_version=55'26, trim_to=0'0, roll_forward_to=52'21))", 2024-04-20T01:48:01.733 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:56.731259+0000", 2024-04-20T01:48:01.734 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9534459340000003, 2024-04-20T01:48:01.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0064248020000000003, 2024-04-20T01:48:01.734 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.734 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.734 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731259+0000", 2024-04-20T01:48:01.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731259+0000", 2024-04-20T01:48:01.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731259+0000", 2024-04-20T01:48:01.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:01.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731283+0000", 2024-04-20T01:48:01.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4437e-05 2024-04-20T01:48:01.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731292+0000", 2024-04-20T01:48:01.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0880000000000001e-06 2024-04-20T01:48:01.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731300+0000", 2024-04-20T01:48:01.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8599999999999993e-06 2024-04-20T01:48:01.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731742+0000", 2024-04-20T01:48:01.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044220700000000002 2024-04-20T01:48:01.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731753+0000", 2024-04-20T01:48:01.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0968000000000001e-05 2024-04-20T01:48:01.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.737642+0000", 2024-04-20T01:48:01.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0058890239999999996 2024-04-20T01:48:01.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.737684+0000", 2024-04-20T01:48:01.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1683000000000001e-05 2024-04-20T01:48:01.737 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.737 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.737 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.738 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 55/30 ECSubWrite(tid=41, reqid=osd.3.0:41, at_version=55'30, trim_to=0'0, roll_forward_to=54'26))", 2024-04-20T01:48:01.738 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:56.732670+0000", 2024-04-20T01:48:01.738 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9520349230000003, 2024-04-20T01:48:01.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080432620000000007, 2024-04-20T01:48:01.738 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.738 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.738 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732670+0000", 2024-04-20T01:48:01.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732670+0000", 2024-04-20T01:48:01.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732669+0000", 2024-04-20T01:48:01.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:01.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732698+0000", 2024-04-20T01:48:01.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8818000000000001e-05 2024-04-20T01:48:01.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732711+0000", 2024-04-20T01:48:01.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2696e-05 2024-04-20T01:48:01.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732721+0000", 2024-04-20T01:48:01.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0006e-05 2024-04-20T01:48:01.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732805+0000", 2024-04-20T01:48:01.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4152000000000003e-05 2024-04-20T01:48:01.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732814+0000", 2024-04-20T01:48:01.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7110000000000007e-06 2024-04-20T01:48:01.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.740690+0000", 2024-04-20T01:48:01.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0078757670000000005 2024-04-20T01:48:01.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.740713+0000", 2024-04-20T01:48:01.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3141000000000001e-05 2024-04-20T01:48:01.742 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.742 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.742 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.742 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 55/30 ECSubWrite(tid=42, reqid=client.4643.0:451, at_version=55'31, trim_to=0'0, roll_forward_to=55'28))", 2024-04-20T01:48:01.742 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:56.739643+0000", 2024-04-20T01:48:01.742 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9450622290000004, 2024-04-20T01:48:01.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056333429999999999, 2024-04-20T01:48:01.742 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.742 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.742 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739643+0000", 2024-04-20T01:48:01.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739643+0000", 2024-04-20T01:48:01.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739641+0000", 2024-04-20T01:48:01.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:01.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739813+0000", 2024-04-20T01:48:01.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017258700000000001 2024-04-20T01:48:01.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739833+0000", 2024-04-20T01:48:01.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9312000000000001e-05 2024-04-20T01:48:01.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739844+0000", 2024-04-20T01:48:01.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1303e-05 2024-04-20T01:48:01.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.740000+0000", 2024-04-20T01:48:01.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015592899999999999 2024-04-20T01:48:01.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.740013+0000", 2024-04-20T01:48:01.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2659e-05 2024-04-20T01:48:01.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.745246+0000", 2024-04-20T01:48:01.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0052337290000000003 2024-04-20T01:48:01.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.745276+0000", 2024-04-20T01:48:01.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9688000000000001e-05 2024-04-20T01:48:01.746 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.746 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.746 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.746 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 55/30 ECSubWrite(tid=43, reqid=client.4643.0:452, at_version=55'32, trim_to=0'0, roll_forward_to=55'28))", 2024-04-20T01:48:01.746 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:56.741637+0000", 2024-04-20T01:48:01.747 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9430683870000003, 2024-04-20T01:48:01.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044886359999999998, 2024-04-20T01:48:01.747 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.747 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.747 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.741637+0000", 2024-04-20T01:48:01.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.741637+0000", 2024-04-20T01:48:01.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.741636+0000", 2024-04-20T01:48:01.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:01.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.741699+0000", 2024-04-20T01:48:01.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3514000000000002e-05 2024-04-20T01:48:01.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.741716+0000", 2024-04-20T01:48:01.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6744000000000001e-05 2024-04-20T01:48:01.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.741727+0000", 2024-04-20T01:48:01.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0616e-05 2024-04-20T01:48:01.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.741781+0000", 2024-04-20T01:48:01.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4110000000000002e-05 2024-04-20T01:48:01.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.741792+0000", 2024-04-20T01:48:01.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1392e-05 2024-04-20T01:48:01.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.746103+0000", 2024-04-20T01:48:01.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043107559999999998 2024-04-20T01:48:01.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.746125+0000", 2024-04-20T01:48:01.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2280999999999999e-05 2024-04-20T01:48:01.750 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.750 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.751 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.751 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 58/57 ECSubWrite(tid=121, reqid=client.4643.0:485, at_version=58'81, trim_to=0'0, roll_forward_to=58'78))", 2024-04-20T01:48:01.751 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:59.819732+0000", 2024-04-20T01:48:01.751 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.8649734520000001, 2024-04-20T01:48:01.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053001919999999996, 2024-04-20T01:48:01.751 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.751 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.751 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819732+0000", 2024-04-20T01:48:01.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819732+0000", 2024-04-20T01:48:01.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819731+0000", 2024-04-20T01:48:01.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:01.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819761+0000", 2024-04-20T01:48:01.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0620999999999997e-05 2024-04-20T01:48:01.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819775+0000", 2024-04-20T01:48:01.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4042999999999999e-05 2024-04-20T01:48:01.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819785+0000", 2024-04-20T01:48:01.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9369999999999999e-06 2024-04-20T01:48:01.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819822+0000", 2024-04-20T01:48:01.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6430000000000002e-05 2024-04-20T01:48:01.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819829+0000", 2024-04-20T01:48:01.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0299999999999996e-06 2024-04-20T01:48:01.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.825010+0000", 2024-04-20T01:48:01.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0051810900000000002 2024-04-20T01:48:01.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.825032+0000", 2024-04-20T01:48:01.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1759e-05 2024-04-20T01:48:01.755 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.755 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.755 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.755 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 58/57 ECSubWrite(tid=123, reqid=client.4643.0:485, at_version=0'0, trim_to=0'0, roll_forward_to=58'81))", 2024-04-20T01:48:01.755 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:59.826183+0000", 2024-04-20T01:48:01.755 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.8585222639999999, 2024-04-20T01:48:01.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040342290000000003, 2024-04-20T01:48:01.755 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:01.755 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:01.755 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:01.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:01.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.826183+0000", 2024-04-20T01:48:01.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:01.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.826183+0000", 2024-04-20T01:48:01.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:01.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:01.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.826182+0000", 2024-04-20T01:48:01.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:01.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:01.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.826204+0000", 2024-04-20T01:48:01.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1614e-05 2024-04-20T01:48:01.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:01.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.826215+0000", 2024-04-20T01:48:01.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1667e-05 2024-04-20T01:48:01.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:01.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.826224+0000", 2024-04-20T01:48:01.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1640000000000001e-06 2024-04-20T01:48:01.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:01.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.826268+0000", 2024-04-20T01:48:01.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4573000000000002e-05 2024-04-20T01:48:01.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:01.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.826280+0000", 2024-04-20T01:48:01.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2084e-05 2024-04-20T01:48:01.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:01.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.830200+0000", 2024-04-20T01:48:01.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003920176 2024-04-20T01:48:01.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:01.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:01.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:01.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.830217+0000", 2024-04-20T01:48:01.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6531000000000001e-05 2024-04-20T01:48:01.759 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.759 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.759 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.759 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:01.759 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:01.759 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:01.759 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-20T01:48:01.761 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:48:01.761 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:48:01.761 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:48:01.762 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-20T01:48:01.763 INFO:tasks.rados.rados.0.smithi165.stdout:208: done (1 left) 2024-04-20T01:48:01.763 INFO:tasks.rados.rados.0.smithi165.stdout:209: read oid 49 snap 15 2024-04-20T01:48:01.763 INFO:tasks.rados.rados.0.smithi165.stdout:209: expect (ObjNum 48 snap 0 seq_num 48) 2024-04-20T01:48:01.763 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 26 current snap is 16 2024-04-20T01:48:01.763 INFO:tasks.rados.rados.0.smithi165.stdout:210: seq_num 96 ranges {3194880=458752,3653632=688128,4341760=475136} 2024-04-20T01:48:01.764 INFO:tasks.rados.rados.0.smithi165.stdout:210: writing smithi16579500-26 from 3194880 to 3653632 tid 1 2024-04-20T01:48:01.764 INFO:tasks.rados.rados.0.smithi165.stdout:210: writing smithi16579500-26 from 3653632 to 4341760 tid 2 2024-04-20T01:48:01.764 INFO:tasks.rados.rados.0.smithi165.stdout:210: writing smithi16579500-26 from 4341760 to 4816896 tid 3 2024-04-20T01:48:01.764 INFO:tasks.rados.rados.0.smithi165.stdout:211: delete oid 48 current snap is 16 2024-04-20T01:48:01.764 INFO:tasks.rados.rados.0.smithi165.stdout:207: expect (ObjNum 4 snap 0 seq_num 4) 2024-04-20T01:48:01.764 INFO:tasks.rados.rados.0.smithi165.stdout:209: expect (ObjNum 48 snap 0 seq_num 48) 2024-04-20T01:48:01.769 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:01.784+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:01.777 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:48:01.781 INFO:tasks.rados.rados.0.smithi165.stdout:210: finishing write tid 1 to smithi16579500-26 2024-04-20T01:48:01.781 INFO:tasks.rados.rados.0.smithi165.stdout:207: done (3 left) 2024-04-20T01:48:01.781 INFO:tasks.rados.rados.0.smithi165.stdout:209: done (2 left) 2024-04-20T01:48:01.781 INFO:tasks.rados.rados.0.smithi165.stdout:211: done (1 left) 2024-04-20T01:48:01.781 INFO:tasks.rados.rados.0.smithi165.stdout:212: rollback oid 48 current snap is 16 2024-04-20T01:48:01.781 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 48 to 14 2024-04-20T01:48:01.781 INFO:tasks.rados.rados.0.smithi165.stdout:210: finishing write tid 2 to smithi16579500-26 2024-04-20T01:48:01.781 INFO:tasks.rados.rados.0.smithi165.stdout:210: finishing write tid 3 to smithi16579500-26 2024-04-20T01:48:01.781 INFO:tasks.rados.rados.0.smithi165.stdout:210: finishing write tid 4 to smithi16579500-26 2024-04-20T01:48:01.781 INFO:tasks.rados.rados.0.smithi165.stdout:210: finishing write tid 5 to smithi16579500-26 2024-04-20T01:48:01.781 INFO:tasks.rados.rados.0.smithi165.stdout:210: oid 26 updating version 0 to 63 2024-04-20T01:48:01.781 INFO:tasks.rados.rados.0.smithi165.stdout:210: oid 26 updating version 63 to 64 2024-04-20T01:48:01.781 INFO:tasks.rados.rados.0.smithi165.stdout:210: oid 26 updating version 64 to 65 2024-04-20T01:48:01.782 INFO:tasks.rados.rados.0.smithi165.stdout:210: oid 26 version 65 is already newer than 62 2024-04-20T01:48:01.782 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 26 v 65 (ObjNum 96 snap 16 seq_num 96) dirty exists 2024-04-20T01:48:01.782 INFO:tasks.rados.rados.0.smithi165.stdout:210: left oid 26 (ObjNum 96 snap 16 seq_num 96) 2024-04-20T01:48:01.782 INFO:tasks.rados.rados.0.smithi165.stdout:210: done (1 left) 2024-04-20T01:48:01.782 INFO:tasks.rados.rados.0.smithi165.stdout:213: delete oid 12 current snap is 16 2024-04-20T01:48:01.782 INFO:tasks.rados.rados.0.smithi165.stdout:212: finishing rollback tid 1 to smithi16579500-48 2024-04-20T01:48:01.782 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 48 v 0 (ObjNum 673 snap 0 seq_num 3856854704) dirty dne 2024-04-20T01:48:01.782 INFO:tasks.rados.rados.0.smithi165.stdout:212: done (1 left) 2024-04-20T01:48:01.783 INFO:tasks.rados.rados.0.smithi165.stdout:213: done (0 left) 2024-04-20T01:48:01.783 INFO:tasks.rados.rados.0.smithi165.stdout:214: delete oid 20 current snap is 16 2024-04-20T01:48:01.791 INFO:tasks.rados.rados.0.smithi165.stdout:214: done (0 left) 2024-04-20T01:48:01.791 INFO:tasks.rados.rados.0.smithi165.stdout:215: delete oid 11 current snap is 16 2024-04-20T01:48:01.799 INFO:tasks.rados.rados.0.smithi165.stdout:215: done (0 left) 2024-04-20T01:48:01.799 INFO:tasks.rados.rados.0.smithi165.stdout:216: copy_from oid 3 from oid 7 current snap is 16 2024-04-20T01:48:01.799 INFO:tasks.rados.rados.0.smithi165.stdout:217: read oid 14 snap 14 2024-04-20T01:48:01.799 INFO:tasks.rados.rados.0.smithi165.stdout:217: expect (ObjNum 7 snap 0 seq_num 7) 2024-04-20T01:48:01.799 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 38 current snap is 16 2024-04-20T01:48:01.799 INFO:tasks.rados.rados.0.smithi165.stdout:218: seq_num 101 ranges {2883584=540672,3424256=507904,3932160=622592,4554752=32768} 2024-04-20T01:48:01.803 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:48:01.817 INFO:tasks.rados.rados.0.smithi165.stdout:218: writing smithi16579500-38 from 2883584 to 3424256 tid 1 2024-04-20T01:48:01.820 INFO:tasks.rados.rados.0.smithi165.stdout:218: writing smithi16579500-38 from 3424256 to 3932160 tid 2 2024-04-20T01:48:01.824 INFO:tasks.rados.rados.0.smithi165.stdout:218: writing smithi16579500-38 from 3932160 to 4554752 tid 3 2024-04-20T01:48:01.824 INFO:tasks.rados.rados.0.smithi165.stdout:218: writing smithi16579500-38 from 4554752 to 4587520 tid 4 2024-04-20T01:48:01.824 INFO:tasks.rados.rados.0.smithi165.stdout:219: snap_remove snap 10 2024-04-20T01:48:01.829 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:48:01.850 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:48:01.870 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:01.884+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:01.872 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:48:01.893 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:48:01.914 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:48:01.919 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:01.919 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:01.931 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-20T01:48:01.937 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:48:01.961 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:48:01.971 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:01.985+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:01.985 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:48:02.006 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:48:02.027 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:48:02.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:48:02.063+0000 7f06acb8c700 0 -- 172.21.15.44:0/3092735850 >> [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] conn(0x7f06a821f880 msgr2=0x7f06a8221c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-20T01:48:02.051 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:48:02.053 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:48:02.053 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:48:02.053 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:48:02.053 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:48:02.053 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:48:02.053 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:48:02.053 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:48:02.053 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:48:02.053 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:48:02.053 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:48:02.053 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:48:02.053 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:48:02.054 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:48:02.054 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:48:02.054 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:48:02.085 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:02.085 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:02.085 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:02.100+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:02.097 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-20T01:48:02.180 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:02.195+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:02.273 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:02.287+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:02.280 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:02.281 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:02.292 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-20T01:48:02.377 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:02.391+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:02.475 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:02.489+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:02.479 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:02.479 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:02.491 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-20T01:48:02.576 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:48:02.591+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:48:02.677 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:02.677 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:02.677 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:02.691+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:02.689 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-20T01:48:02.692 INFO:tasks.rados.rados.0.smithi165.stdout:219: done (3 left) 2024-04-20T01:48:02.692 INFO:tasks.rados.rados.0.smithi165.stdout:220: snap_remove snap 13 2024-04-20T01:48:02.700 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is unset 2024-04-20T01:48:02.778 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:02.793+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:02.843 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:02.843 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:02.855 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-20T01:48:02.879 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:48:02.893+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:48:02.980 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:02.995+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:02.998 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:02.999 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:03.010 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-20T01:48:03.081 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:03.095+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:03.154 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:03.155 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:03.166 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-20T01:48:03.182 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:03.196+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:03.282 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:03.297+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:03.310 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:03.311 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:03.322 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-20T01:48:03.384 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:03.398+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:03.466 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:03.466 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:03.477 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-20T01:48:03.484 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:03.499+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:03.585 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:03.599+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:03.621 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:03.622 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:03.633 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-20T01:48:03.686 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:48:03.700+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:48:03.696 INFO:tasks.rados.rados.0.smithi165.stdout:220: done (3 left) 2024-04-20T01:48:03.696 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 40 current snap is 16 2024-04-20T01:48:03.696 INFO:tasks.rados.rados.0.smithi165.stdout:221: seq_num 102 ranges {0=557056,557056=638976,1196032=524288} 2024-04-20T01:48:03.699 INFO:tasks.rados.rados.0.smithi165.stdout:221: writing smithi16579500-40 from 0 to 557056 tid 1 2024-04-20T01:48:03.703 INFO:tasks.rados.rados.0.smithi165.stdout:221: writing smithi16579500-40 from 557056 to 1196032 tid 2 2024-04-20T01:48:03.707 INFO:tasks.rados.rados.0.smithi165.stdout:221: writing smithi16579500-40 from 1196032 to 1720320 tid 3 2024-04-20T01:48:03.707 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 39 current snap is 16 2024-04-20T01:48:03.707 INFO:tasks.rados.rados.0.smithi165.stdout:216: finishing copy_from to smithi16579500-3 2024-04-20T01:48:03.707 INFO:tasks.rados.rados.0.smithi165.stdout:216: got expected ENOENT (src dne) 2024-04-20T01:48:03.707 INFO:tasks.rados.rados.0.smithi165.stdout:216: finishing copy_from racing read to smithi16579500-3 2024-04-20T01:48:03.708 INFO:tasks.rados.rados.0.smithi165.stdout:217: expect (ObjNum 7 snap 0 seq_num 7) 2024-04-20T01:48:03.711 INFO:tasks.rados.rados.0.smithi165.stdout:218: finishing write tid 1 to smithi16579500-38 2024-04-20T01:48:03.711 INFO:tasks.rados.rados.0.smithi165.stdout:218: finishing write tid 2 to smithi16579500-38 2024-04-20T01:48:03.711 INFO:tasks.rados.rados.0.smithi165.stdout:218: finishing write tid 3 to smithi16579500-38 2024-04-20T01:48:03.711 INFO:tasks.rados.rados.0.smithi165.stdout:222: seq_num 103 ranges {3194880=573440,3768320=786432,4554752=376832} 2024-04-20T01:48:03.729 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-20T01:48:03.729 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-20T01:48:03.729 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-04-20T01:48:03.729 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-20T01:48:03.732 INFO:tasks.rados.rados.0.smithi165.stdout:222: writing smithi16579500-39 from 3194880 to 3768320 tid 1 2024-04-20T01:48:03.736 INFO:tasks.rados.rados.0.smithi165.stdout:222: writing smithi16579500-39 from 3768320 to 4554752 tid 2 2024-04-20T01:48:03.739 INFO:tasks.rados.rados.0.smithi165.stdout:222: writing smithi16579500-39 from 4554752 to 4931584 tid 3 2024-04-20T01:48:03.739 INFO:tasks.rados.rados.0.smithi165.stdout:216: done (4 left) 2024-04-20T01:48:03.739 INFO:tasks.rados.rados.0.smithi165.stdout:217: done (3 left) 2024-04-20T01:48:03.740 INFO:tasks.rados.rados.0.smithi165.stdout:223: copy_from oid 35 from oid 13 current snap is 16 2024-04-20T01:48:03.740 INFO:tasks.rados.rados.0.smithi165.stdout:218: finishing write tid 4 to smithi16579500-38 2024-04-20T01:48:03.740 INFO:tasks.rados.rados.0.smithi165.stdout:218: finishing write tid 5 to smithi16579500-38 2024-04-20T01:48:03.740 INFO:tasks.rados.rados.0.smithi165.stdout:218: finishing write tid 6 to smithi16579500-38 2024-04-20T01:48:03.740 INFO:tasks.rados.rados.0.smithi165.stdout:218: oid 38 updating version 0 to 67 2024-04-20T01:48:03.740 INFO:tasks.rados.rados.0.smithi165.stdout:218: oid 38 updating version 67 to 69 2024-04-20T01:48:03.740 INFO:tasks.rados.rados.0.smithi165.stdout:218: oid 38 version 69 is already newer than 68 2024-04-20T01:48:03.740 INFO:tasks.rados.rados.0.smithi165.stdout:218: oid 38 version 69 is already newer than 66 2024-04-20T01:48:03.740 INFO:tasks.rados.rados.0.smithi165.stdout:218: oid 38 version 69 is already newer than 65 2024-04-20T01:48:03.740 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 38 v 69 (ObjNum 101 snap 16 seq_num 101) dirty exists 2024-04-20T01:48:03.740 INFO:tasks.rados.rados.0.smithi165.stdout:218: left oid 38 (ObjNum 101 snap 16 seq_num 101) 2024-04-20T01:48:03.740 INFO:tasks.rados.rados.0.smithi165.stdout:221: finishing write tid 1 to smithi16579500-40 2024-04-20T01:48:03.740 INFO:tasks.rados.rados.0.smithi165.stdout:221: finishing write tid 2 to smithi16579500-40 2024-04-20T01:48:03.740 INFO:tasks.rados.rados.0.smithi165.stdout:218: done (3 left) 2024-04-20T01:48:03.741 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 2 current snap is 16 2024-04-20T01:48:03.741 INFO:tasks.rados.rados.0.smithi165.stdout:221: finishing write tid 3 to smithi16579500-40 2024-04-20T01:48:03.741 INFO:tasks.rados.rados.0.smithi165.stdout:221: finishing write tid 4 to smithi16579500-40 2024-04-20T01:48:03.741 INFO:tasks.rados.rados.0.smithi165.stdout:224: seq_num 104 ranges {458752=589824,1048576=540672,1589248=622592} 2024-04-20T01:48:03.745 INFO:tasks.rados.rados.0.smithi165.stdout:224: writing smithi16579500-2 from 458752 to 1048576 tid 1 2024-04-20T01:48:03.748 INFO:tasks.rados.rados.0.smithi165.stdout:224: writing smithi16579500-2 from 1048576 to 1589248 tid 2 2024-04-20T01:48:03.753 INFO:tasks.rados.rados.0.smithi165.stdout:224: writing smithi16579500-2 from 1589248 to 2211840 tid 3 2024-04-20T01:48:03.753 INFO:tasks.rados.rados.0.smithi165.stdout:225: rmattr oid 41 current snap is 16 2024-04-20T01:48:03.753 INFO:tasks.rados.rados.0.smithi165.stdout:221: finishing write tid 5 to smithi16579500-40 2024-04-20T01:48:03.753 INFO:tasks.rados.rados.0.smithi165.stdout:221: oid 40 updating version 0 to 18 2024-04-20T01:48:03.753 INFO:tasks.rados.rados.0.smithi165.stdout:221: oid 40 version 18 is already newer than 15 2024-04-20T01:48:03.753 INFO:tasks.rados.rados.0.smithi165.stdout:221: oid 40 version 18 is already newer than 16 2024-04-20T01:48:03.753 INFO:tasks.rados.rados.0.smithi165.stdout:221: oid 40 version 18 is already newer than 17 2024-04-20T01:48:03.754 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 40 v 18 (ObjNum 102 snap 16 seq_num 102) dirty exists 2024-04-20T01:48:03.754 INFO:tasks.rados.rados.0.smithi165.stdout:221: left oid 40 (ObjNum 102 snap 16 seq_num 102) 2024-04-20T01:48:03.754 INFO:tasks.rados.rados.0.smithi165.stdout:221: done (4 left) 2024-04-20T01:48:03.754 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 22 current snap is 16 2024-04-20T01:48:03.754 INFO:tasks.rados.rados.0.smithi165.stdout:222: finishing write tid 1 to smithi16579500-39 2024-04-20T01:48:03.754 INFO:tasks.rados.rados.0.smithi165.stdout:222: finishing write tid 2 to smithi16579500-39 2024-04-20T01:48:03.754 INFO:tasks.rados.rados.0.smithi165.stdout:222: finishing write tid 3 to smithi16579500-39 2024-04-20T01:48:03.754 INFO:tasks.rados.rados.0.smithi165.stdout:226: seq_num 105 ranges {360448=606208,966656=688128,1654784=475136} 2024-04-20T01:48:03.758 INFO:tasks.rados.rados.0.smithi165.stdout:226: writing smithi16579500-22 from 360448 to 966656 tid 1 2024-04-20T01:48:03.762 INFO:tasks.rados.rados.0.smithi165.stdout:226: writing smithi16579500-22 from 966656 to 1654784 tid 2 2024-04-20T01:48:03.766 INFO:tasks.rados.rados.0.smithi165.stdout:226: writing smithi16579500-22 from 1654784 to 2129920 tid 3 2024-04-20T01:48:03.766 INFO:tasks.rados.rados.0.smithi165.stdout:227: read oid 24 snap -1 2024-04-20T01:48:03.766 INFO:tasks.rados.rados.0.smithi165.stdout:227: expect (ObjNum 81 snap 8 seq_num 81) 2024-04-20T01:48:03.766 INFO:tasks.rados.rados.0.smithi165.stdout:228: rollback oid 31 current snap is 16 2024-04-20T01:48:03.766 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 31 to 15 2024-04-20T01:48:03.766 INFO:tasks.rados.rados.0.smithi165.stdout:222: finishing write tid 4 to smithi16579500-39 2024-04-20T01:48:03.766 INFO:tasks.rados.rados.0.smithi165.stdout:223: finishing copy_from racing read to smithi16579500-35 2024-04-20T01:48:03.766 INFO:tasks.rados.rados.0.smithi165.stdout:222: finishing write tid 5 to smithi16579500-39 2024-04-20T01:48:03.766 INFO:tasks.rados.rados.0.smithi165.stdout:222: oid 39 updating version 0 to 94 2024-04-20T01:48:03.766 INFO:tasks.rados.rados.0.smithi165.stdout:222: oid 39 version 94 is already newer than 93 2024-04-20T01:48:03.766 INFO:tasks.rados.rados.0.smithi165.stdout:222: oid 39 version 94 is already newer than 91 2024-04-20T01:48:03.767 INFO:tasks.rados.rados.0.smithi165.stdout:222: oid 39 version 94 is already newer than 92 2024-04-20T01:48:03.767 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 39 v 94 (ObjNum 103 snap 16 seq_num 103) dirty exists 2024-04-20T01:48:03.767 INFO:tasks.rados.rados.0.smithi165.stdout:222: left oid 39 (ObjNum 103 snap 16 seq_num 103) 2024-04-20T01:48:03.767 INFO:tasks.rados.rados.0.smithi165.stdout:224: finishing write tid 1 to smithi16579500-2 2024-04-20T01:48:03.767 INFO:tasks.rados.rados.0.smithi165.stdout:224: finishing write tid 2 to smithi16579500-2 2024-04-20T01:48:03.767 INFO:tasks.rados.rados.0.smithi165.stdout:222: done (6 left) 2024-04-20T01:48:03.767 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 48 current snap is 16 2024-04-20T01:48:03.767 INFO:tasks.rados.rados.0.smithi165.stdout:229: seq_num 106 ranges {0=622592,622592=442368,1064960=524288,1589248=196608} 2024-04-20T01:48:03.769 INFO:tasks.rados.rados.0.smithi165.stdout:229: writing smithi16579500-48 from 0 to 622592 tid 1 2024-04-20T01:48:03.772 INFO:tasks.rados.rados.0.smithi165.stdout:229: writing smithi16579500-48 from 622592 to 1064960 tid 2 2024-04-20T01:48:03.775 INFO:tasks.rados.rados.0.smithi165.stdout:229: writing smithi16579500-48 from 1064960 to 1589248 tid 3 2024-04-20T01:48:03.777 INFO:tasks.rados.rados.0.smithi165.stdout:229: writing smithi16579500-48 from 1589248 to 1785856 tid 4 2024-04-20T01:48:03.777 INFO:tasks.rados.rados.0.smithi165.stdout:230: snap_create 2024-04-20T01:48:03.777 INFO:tasks.rados.rados.0.smithi165.stdout:224: finishing write tid 3 to smithi16579500-2 2024-04-20T01:48:03.777 INFO:tasks.rados.rados.0.smithi165.stdout:224: finishing write tid 4 to smithi16579500-2 2024-04-20T01:48:03.778 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 41 v 101 (ObjNum 75 snap 7 seq_num 75) dirty exists 2024-04-20T01:48:03.778 INFO:tasks.rados.rados.0.smithi165.stdout:224: finishing write tid 5 to smithi16579500-2 2024-04-20T01:48:03.778 INFO:tasks.rados.rados.0.smithi165.stdout:224: oid 2 updating version 0 to 97 2024-04-20T01:48:03.778 INFO:tasks.rados.rados.0.smithi165.stdout:224: oid 2 updating version 97 to 99 2024-04-20T01:48:03.778 INFO:tasks.rados.rados.0.smithi165.stdout:224: oid 2 version 99 is already newer than 96 2024-04-20T01:48:03.778 INFO:tasks.rados.rados.0.smithi165.stdout:224: oid 2 version 99 is already newer than 98 2024-04-20T01:48:03.778 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 2 v 99 (ObjNum 104 snap 16 seq_num 104) dirty exists 2024-04-20T01:48:03.778 INFO:tasks.rados.rados.0.smithi165.stdout:224: left oid 2 (ObjNum 104 snap 16 seq_num 104) 2024-04-20T01:48:03.778 INFO:tasks.rados.rados.0.smithi165.stdout:228: finishing rollback tid 0 to smithi16579500-31 2024-04-20T01:48:03.778 INFO:tasks.rados.rados.0.smithi165.stdout:223: finishing copy_from to smithi16579500-35 2024-04-20T01:48:03.778 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 35 v 38 (ObjNum 70 snap 5 seq_num 70) dirty exists 2024-04-20T01:48:03.778 INFO:tasks.rados.rados.0.smithi165.stdout:226: finishing write tid 1 to smithi16579500-22 2024-04-20T01:48:03.778 INFO:tasks.rados.rados.0.smithi165.stdout:228: finishing rollback tid 1 to smithi16579500-31 2024-04-20T01:48:03.779 INFO:tasks.rados.rados.0.smithi165.stdout:228: finishing rollback tid 2 to smithi16579500-31 2024-04-20T01:48:03.779 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 31 v 79 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-04-20T01:48:03.779 INFO:tasks.rados.rados.0.smithi165.stdout:229: finishing write tid 1 to smithi16579500-48 2024-04-20T01:48:03.780 INFO:tasks.rados.rados.0.smithi165.stdout:226: finishing write tid 2 to smithi16579500-22 2024-04-20T01:48:03.780 INFO:tasks.rados.rados.0.smithi165.stdout:226: finishing write tid 3 to smithi16579500-22 2024-04-20T01:48:03.780 INFO:tasks.rados.rados.0.smithi165.stdout:226: finishing write tid 4 to smithi16579500-22 2024-04-20T01:48:03.782 INFO:tasks.rados.rados.0.smithi165.stdout:229: finishing write tid 2 to smithi16579500-48 2024-04-20T01:48:03.782 INFO:tasks.rados.rados.0.smithi165.stdout:226: finishing write tid 5 to smithi16579500-22 2024-04-20T01:48:03.783 INFO:tasks.rados.rados.0.smithi165.stdout:226: oid 22 updating version 0 to 14 2024-04-20T01:48:03.783 INFO:tasks.rados.rados.0.smithi165.stdout:226: oid 22 updating version 14 to 16 2024-04-20T01:48:03.783 INFO:tasks.rados.rados.0.smithi165.stdout:226: oid 22 version 16 is already newer than 15 2024-04-20T01:48:03.783 INFO:tasks.rados.rados.0.smithi165.stdout:226: oid 22 version 16 is already newer than 13 2024-04-20T01:48:03.783 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 22 v 16 (ObjNum 105 snap 16 seq_num 105) dirty exists 2024-04-20T01:48:03.783 INFO:tasks.rados.rados.0.smithi165.stdout:226: left oid 22 (ObjNum 105 snap 16 seq_num 105) 2024-04-20T01:48:03.783 INFO:tasks.rados.rados.0.smithi165.stdout:229: finishing write tid 3 to smithi16579500-48 2024-04-20T01:48:03.783 INFO:tasks.rados.rados.0.smithi165.stdout:229: finishing write tid 4 to smithi16579500-48 2024-04-20T01:48:03.784 INFO:tasks.rados.rados.0.smithi165.stdout:229: finishing write tid 5 to smithi16579500-48 2024-04-20T01:48:03.786 INFO:tasks.rados.rados.0.smithi165.stdout:229: finishing write tid 6 to smithi16579500-48 2024-04-20T01:48:03.786 INFO:tasks.rados.rados.0.smithi165.stdout:229: oid 48 updating version 0 to 42 2024-04-20T01:48:03.786 INFO:tasks.rados.rados.0.smithi165.stdout:229: oid 48 version 42 is already newer than 40 2024-04-20T01:48:03.786 INFO:tasks.rados.rados.0.smithi165.stdout:229: oid 48 version 42 is already newer than 41 2024-04-20T01:48:03.786 INFO:tasks.rados.rados.0.smithi165.stdout:229: oid 48 version 42 is already newer than 39 2024-04-20T01:48:03.786 INFO:tasks.rados.rados.0.smithi165.stdout:229: oid 48 updating version 42 to 43 2024-04-20T01:48:03.786 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 48 v 43 (ObjNum 106 snap 16 seq_num 106) dirty exists 2024-04-20T01:48:03.786 INFO:tasks.rados.rados.0.smithi165.stdout:229: left oid 48 (ObjNum 106 snap 16 seq_num 106) 2024-04-20T01:48:03.787 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:03.800+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:03.792 INFO:tasks.rados.rados.0.smithi165.stdout:227: expect (ObjNum 81 snap 8 seq_num 81) 2024-04-20T01:48:03.859 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:03.860 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:03.872 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-20T01:48:03.887 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:03.901+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:03.988 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:04.002+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:04.003 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:04.003 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:04.014 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-20T01:48:04.067 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-20T01:48:04.110 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:04.125+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:04.131 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:48:04.131 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:04.132 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 62, 2024-04-20T01:48:04.132 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "9c2f64c1-aa35-4337-9f5c-081b9e3a2938", 2024-04-20T01:48:04.132 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2024-04-20T01:47:02.934241+0000", 2024-04-20T01:48:04.132 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-04-20T01:48:03.704451+0000", 2024-04-20T01:48:04.132 INFO:teuthology.orchestra.run.smithi044.stdout: "last_up_change": "2024-04-20T01:47:15.986754+0000", 2024-04-20T01:48:04.132 INFO:teuthology.orchestra.run.smithi044.stdout: "last_in_change": "2024-04-20T01:47:07.894719+0000", 2024-04-20T01:48:04.132 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-20T01:48:04.132 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_num": 5799936, 2024-04-20T01:48:04.132 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_set": [ 2024-04-20T01:48:04.132 INFO:teuthology.orchestra.run.smithi044.stdout: "pglog_hardlimit", 2024-04-20T01:48:04.132 INFO:teuthology.orchestra.run.smithi044.stdout: "purged_snapdirs", 2024-04-20T01:48:04.132 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_deletes", 2024-04-20T01:48:04.132 INFO:teuthology.orchestra.run.smithi044.stdout: "sortbitwise" 2024-04-20T01:48:04.132 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:48:04.132 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_version": 5, 2024-04-20T01:48:04.133 INFO:teuthology.orchestra.run.smithi044.stdout: "full_ratio": 0.94999998807907104, 2024-04-20T01:48:04.133 INFO:teuthology.orchestra.run.smithi044.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-20T01:48:04.133 INFO:teuthology.orchestra.run.smithi044.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-20T01:48:04.133 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_snapshot": "", 2024-04-20T01:48:04.133 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_max": 3, 2024-04-20T01:48:04.133 INFO:teuthology.orchestra.run.smithi044.stdout: "max_osd": 16, 2024-04-20T01:48:04.133 INFO:teuthology.orchestra.run.smithi044.stdout: "require_min_compat_client": "luminous", 2024-04-20T01:48:04.133 INFO:teuthology.orchestra.run.smithi044.stdout: "min_compat_client": "jewel", 2024-04-20T01:48:04.133 INFO:teuthology.orchestra.run.smithi044.stdout: "require_osd_release": "reef", 2024-04-20T01:48:04.133 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_crimson": false, 2024-04-20T01:48:04.133 INFO:teuthology.orchestra.run.smithi044.stdout: "pools": [ 2024-04-20T01:48:04.133 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.133 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 1, 2024-04-20T01:48:04.133 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": ".mgr", 2024-04-20T01:48:04.134 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-04-20T01:47:15.195181+0000", 2024-04-20T01:48:04.134 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T01:48:04.134 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool", 2024-04-20T01:48:04.134 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 1, 2024-04-20T01:48:04.134 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 2, 2024-04-20T01:48:04.134 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 1, 2024-04-20T01:48:04.134 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 0, 2024-04-20T01:48:04.134 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-04-20T01:48:04.134 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-04-20T01:48:04.134 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-04-20T01:48:04.134 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-20T01:48:04.135 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-04-20T01:48:04.135 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-04-20T01:48:04.135 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 1, 2024-04-20T01:48:04.135 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 1, 2024-04-20T01:48:04.135 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 1, 2024-04-20T01:48:04.135 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 1, 2024-04-20T01:48:04.136 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 1, 2024-04-20T01:48:04.136 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-04-20T01:48:04.136 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-04-20T01:48:04.136 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-04-20T01:48:04.136 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-04-20T01:48:04.136 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-04-20T01:48:04.136 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-04-20T01:48:04.136 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-04-20T01:48:04.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.136 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "24", 2024-04-20T01:48:04.136 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-04-20T01:48:04.136 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-20T01:48:04.136 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-04-20T01:48:04.136 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-04-20T01:48:04.137 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-04-20T01:48:04.137 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 0, 2024-04-20T01:48:04.137 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 0, 2024-04-20T01:48:04.137 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-04-20T01:48:04.137 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-04-20T01:48:04.137 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-04-20T01:48:04.137 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-04-20T01:48:04.137 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-04-20T01:48:04.137 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-04-20T01:48:04.137 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-04-20T01:48:04.137 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-04-20T01:48:04.137 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-04-20T01:48:04.137 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-04-20T01:48:04.137 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-04-20T01:48:04.138 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-20T01:48:04.138 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-20T01:48:04.138 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-20T01:48:04.138 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-04-20T01:48:04.138 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-04-20T01:48:04.138 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "", 2024-04-20T01:48:04.138 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-04-20T01:48:04.138 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-04-20T01:48:04.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.138 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-04-20T01:48:04.138 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-04-20T01:48:04.138 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-04-20T01:48:04.138 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-04-20T01:48:04.138 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-04-20T01:48:04.139 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-04-20T01:48:04.139 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-04-20T01:48:04.139 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-04-20T01:48:04.139 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 0, 2024-04-20T01:48:04.139 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-04-20T01:48:04.139 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-04-20T01:48:04.139 INFO:teuthology.orchestra.run.smithi044.stdout: "options": { 2024-04-20T01:48:04.139 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_max": 32, 2024-04-20T01:48:04.139 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_min": 1 2024-04-20T01:48:04.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.139 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-04-20T01:48:04.139 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": {} 2024-04-20T01:48:04.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.139 INFO:teuthology.orchestra.run.smithi044.stdout: "read_balance": { 2024-04-20T01:48:04.139 INFO:teuthology.orchestra.run.smithi044.stdout: "score_acting": 15.380000114440918, 2024-04-20T01:48:04.140 INFO:teuthology.orchestra.run.smithi044.stdout: "score_stable": 15.380000114440918, 2024-04-20T01:48:04.140 INFO:teuthology.orchestra.run.smithi044.stdout: "optimal_score": 0.12999999523162842, 2024-04-20T01:48:04.140 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_acting": 2, 2024-04-20T01:48:04.140 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_stable": 2, 2024-04-20T01:48:04.140 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity_weighted": 1, 2024-04-20T01:48:04.140 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity": 1, 2024-04-20T01:48:04.140 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity_weighted": 1 2024-04-20T01:48:04.140 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.140 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 2, 2024-04-20T01:48:04.140 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": "rbd", 2024-04-20T01:48:04.140 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-04-20T01:47:18.448004+0000", 2024-04-20T01:48:04.140 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T01:48:04.140 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool", 2024-04-20T01:48:04.141 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 1, 2024-04-20T01:48:04.141 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 2, 2024-04-20T01:48:04.141 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 1, 2024-04-20T01:48:04.141 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 0, 2024-04-20T01:48:04.141 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-04-20T01:48:04.141 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-04-20T01:48:04.141 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-04-20T01:48:04.141 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-20T01:48:04.141 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-04-20T01:48:04.141 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-04-20T01:48:04.141 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 8, 2024-04-20T01:48:04.141 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 8, 2024-04-20T01:48:04.141 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 8, 2024-04-20T01:48:04.142 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 8, 2024-04-20T01:48:04.142 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 8, 2024-04-20T01:48:04.142 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-04-20T01:48:04.142 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-04-20T01:48:04.142 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-04-20T01:48:04.142 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-04-20T01:48:04.142 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-04-20T01:48:04.142 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-04-20T01:48:04.142 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-04-20T01:48:04.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.142 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "27", 2024-04-20T01:48:04.142 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-04-20T01:48:04.142 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-20T01:48:04.142 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-04-20T01:48:04.142 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-04-20T01:48:04.143 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-04-20T01:48:04.143 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 0, 2024-04-20T01:48:04.143 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 0, 2024-04-20T01:48:04.143 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-04-20T01:48:04.143 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-04-20T01:48:04.143 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-04-20T01:48:04.143 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-04-20T01:48:04.143 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-04-20T01:48:04.143 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-04-20T01:48:04.143 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-04-20T01:48:04.143 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-04-20T01:48:04.143 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-04-20T01:48:04.143 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-04-20T01:48:04.143 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-04-20T01:48:04.143 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-20T01:48:04.144 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-20T01:48:04.144 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-20T01:48:04.144 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-04-20T01:48:04.144 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-04-20T01:48:04.144 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "", 2024-04-20T01:48:04.144 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-04-20T01:48:04.144 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-04-20T01:48:04.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.144 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-04-20T01:48:04.144 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-04-20T01:48:04.144 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-04-20T01:48:04.144 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-04-20T01:48:04.144 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-04-20T01:48:04.144 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-04-20T01:48:04.144 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-04-20T01:48:04.145 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-04-20T01:48:04.145 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 0, 2024-04-20T01:48:04.145 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-04-20T01:48:04.145 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-04-20T01:48:04.145 INFO:teuthology.orchestra.run.smithi044.stdout: "options": {}, 2024-04-20T01:48:04.145 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-04-20T01:48:04.145 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd": {} 2024-04-20T01:48:04.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.145 INFO:teuthology.orchestra.run.smithi044.stdout: "read_balance": { 2024-04-20T01:48:04.145 INFO:teuthology.orchestra.run.smithi044.stdout: "score_acting": 3.9900000095367432, 2024-04-20T01:48:04.145 INFO:teuthology.orchestra.run.smithi044.stdout: "score_stable": 3.9900000095367432, 2024-04-20T01:48:04.145 INFO:teuthology.orchestra.run.smithi044.stdout: "optimal_score": 0.68999999761581421, 2024-04-20T01:48:04.145 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_acting": 2.75, 2024-04-20T01:48:04.145 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_stable": 2.75, 2024-04-20T01:48:04.145 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity_weighted": 1, 2024-04-20T01:48:04.145 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity": 1, 2024-04-20T01:48:04.146 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity_weighted": 1 2024-04-20T01:48:04.146 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.146 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.146 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 3, 2024-04-20T01:48:04.146 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": "unique_pool_0", 2024-04-20T01:48:04.146 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-04-20T01:47:30.734962+0000", 2024-04-20T01:48:04.146 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 8193, 2024-04-20T01:48:04.146 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-04-20T01:48:04.146 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 3, 2024-04-20T01:48:04.146 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 7, 2024-04-20T01:48:04.146 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 5, 2024-04-20T01:48:04.146 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 1, 2024-04-20T01:48:04.146 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-04-20T01:48:04.146 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-04-20T01:48:04.147 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-04-20T01:48:04.147 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-20T01:48:04.147 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-04-20T01:48:04.147 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-04-20T01:48:04.147 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 16, 2024-04-20T01:48:04.147 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 16, 2024-04-20T01:48:04.147 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 16, 2024-04-20T01:48:04.147 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 16, 2024-04-20T01:48:04.147 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 16, 2024-04-20T01:48:04.147 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-04-20T01:48:04.147 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-04-20T01:48:04.147 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-04-20T01:48:04.147 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-04-20T01:48:04.147 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-04-20T01:48:04.147 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-04-20T01:48:04.148 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-04-20T01:48:04.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.148 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "62", 2024-04-20T01:48:04.148 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-04-20T01:48:04.148 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-20T01:48:04.148 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-04-20T01:48:04.148 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-04-20T01:48:04.148 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-04-20T01:48:04.148 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 30, 2024-04-20T01:48:04.148 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 62, 2024-04-20T01:48:04.148 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-04-20T01:48:04.148 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-04-20T01:48:04.148 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-04-20T01:48:04.148 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-04-20T01:48:04.148 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-04-20T01:48:04.149 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-04-20T01:48:04.149 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-04-20T01:48:04.149 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-04-20T01:48:04.149 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-04-20T01:48:04.149 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-04-20T01:48:04.149 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-04-20T01:48:04.149 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-20T01:48:04.149 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-20T01:48:04.149 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-20T01:48:04.149 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-04-20T01:48:04.149 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-04-20T01:48:04.149 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "shecprofile", 2024-04-20T01:48:04.149 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-04-20T01:48:04.149 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-04-20T01:48:04.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.150 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-04-20T01:48:04.150 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-04-20T01:48:04.150 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-04-20T01:48:04.150 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-04-20T01:48:04.150 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-04-20T01:48:04.150 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-04-20T01:48:04.150 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-04-20T01:48:04.150 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-04-20T01:48:04.150 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 16384, 2024-04-20T01:48:04.150 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-04-20T01:48:04.150 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-04-20T01:48:04.150 INFO:teuthology.orchestra.run.smithi044.stdout: "options": {}, 2024-04-20T01:48:04.150 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-04-20T01:48:04.150 INFO:teuthology.orchestra.run.smithi044.stdout: "rados": {} 2024-04-20T01:48:04.150 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.151 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.151 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:48:04.151 INFO:teuthology.orchestra.run.smithi044.stdout: "osds": [ 2024-04-20T01:48:04.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.151 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 0, 2024-04-20T01:48:04.151 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "3903b88f-ff2b-4252-9480-92b701b1e59b", 2024-04-20T01:48:04.151 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:04.151 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:04.151 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:04.151 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:04.151 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:04.151 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:04.151 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:04.151 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:04.151 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:04.152 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:04.152 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:04.152 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.152 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.152 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6824", 2024-04-20T01:48:04.152 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:48:04.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.152 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.152 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.152 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6825", 2024-04-20T01:48:04.152 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:48:04.152 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.152 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.152 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:04.153 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.153 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.153 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6826", 2024-04-20T01:48:04.153 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:48:04.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.153 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.153 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.153 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6827", 2024-04-20T01:48:04.153 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:48:04.153 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.153 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.153 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:04.153 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.154 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.154 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6830", 2024-04-20T01:48:04.154 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:48:04.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.154 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.154 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6831", 2024-04-20T01:48:04.154 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:48:04.154 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.154 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.154 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:04.154 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.154 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.155 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.155 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6828", 2024-04-20T01:48:04.155 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:48:04.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.155 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.155 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6829", 2024-04-20T01:48:04.155 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:48:04.155 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.155 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.155 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6825/43518", 2024-04-20T01:48:04.155 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6827/43518", 2024-04-20T01:48:04.155 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6831/43518", 2024-04-20T01:48:04.155 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6829/43518", 2024-04-20T01:48:04.156 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:04.156 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:04.156 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:04.156 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.156 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 1, 2024-04-20T01:48:04.156 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "c5ff0175-b4f0-40a5-86c7-7a0a351bf070", 2024-04-20T01:48:04.156 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:04.156 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:04.156 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:04.156 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:04.156 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:04.156 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:04.156 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:04.156 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 0, 2024-04-20T01:48:04.157 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:04.157 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:04.157 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:04.157 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.157 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.157 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6816", 2024-04-20T01:48:04.157 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:48:04.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.157 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.157 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6817", 2024-04-20T01:48:04.157 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:48:04.157 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.157 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.158 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:04.158 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.158 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.158 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6818", 2024-04-20T01:48:04.158 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:48:04.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.158 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.158 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6819", 2024-04-20T01:48:04.158 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:48:04.158 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.158 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.159 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:04.159 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.159 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.159 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6822", 2024-04-20T01:48:04.159 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:48:04.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.159 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.159 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6823", 2024-04-20T01:48:04.159 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:48:04.159 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.159 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.159 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6820", 2024-04-20T01:48:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:48:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6821", 2024-04-20T01:48:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:48:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.121:6817/74361", 2024-04-20T01:48:04.160 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.121:6819/74361", 2024-04-20T01:48:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.121:6823/74361", 2024-04-20T01:48:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.121:6821/74361", 2024-04-20T01:48:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 2, 2024-04-20T01:48:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "7af325c1-36b5-420c-a2a9-44c00a002977", 2024-04-20T01:48:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:04.161 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6809", 2024-04-20T01:48:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:48:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6811", 2024-04-20T01:48:04.162 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:48:04.163 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.163 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.163 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:04.163 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.163 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.163 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6814", 2024-04-20T01:48:04.163 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:48:04.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.163 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.163 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6816", 2024-04-20T01:48:04.163 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:48:04.163 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.164 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.164 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:04.164 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.164 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.164 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6822", 2024-04-20T01:48:04.164 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:48:04.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.164 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.164 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6823", 2024-04-20T01:48:04.164 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:48:04.164 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.164 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.164 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:04.165 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.165 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.165 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6819", 2024-04-20T01:48:04.165 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:48:04.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.165 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.165 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6821", 2024-04-20T01:48:04.165 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:48:04.165 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.165 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.165 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.123:6811/43108", 2024-04-20T01:48:04.166 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.123:6816/43108", 2024-04-20T01:48:04.166 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.123:6823/43108", 2024-04-20T01:48:04.166 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.123:6821/43108", 2024-04-20T01:48:04.166 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:04.166 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:04.166 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:04.166 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.166 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 3, 2024-04-20T01:48:04.166 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "99334f43-83ea-4c3f-b91d-ae72c8361b8c", 2024-04-20T01:48:04.166 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:04.166 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:04.166 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:04.166 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:04.167 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:04.167 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:04.167 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:04.167 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:04.167 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:04.167 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:04.167 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:04.167 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.167 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.167 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6824", 2024-04-20T01:48:04.167 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:48:04.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.167 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.168 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6825", 2024-04-20T01:48:04.168 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:48:04.168 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.168 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.168 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:04.168 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.168 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.168 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6826", 2024-04-20T01:48:04.168 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:48:04.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.168 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.168 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6827", 2024-04-20T01:48:04.169 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:48:04.169 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.169 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.169 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:04.169 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.169 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.169 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6830", 2024-04-20T01:48:04.169 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:48:04.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.169 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.169 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6831", 2024-04-20T01:48:04.169 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:48:04.170 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.170 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.170 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:04.170 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.170 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.170 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6828", 2024-04-20T01:48:04.170 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:48:04.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.170 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.170 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.170 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6829", 2024-04-20T01:48:04.170 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:48:04.170 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.165:6825/74545", 2024-04-20T01:48:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.165:6827/74545", 2024-04-20T01:48:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.165:6831/74545", 2024-04-20T01:48:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.165:6829/74545", 2024-04-20T01:48:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 4, 2024-04-20T01:48:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "8764d799-af67-4e6c-889f-9fdd91201cc2", 2024-04-20T01:48:04.171 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6809", 2024-04-20T01:48:04.172 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:48:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6811", 2024-04-20T01:48:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:48:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6813", 2024-04-20T01:48:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:48:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6815", 2024-04-20T01:48:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:48:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6821", 2024-04-20T01:48:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:48:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.174 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6823", 2024-04-20T01:48:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:48:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6817", 2024-04-20T01:48:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:48:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.175 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6819", 2024-04-20T01:48:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:48:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6811/43517", 2024-04-20T01:48:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6815/43517", 2024-04-20T01:48:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6823/43517", 2024-04-20T01:48:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6819/43517", 2024-04-20T01:48:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 5, 2024-04-20T01:48:04.176 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "2556b2fa-4dcc-479c-a379-6f9fe96e9c77", 2024-04-20T01:48:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.177 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6800", 2024-04-20T01:48:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:48:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6801", 2024-04-20T01:48:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:48:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6802", 2024-04-20T01:48:04.178 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:48:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6803", 2024-04-20T01:48:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:48:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6806", 2024-04-20T01:48:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:48:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6807", 2024-04-20T01:48:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:48:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6804", 2024-04-20T01:48:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:48:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.180 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6805", 2024-04-20T01:48:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:48:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.121:6801/74354", 2024-04-20T01:48:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.121:6803/74354", 2024-04-20T01:48:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.121:6807/74354", 2024-04-20T01:48:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.121:6805/74354", 2024-04-20T01:48:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.181 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 6, 2024-04-20T01:48:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "baeaae3a-fff4-4ccc-a08a-ff32db4531ba", 2024-04-20T01:48:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.182 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.183 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6800", 2024-04-20T01:48:04.183 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:48:04.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.183 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.183 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6801", 2024-04-20T01:48:04.183 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:48:04.183 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.183 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.183 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:04.183 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.183 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.183 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6802", 2024-04-20T01:48:04.184 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:48:04.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.184 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.184 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6803", 2024-04-20T01:48:04.184 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:48:04.184 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.184 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.184 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:04.184 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.184 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.184 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6806", 2024-04-20T01:48:04.184 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:48:04.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.185 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.185 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6807", 2024-04-20T01:48:04.185 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:48:04.185 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.185 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.185 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:04.185 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.185 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.185 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6804", 2024-04-20T01:48:04.185 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:48:04.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.186 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.186 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6805", 2024-04-20T01:48:04.186 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:48:04.186 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.186 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.186 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.123:6801/43107", 2024-04-20T01:48:04.186 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.123:6803/43107", 2024-04-20T01:48:04.186 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.123:6807/43107", 2024-04-20T01:48:04.186 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.123:6805/43107", 2024-04-20T01:48:04.186 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:04.186 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:04.186 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:04.186 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.187 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 7, 2024-04-20T01:48:04.187 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "0019f005-3e62-4d9d-b334-f1d347a484ce", 2024-04-20T01:48:04.187 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:04.187 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:04.187 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:04.187 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:04.187 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:04.187 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:04.187 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:04.187 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:04.187 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:04.187 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:04.187 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:04.188 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.188 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.188 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6800", 2024-04-20T01:48:04.188 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:48:04.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.188 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.188 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6801", 2024-04-20T01:48:04.188 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:48:04.188 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.188 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.188 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:04.188 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.189 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.189 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6802", 2024-04-20T01:48:04.189 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:48:04.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.189 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.189 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6803", 2024-04-20T01:48:04.189 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:48:04.189 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.189 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.189 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:04.189 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.190 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.190 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6806", 2024-04-20T01:48:04.190 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:48:04.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.190 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.190 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6807", 2024-04-20T01:48:04.190 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:48:04.190 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.190 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.190 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:04.190 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.190 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.191 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6804", 2024-04-20T01:48:04.191 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:48:04.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.191 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.191 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6805", 2024-04-20T01:48:04.191 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:48:04.191 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.191 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.191 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.165:6801/74543", 2024-04-20T01:48:04.191 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.165:6803/74543", 2024-04-20T01:48:04.191 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.165:6807/74543", 2024-04-20T01:48:04.191 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.165:6805/74543", 2024-04-20T01:48:04.191 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:04.191 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:04.192 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:04.192 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.192 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 8, 2024-04-20T01:48:04.192 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "6f5200a3-8980-4241-a6fc-ef604fa612b2", 2024-04-20T01:48:04.192 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:04.192 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:04.192 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:04.192 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:04.192 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:04.192 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:04.192 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:04.192 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 57, 2024-04-20T01:48:04.192 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:04.193 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:04.193 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:04.193 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.193 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.193 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6808", 2024-04-20T01:48:04.193 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:48:04.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.193 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.193 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6810", 2024-04-20T01:48:04.193 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:48:04.193 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.193 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.193 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:04.194 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.194 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.194 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6812", 2024-04-20T01:48:04.194 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:48:04.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.194 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.194 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6814", 2024-04-20T01:48:04.194 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:48:04.194 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.194 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.194 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:04.194 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.195 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.195 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6820", 2024-04-20T01:48:04.195 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:48:04.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.195 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.195 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6822", 2024-04-20T01:48:04.195 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:48:04.195 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.195 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.195 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:04.195 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.196 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.196 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6816", 2024-04-20T01:48:04.196 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:48:04.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.196 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.196 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6818", 2024-04-20T01:48:04.196 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:48:04.196 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.196 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.196 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6810/43515", 2024-04-20T01:48:04.196 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6814/43515", 2024-04-20T01:48:04.196 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6822/43515", 2024-04-20T01:48:04.196 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6818/43515", 2024-04-20T01:48:04.196 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:04.197 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:04.197 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:04.197 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.197 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 9, 2024-04-20T01:48:04.197 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "29cf9aaa-3a99-4605-92f4-64f1f6a1728d", 2024-04-20T01:48:04.197 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:04.197 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:04.197 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:04.197 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:04.197 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:04.197 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:04.197 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:04.197 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 0, 2024-04-20T01:48:04.198 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:04.198 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:04.198 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:04.198 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.198 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.198 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6824", 2024-04-20T01:48:04.198 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:48:04.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.198 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.198 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6825", 2024-04-20T01:48:04.198 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:48:04.198 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.198 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.199 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:04.199 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.199 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.199 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6826", 2024-04-20T01:48:04.199 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:48:04.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.199 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.199 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6827", 2024-04-20T01:48:04.199 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:48:04.199 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.199 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.200 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:04.200 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.200 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.200 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6830", 2024-04-20T01:48:04.200 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:48:04.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.200 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.200 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6831", 2024-04-20T01:48:04.200 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:48:04.200 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.200 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.200 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:04.200 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.201 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.201 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6828", 2024-04-20T01:48:04.201 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:48:04.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.201 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.201 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6829", 2024-04-20T01:48:04.201 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:48:04.201 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.201 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.201 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.121:6825/74379", 2024-04-20T01:48:04.201 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.121:6827/74379", 2024-04-20T01:48:04.201 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.121:6831/74379", 2024-04-20T01:48:04.202 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.121:6829/74379", 2024-04-20T01:48:04.202 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:04.202 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:04.202 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:04.202 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.202 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 10, 2024-04-20T01:48:04.202 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "705906c4-1514-4754-ac40-45226a837180", 2024-04-20T01:48:04.202 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:04.202 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:04.202 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:04.202 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:04.202 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:04.202 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:04.203 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:04.203 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:04.203 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:04.203 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:04.203 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:04.203 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.203 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.203 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6808", 2024-04-20T01:48:04.203 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:48:04.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.203 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.203 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6810", 2024-04-20T01:48:04.203 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:48:04.203 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.204 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.204 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:04.204 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.204 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.204 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6812", 2024-04-20T01:48:04.204 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:48:04.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.204 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.204 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6813", 2024-04-20T01:48:04.204 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:48:04.204 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.204 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.205 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:04.205 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.205 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.205 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6818", 2024-04-20T01:48:04.205 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:48:04.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.205 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.205 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6820", 2024-04-20T01:48:04.205 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:48:04.205 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.205 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.206 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:04.206 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.206 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.206 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6815", 2024-04-20T01:48:04.206 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:48:04.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.206 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.206 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6817", 2024-04-20T01:48:04.206 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:48:04.206 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.206 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.206 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.123:6810/43109", 2024-04-20T01:48:04.206 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.123:6813/43109", 2024-04-20T01:48:04.207 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.123:6820/43109", 2024-04-20T01:48:04.207 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.123:6817/43109", 2024-04-20T01:48:04.207 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:04.207 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:04.207 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:04.207 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.207 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 11, 2024-04-20T01:48:04.207 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "b45f2645-72d1-4d72-89e9-2212f5aa074c", 2024-04-20T01:48:04.207 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:04.207 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:04.207 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:04.207 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:04.207 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:04.208 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:04.208 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:04.208 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:04.208 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:04.208 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:04.208 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:04.208 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.208 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.208 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6808", 2024-04-20T01:48:04.208 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:48:04.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.208 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.208 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6809", 2024-04-20T01:48:04.209 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:48:04.209 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.209 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.209 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:04.209 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.209 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.209 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6810", 2024-04-20T01:48:04.209 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:48:04.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.209 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.209 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6811", 2024-04-20T01:48:04.209 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:48:04.209 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.210 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.210 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:04.210 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.210 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.210 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6814", 2024-04-20T01:48:04.210 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:48:04.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.210 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.210 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6815", 2024-04-20T01:48:04.210 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:48:04.210 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.210 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.211 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:04.211 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.211 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.211 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6812", 2024-04-20T01:48:04.211 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:48:04.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.211 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.211 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6813", 2024-04-20T01:48:04.211 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:48:04.211 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.211 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.212 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.165:6809/74544", 2024-04-20T01:48:04.212 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.165:6811/74544", 2024-04-20T01:48:04.212 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.165:6815/74544", 2024-04-20T01:48:04.212 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.165:6813/74544", 2024-04-20T01:48:04.212 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:04.212 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:04.212 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:04.212 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.212 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 12, 2024-04-20T01:48:04.212 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "3ea410b9-a368-467b-a179-2e3aba562d63", 2024-04-20T01:48:04.212 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:04.212 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:04.212 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:04.212 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:04.213 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:04.213 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:04.213 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:04.213 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 0, 2024-04-20T01:48:04.213 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:04.213 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:04.213 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:04.213 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.213 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.213 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6800", 2024-04-20T01:48:04.213 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:48:04.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.213 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.214 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6801", 2024-04-20T01:48:04.214 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:48:04.214 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.214 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.214 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:04.214 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.214 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.214 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6802", 2024-04-20T01:48:04.214 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:48:04.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.214 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.214 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6803", 2024-04-20T01:48:04.215 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:48:04.215 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.215 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.215 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:04.215 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.215 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.215 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6806", 2024-04-20T01:48:04.215 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:48:04.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.215 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.215 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6807", 2024-04-20T01:48:04.215 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:48:04.216 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.216 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.216 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:04.216 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.216 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.216 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6804", 2024-04-20T01:48:04.216 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:48:04.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.216 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.216 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6805", 2024-04-20T01:48:04.216 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:48:04.216 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.217 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.217 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6801/43516", 2024-04-20T01:48:04.217 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6803/43516", 2024-04-20T01:48:04.217 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6807/43516", 2024-04-20T01:48:04.217 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6805/43516", 2024-04-20T01:48:04.217 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:04.217 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:04.217 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:04.217 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.217 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 13, 2024-04-20T01:48:04.217 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "15a5154b-e7cb-4b57-8468-dcb048d93f63", 2024-04-20T01:48:04.217 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:04.217 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:04.218 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:04.218 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:04.218 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:04.218 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:04.218 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 22, 2024-04-20T01:48:04.218 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:04.218 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:04.218 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:04.218 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:04.218 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.218 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.218 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6808", 2024-04-20T01:48:04.218 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:48:04.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.219 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.219 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6809", 2024-04-20T01:48:04.219 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:48:04.219 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.219 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.219 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:04.219 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.219 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.219 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6810", 2024-04-20T01:48:04.219 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:48:04.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.220 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.220 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6811", 2024-04-20T01:48:04.220 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:48:04.220 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.220 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.220 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:04.220 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.220 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.220 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6814", 2024-04-20T01:48:04.220 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:48:04.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.221 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.221 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6815", 2024-04-20T01:48:04.221 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:48:04.221 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.221 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.221 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:04.221 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.221 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.221 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6812", 2024-04-20T01:48:04.221 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:48:04.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.221 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.221 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6813", 2024-04-20T01:48:04.222 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:48:04.222 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.222 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.222 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.121:6809/74355", 2024-04-20T01:48:04.222 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.121:6811/74355", 2024-04-20T01:48:04.222 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.121:6815/74355", 2024-04-20T01:48:04.222 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.121:6813/74355", 2024-04-20T01:48:04.222 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:04.222 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:04.222 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:04.222 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.223 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 14, 2024-04-20T01:48:04.223 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "23b53da2-f848-4e76-9632-4d9effd5b430", 2024-04-20T01:48:04.223 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:04.223 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:04.223 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:04.223 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:04.223 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:04.223 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:04.223 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:04.223 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:04.223 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:04.223 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:04.223 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:04.223 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.223 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.224 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6824", 2024-04-20T01:48:04.224 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:48:04.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.224 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.224 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6825", 2024-04-20T01:48:04.224 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:48:04.224 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.224 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.224 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:04.224 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.224 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.224 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6826", 2024-04-20T01:48:04.225 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:48:04.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.225 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.225 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6827", 2024-04-20T01:48:04.225 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:48:04.225 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.225 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.225 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:04.225 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.225 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.225 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6830", 2024-04-20T01:48:04.225 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:48:04.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.226 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.226 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6831", 2024-04-20T01:48:04.226 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:48:04.226 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.226 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.226 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:04.226 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.226 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.226 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6828", 2024-04-20T01:48:04.226 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:48:04.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.227 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.227 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6829", 2024-04-20T01:48:04.227 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:48:04.227 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.227 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.227 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.123:6825/43110", 2024-04-20T01:48:04.227 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.123:6827/43110", 2024-04-20T01:48:04.227 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.123:6831/43110", 2024-04-20T01:48:04.227 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.123:6829/43110", 2024-04-20T01:48:04.227 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:04.227 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:04.227 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:04.227 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.228 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.228 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 15, 2024-04-20T01:48:04.228 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "546efc0f-82a3-47d2-aeec-2d849eb1faa1", 2024-04-20T01:48:04.228 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:04.228 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:04.228 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:04.228 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:04.228 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:04.228 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:04.228 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:04.228 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 0, 2024-04-20T01:48:04.228 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:04.228 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:04.228 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:04.228 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.229 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.229 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6816", 2024-04-20T01:48:04.229 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:48:04.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.229 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.229 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6817", 2024-04-20T01:48:04.229 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:48:04.229 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.229 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.229 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:04.229 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.229 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.230 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6818", 2024-04-20T01:48:04.230 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:48:04.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.230 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.230 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6819", 2024-04-20T01:48:04.230 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:48:04.230 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.230 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.230 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:04.230 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.230 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.230 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6822", 2024-04-20T01:48:04.231 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:48:04.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.231 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.231 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6823", 2024-04-20T01:48:04.231 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:48:04.231 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.231 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.231 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:04.231 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:04.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.231 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:04.231 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6820", 2024-04-20T01:48:04.231 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:48:04.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.232 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:04.232 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6821", 2024-04-20T01:48:04.232 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:48:04.232 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.232 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.232 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.165:6817/74562", 2024-04-20T01:48:04.232 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.165:6819/74562", 2024-04-20T01:48:04.232 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.165:6823/74562", 2024-04-20T01:48:04.232 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.165:6821/74562", 2024-04-20T01:48:04.232 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:04.232 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:04.232 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:04.232 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.233 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.233 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:48:04.233 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_xinfo": [ 2024-04-20T01:48:04.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.233 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 0, 2024-04-20T01:48:04.233 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:04.233 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:04.233 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:04.233 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:04.233 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:04.233 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.124928+0000", 2024-04-20T01:48:04.233 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:04.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.233 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 1, 2024-04-20T01:48:04.234 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:04.234 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:04.234 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:04.234 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:04.234 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:04.234 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.093711+0000", 2024-04-20T01:48:04.234 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:04.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.234 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 2, 2024-04-20T01:48:04.234 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:04.234 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:04.234 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:04.234 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:04.234 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:04.235 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.008326+0000", 2024-04-20T01:48:04.235 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:04.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.235 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 3, 2024-04-20T01:48:04.235 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:04.235 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:04.235 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:04.235 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:04.235 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:04.235 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.085397+0000", 2024-04-20T01:48:04.235 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:04.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.235 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 4, 2024-04-20T01:48:04.235 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:04.236 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:04.236 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:04.236 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:04.236 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:04.236 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.117767+0000", 2024-04-20T01:48:04.236 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:04.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.236 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.236 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 5, 2024-04-20T01:48:04.236 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:04.236 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:04.236 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:04.236 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:04.236 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:04.236 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:12.999445+0000", 2024-04-20T01:48:04.237 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:04.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.237 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 6, 2024-04-20T01:48:04.237 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:04.237 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:04.237 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:04.237 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:04.237 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:04.237 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.032367+0000", 2024-04-20T01:48:04.237 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:04.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.237 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 7, 2024-04-20T01:48:04.237 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:04.238 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:04.238 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:04.238 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:04.238 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:04.238 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.058776+0000", 2024-04-20T01:48:04.238 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:04.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.238 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 8, 2024-04-20T01:48:04.238 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:04.238 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:04.238 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:04.238 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:04.238 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:04.238 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.113013+0000", 2024-04-20T01:48:04.239 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:04.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.239 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 9, 2024-04-20T01:48:04.239 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:04.239 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:04.239 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:04.239 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:04.239 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:04.239 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.110714+0000", 2024-04-20T01:48:04.239 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:04.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.239 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 10, 2024-04-20T01:48:04.239 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:04.239 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:04.240 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:04.240 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:04.240 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:04.240 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.092533+0000", 2024-04-20T01:48:04.240 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:04.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.240 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 11, 2024-04-20T01:48:04.240 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:04.240 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:04.240 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:04.240 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:04.240 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:04.240 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.083135+0000", 2024-04-20T01:48:04.240 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:04.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.241 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 12, 2024-04-20T01:48:04.241 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:04.241 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:04.241 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:04.241 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:04.241 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:04.241 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.094847+0000", 2024-04-20T01:48:04.241 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:04.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.241 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 13, 2024-04-20T01:48:04.241 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:04.241 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:04.242 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:04.242 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:04.242 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:04.242 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.059856+0000", 2024-04-20T01:48:04.242 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:04.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.242 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 14, 2024-04-20T01:48:04.242 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:04.242 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:04.242 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:04.242 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:04.242 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:04.242 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.128995+0000", 2024-04-20T01:48:04.242 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:04.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.243 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 15, 2024-04-20T01:48:04.243 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:04.243 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:04.243 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:04.243 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:04.243 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:04.243 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.061883+0000", 2024-04-20T01:48:04.243 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:04.243 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.243 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:48:04.243 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap": [], 2024-04-20T01:48:04.243 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap_items": [], 2024-04-20T01:48:04.243 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap_primaries": [], 2024-04-20T01:48:04.243 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_temp": [], 2024-04-20T01:48:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_temp": [], 2024-04-20T01:48:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "blocklist": {}, 2024-04-20T01:48:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "range_blocklist": {}, 2024-04-20T01:48:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profiles": { 2024-04-20T01:48:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "default": { 2024-04-20T01:48:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-failure-domain": "osd", 2024-04-20T01:48:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "k": "2", 2024-04-20T01:48:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "m": "1", 2024-04-20T01:48:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "plugin": "jerasure", 2024-04-20T01:48:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "technique": "reed_sol_van" 2024-04-20T01:48:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "shecprofile": { 2024-04-20T01:48:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "c": "2", 2024-04-20T01:48:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-device-class": "", 2024-04-20T01:48:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-failure-domain": "osd", 2024-04-20T01:48:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-root": "default", 2024-04-20T01:48:04.245 INFO:teuthology.orchestra.run.smithi044.stdout: "k": "4", 2024-04-20T01:48:04.245 INFO:teuthology.orchestra.run.smithi044.stdout: "m": "3", 2024-04-20T01:48:04.245 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "shecprofile", 2024-04-20T01:48:04.245 INFO:teuthology.orchestra.run.smithi044.stdout: "plugin": "shec", 2024-04-20T01:48:04.245 INFO:teuthology.orchestra.run.smithi044.stdout: "technique": "multiple" 2024-04-20T01:48:04.245 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.245 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps_queue": [ 2024-04-20T01:48:04.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.245 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 3, 2024-04-20T01:48:04.245 INFO:teuthology.orchestra.run.smithi044.stdout: "snaps": [ 2024-04-20T01:48:04.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.245 INFO:teuthology.orchestra.run.smithi044.stdout: "begin": 20, 2024-04-20T01:48:04.245 INFO:teuthology.orchestra.run.smithi044.stdout: "length": 1 2024-04-20T01:48:04.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:04.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.246 INFO:teuthology.orchestra.run.smithi044.stdout: "begin": 24, 2024-04-20T01:48:04.246 INFO:teuthology.orchestra.run.smithi044.stdout: "length": 2 2024-04-20T01:48:04.246 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.246 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.246 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.246 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:48:04.246 INFO:teuthology.orchestra.run.smithi044.stdout: "new_removed_snaps": [ 2024-04-20T01:48:04.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.246 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 3, 2024-04-20T01:48:04.246 INFO:teuthology.orchestra.run.smithi044.stdout: "snaps": [ 2024-04-20T01:48:04.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:04.246 INFO:teuthology.orchestra.run.smithi044.stdout: "begin": 25, 2024-04-20T01:48:04.246 INFO:teuthology.orchestra.run.smithi044.stdout: "length": 1 2024-04-20T01:48:04.246 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.247 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:04.247 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.247 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:48:04.247 INFO:teuthology.orchestra.run.smithi044.stdout: "new_purged_snaps": [], 2024-04-20T01:48:04.247 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_node_flags": {}, 2024-04-20T01:48:04.247 INFO:teuthology.orchestra.run.smithi044.stdout: "device_class_flags": {}, 2024-04-20T01:48:04.247 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": { 2024-04-20T01:48:04.247 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode_enabled": false, 2024-04-20T01:48:04.247 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_bucket_count": 0, 2024-04-20T01:48:04.247 INFO:teuthology.orchestra.run.smithi044.stdout: "degraded_stretch_mode": 0, 2024-04-20T01:48:04.247 INFO:teuthology.orchestra.run.smithi044.stdout: "recovering_stretch_mode": 0, 2024-04-20T01:48:04.247 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode_bucket": 0 2024-04-20T01:48:04.247 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:04.247 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:04.249 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:04.250 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:04.251 INFO:tasks.thrashosds.thrasher:j is {'epoch': 62, 'fsid': '9c2f64c1-aa35-4337-9f5c-081b9e3a2938', 'created': '2024-04-20T01:47:02.934241+0000', 'modified': '2024-04-20T01:48:03.704451+0000', 'last_up_change': '2024-04-20T01:47:15.986754+0000', 'last_in_change': '2024-04-20T01:47:07.894719+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T01:47:15.195181+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 15.380000114440918, 'score_stable': 15.380000114440918, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-04-20T01:47:18.448004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 3.990000009536743, 'score_stable': 3.990000009536743, 'optimal_score': 0.6899999976158142, 'raw_score_acting': 2.75, 'raw_score_stable': 2.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-04-20T01:47:30.734962+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 7, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 30, 'snap_epoch': 62, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'shecprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '3903b88f-ff2b-4252-9480-92b701b1e59b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6824', 'nonce': 43518}, {'type': 'v1', 'addr': '172.21.15.44:6825', 'nonce': 43518}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6826', 'nonce': 43518}, {'type': 'v1', 'addr': '172.21.15.44:6827', 'nonce': 43518}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6830', 'nonce': 43518}, {'type': 'v1', 'addr': '172.21.15.44:6831', 'nonce': 43518}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6828', 'nonce': 43518}, {'type': 'v1', 'addr': '172.21.15.44:6829', 'nonce': 43518}]}, 'public_addr': '172.21.15.44:6825/43518', 'cluster_addr': '172.21.15.44:6827/43518', 'heartbeat_back_addr': '172.21.15.44:6831/43518', 'heartbeat_front_addr': '172.21.15.44:6829/43518', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c5ff0175-b4f0-40a5-86c7-7a0a351bf070', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6816', 'nonce': 74361}, {'type': 'v1', 'addr': '172.21.15.121:6817', 'nonce': 74361}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6818', 'nonce': 74361}, {'type': 'v1', 'addr': '172.21.15.121:6819', 'nonce': 74361}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6822', 'nonce': 74361}, {'type': 'v1', 'addr': '172.21.15.121:6823', 'nonce': 74361}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6820', 'nonce': 74361}, {'type': 'v1', 'addr': '172.21.15.121:6821', 'nonce': 74361}]}, 'public_addr': '172.21.15.121:6817/74361', 'cluster_addr': '172.21.15.121:6819/74361', 'heartbeat_back_addr': '172.21.15.121:6823/74361', 'heartbeat_front_addr': '172.21.15.121:6821/74361', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '7af325c1-36b5-420c-a2a9-44c00a002977', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6809', 'nonce': 43108}, {'type': 'v1', 'addr': '172.21.15.123:6811', 'nonce': 43108}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6814', 'nonce': 43108}, {'type': 'v1', 'addr': '172.21.15.123:6816', 'nonce': 43108}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6822', 'nonce': 43108}, {'type': 'v1', 'addr': '172.21.15.123:6823', 'nonce': 43108}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6819', 'nonce': 43108}, {'type': 'v1', 'addr': '172.21.15.123:6821', 'nonce': 43108}]}, 'public_addr': '172.21.15.123:6811/43108', 'cluster_addr': '172.21.15.123:6816/43108', 'heartbeat_back_addr': '172.21.15.123:6823/43108', 'heartbeat_front_addr': '172.21.15.123:6821/43108', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '99334f43-83ea-4c3f-b91d-ae72c8361b8c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6824', 'nonce': 74545}, {'type': 'v1', 'addr': '172.21.15.165:6825', 'nonce': 74545}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6826', 'nonce': 74545}, {'type': 'v1', 'addr': '172.21.15.165:6827', 'nonce': 74545}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6830', 'nonce': 74545}, {'type': 'v1', 'addr': '172.21.15.165:6831', 'nonce': 74545}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6828', 'nonce': 74545}, {'type': 'v1', 'addr': '172.21.15.165:6829', 'nonce': 74545}]}, 'public_addr': '172.21.15.165:6825/74545', 'cluster_addr': '172.21.15.165:6827/74545', 'heartbeat_back_addr': '172.21.15.165:6831/74545', 'heartbeat_front_addr': '172.21.15.165:6829/74545', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8764d799-af67-4e6c-889f-9fdd91201cc2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6809', 'nonce': 43517}, {'type': 'v1', 'addr': '172.21.15.44:6811', 'nonce': 43517}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6813', 'nonce': 43517}, {'type': 'v1', 'addr': '172.21.15.44:6815', 'nonce': 43517}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6821', 'nonce': 43517}, {'type': 'v1', 'addr': '172.21.15.44:6823', 'nonce': 43517}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6817', 'nonce': 43517}, {'type': 'v1', 'addr': '172.21.15.44:6819', 'nonce': 43517}]}, 'public_addr': '172.21.15.44:6811/43517', 'cluster_addr': '172.21.15.44:6815/43517', 'heartbeat_back_addr': '172.21.15.44:6823/43517', 'heartbeat_front_addr': '172.21.15.44:6819/43517', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '2556b2fa-4dcc-479c-a379-6f9fe96e9c77', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6800', 'nonce': 74354}, {'type': 'v1', 'addr': '172.21.15.121:6801', 'nonce': 74354}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6802', 'nonce': 74354}, {'type': 'v1', 'addr': '172.21.15.121:6803', 'nonce': 74354}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6806', 'nonce': 74354}, {'type': 'v1', 'addr': '172.21.15.121:6807', 'nonce': 74354}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6804', 'nonce': 74354}, {'type': 'v1', 'addr': '172.21.15.121:6805', 'nonce': 74354}]}, 'public_addr': '172.21.15.121:6801/74354', 'cluster_addr': '172.21.15.121:6803/74354', 'heartbeat_back_addr': '172.21.15.121:6807/74354', 'heartbeat_front_addr': '172.21.15.121:6805/74354', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'baeaae3a-fff4-4ccc-a08a-ff32db4531ba', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6800', 'nonce': 43107}, {'type': 'v1', 'addr': '172.21.15.123:6801', 'nonce': 43107}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6802', 'nonce': 43107}, {'type': 'v1', 'addr': '172.21.15.123:6803', 'nonce': 43107}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6806', 'nonce': 43107}, {'type': 'v1', 'addr': '172.21.15.123:6807', 'nonce': 43107}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6804', 'nonce': 43107}, {'type': 'v1', 'addr': '172.21.15.123:6805', 'nonce': 43107}]}, 'public_addr': '172.21.15.123:6801/43107', 'cluster_addr': '172.21.15.123:6803/43107', 'heartbeat_back_addr': '172.21.15.123:6807/43107', 'heartbeat_front_addr': '172.21.15.123:6805/43107', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '0019f005-3e62-4d9d-b334-f1d347a484ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6800', 'nonce': 74543}, {'type': 'v1', 'addr': '172.21.15.165:6801', 'nonce': 74543}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6802', 'nonce': 74543}, {'type': 'v1', 'addr': '172.21.15.165:6803', 'nonce': 74543}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6806', 'nonce': 74543}, {'type': 'v1', 'addr': '172.21.15.165:6807', 'nonce': 74543}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6804', 'nonce': 74543}, {'type': 'v1', 'addr': '172.21.15.165:6805', 'nonce': 74543}]}, 'public_addr': '172.21.15.165:6801/74543', 'cluster_addr': '172.21.15.165:6803/74543', 'heartbeat_back_addr': '172.21.15.165:6807/74543', 'heartbeat_front_addr': '172.21.15.165:6805/74543', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '6f5200a3-8980-4241-a6fc-ef604fa612b2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6808', 'nonce': 43515}, {'type': 'v1', 'addr': '172.21.15.44:6810', 'nonce': 43515}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6812', 'nonce': 43515}, {'type': 'v1', 'addr': '172.21.15.44:6814', 'nonce': 43515}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6820', 'nonce': 43515}, {'type': 'v1', 'addr': '172.21.15.44:6822', 'nonce': 43515}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6816', 'nonce': 43515}, {'type': 'v1', 'addr': '172.21.15.44:6818', 'nonce': 43515}]}, 'public_addr': '172.21.15.44:6810/43515', 'cluster_addr': '172.21.15.44:6814/43515', 'heartbeat_back_addr': '172.21.15.44:6822/43515', 'heartbeat_front_addr': '172.21.15.44:6818/43515', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '29cf9aaa-3a99-4605-92f4-64f1f6a1728d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6824', 'nonce': 74379}, {'type': 'v1', 'addr': '172.21.15.121:6825', 'nonce': 74379}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6826', 'nonce': 74379}, {'type': 'v1', 'addr': '172.21.15.121:6827', 'nonce': 74379}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6830', 'nonce': 74379}, {'type': 'v1', 'addr': '172.21.15.121:6831', 'nonce': 74379}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6828', 'nonce': 74379}, {'type': 'v1', 'addr': '172.21.15.121:6829', 'nonce': 74379}]}, 'public_addr': '172.21.15.121:6825/74379', 'cluster_addr': '172.21.15.121:6827/74379', 'heartbeat_back_addr': '172.21.15.121:6831/74379', 'heartbeat_front_addr': '172.21.15.121:6829/74379', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '705906c4-1514-4754-ac40-45226a837180', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6808', 'nonce': 43109}, {'type': 'v1', 'addr': '172.21.15.123:6810', 'nonce': 43109}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6812', 'nonce': 43109}, {'type': 'v1', 'addr': '172.21.15.123:6813', 'nonce': 43109}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6818', 'nonce': 43109}, {'type': 'v1', 'addr': '172.21.15.123:6820', 'nonce': 43109}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6815', 'nonce': 43109}, {'type': 'v1', 'addr': '172.21.15.123:6817', 'nonce': 43109}]}, 'public_addr': '172.21.15.123:6810/43109', 'cluster_addr': '172.21.15.123:6813/43109', 'heartbeat_back_addr': '172.21.15.123:6820/43109', 'heartbeat_front_addr': '172.21.15.123:6817/43109', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': 'b45f2645-72d1-4d72-89e9-2212f5aa074c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6808', 'nonce': 74544}, {'type': 'v1', 'addr': '172.21.15.165:6809', 'nonce': 74544}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6810', 'nonce': 74544}, {'type': 'v1', 'addr': '172.21.15.165:6811', 'nonce': 74544}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6814', 'nonce': 74544}, {'type': 'v1', 'addr': '172.21.15.165:6815', 'nonce': 74544}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6812', 'nonce': 74544}, {'type': 'v1', 'addr': '172.21.15.165:6813', 'nonce': 74544}]}, 'public_addr': '172.21.15.165:6809/74544', 'cluster_addr': '172.21.15.165:6811/74544', 'heartbeat_back_addr': '172.21.15.165:6815/74544', 'heartbeat_front_addr': '172.21.15.165:6813/74544', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': '3ea410b9-a368-467b-a179-2e3aba562d63', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6800', 'nonce': 43516}, {'type': 'v1', 'addr': '172.21.15.44:6801', 'nonce': 43516}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6802', 'nonce': 43516}, {'type': 'v1', 'addr': '172.21.15.44:6803', 'nonce': 43516}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6806', 'nonce': 43516}, {'type': 'v1', 'addr': '172.21.15.44:6807', 'nonce': 43516}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6804', 'nonce': 43516}, {'type': 'v1', 'addr': '172.21.15.44:6805', 'nonce': 43516}]}, 'public_addr': '172.21.15.44:6801/43516', 'cluster_addr': '172.21.15.44:6803/43516', 'heartbeat_back_addr': '172.21.15.44:6807/43516', 'heartbeat_front_addr': '172.21.15.44:6805/43516', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': '15a5154b-e7cb-4b57-8468-dcb048d93f63', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6808', 'nonce': 74355}, {'type': 'v1', 'addr': '172.21.15.121:6809', 'nonce': 74355}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6810', 'nonce': 74355}, {'type': 'v1', 'addr': '172.21.15.121:6811', 'nonce': 74355}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6814', 'nonce': 74355}, {'type': 'v1', 'addr': '172.21.15.121:6815', 'nonce': 74355}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6812', 'nonce': 74355}, {'type': 'v1', 'addr': '172.21.15.121:6813', 'nonce': 74355}]}, 'public_addr': '172.21.15.121:6809/74355', 'cluster_addr': '172.21.15.121:6811/74355', 'heartbeat_back_addr': '172.21.15.121:6815/74355', 'heartbeat_front_addr': '172.21.15.121:6813/74355', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '23b53da2-f848-4e76-9632-4d9effd5b430', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6824', 'nonce': 43110}, {'type': 'v1', 'addr': '172.21.15.123:6825', 'nonce': 43110}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6826', 'nonce': 43110}, {'type': 'v1', 'addr': '172.21.15.123:6827', 'nonce': 43110}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6830', 'nonce': 43110}, {'type': 'v1', 'addr': '172.21.15.123:6831', 'nonce': 43110}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6828', 'nonce': 43110}, {'type': 'v1', 'addr': '172.21.15.123:6829', 'nonce': 43110}]}, 'public_addr': '172.21.15.123:6825/43110', 'cluster_addr': '172.21.15.123:6827/43110', 'heartbeat_back_addr': '172.21.15.123:6831/43110', 'heartbeat_front_addr': '172.21.15.123:6829/43110', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': '546efc0f-82a3-47d2-aeec-2d849eb1faa1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6816', 'nonce': 74562}, {'type': 'v1', 'addr': '172.21.15.165:6817', 'nonce': 74562}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6818', 'nonce': 74562}, {'type': 'v1', 'addr': '172.21.15.165:6819', 'nonce': 74562}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6822', 'nonce': 74562}, {'type': 'v1', 'addr': '172.21.15.165:6823', 'nonce': 74562}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6820', 'nonce': 74562}, {'type': 'v1', 'addr': '172.21.15.165:6821', 'nonce': 74562}]}, 'public_addr': '172.21.15.165:6817/74562', 'cluster_addr': '172.21.15.165:6819/74562', 'heartbeat_back_addr': '172.21.15.165:6823/74562', 'heartbeat_front_addr': '172.21.15.165:6821/74562', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.124928+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.093711+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.008326+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.085397+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.117767+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:12.999445+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.032367+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.058776+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.113013+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.110714+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.092533+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.083135+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.094847+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.059856+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.128995+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.061883+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'shecprofile': {'c': '2', 'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'k': '4', 'm': '3', 'name': 'shecprofile', 'plugin': 'shec', 'technique': 'multiple'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 20, 'length': 1}, {'begin': 24, 'length': 2}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 25, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-20T01:48:04.251 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T01:48:04.251 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:48:04.252 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-20T01:48:04.252 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:04.264+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:04.252 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:48:04.271 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:48:04.293 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:48:04.314 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:48:04.334 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:48:04.350 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:04.363+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:04.355 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:48:04.376 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:48:04.398 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:48:04.419 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:04.419 INFO:teuthology.orchestra.run.smithi121.stdout: "size": 20, 2024-04-20T01:48:04.419 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 600, 2024-04-20T01:48:04.420 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-04-20T01:48:04.420 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.420 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.9s1 47/30 ECSubWrite(tid=13, reqid=osd.7.0:13, at_version=47'8, trim_to=0'0, roll_forward_to=42'6))", 2024-04-20T01:48:04.420 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.708967+0000", 2024-04-20T01:48:04.420 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 15.724157787999999, 2024-04-20T01:48:04.420 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0053408939999999997, 2024-04-20T01:48:04.420 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:04.420 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:04.420 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:04.420 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.420 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:04.420 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.708967+0000", 2024-04-20T01:48:04.420 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.420 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.420 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.421 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:04.421 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.708967+0000", 2024-04-20T01:48:04.421 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.421 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.421 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.421 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:04.421 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.708966+0000", 2024-04-20T01:48:04.421 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:04.421 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.421 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.421 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:04.421 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.708986+0000", 2024-04-20T01:48:04.421 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9799000000000001e-05 2024-04-20T01:48:04.421 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.422 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.422 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:04.422 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.708996+0000", 2024-04-20T01:48:04.422 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.9939999999999995e-06 2024-04-20T01:48:04.422 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.422 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.422 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:04.422 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.709004+0000", 2024-04-20T01:48:04.422 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.2390000000000002e-06 2024-04-20T01:48:04.422 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.422 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.422 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:04.422 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.709244+0000", 2024-04-20T01:48:04.422 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000240113 2024-04-20T01:48:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.709250+0000", 2024-04-20T01:48:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.4739999999999996e-06 2024-04-20T01:48:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.714281+0000", 2024-04-20T01:48:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0050313019999999996 2024-04-20T01:48:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.714308+0000", 2024-04-20T01:48:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6342e-05 2024-04-20T01:48:04.423 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.424 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.424 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.424 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.424 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.424 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.as2 47/30 ECSubWrite(tid=32, reqid=osd.6.0:31, at_version=0'0, trim_to=0'0, roll_forward_to=47'12))", 2024-04-20T01:48:04.424 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.722942+0000", 2024-04-20T01:48:04.424 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 15.710182461, 2024-04-20T01:48:04.424 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0050181330000000001, 2024-04-20T01:48:04.424 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:04.424 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:04.424 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:04.424 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.424 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:04.424 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.722942+0000", 2024-04-20T01:48:04.425 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.425 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.425 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.425 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:04.425 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.722942+0000", 2024-04-20T01:48:04.425 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.425 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.425 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.425 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:04.425 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.722941+0000", 2024-04-20T01:48:04.425 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:04.425 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.425 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.425 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:04.425 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.722952+0000", 2024-04-20T01:48:04.425 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0556e-05 2024-04-20T01:48:04.426 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.426 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.426 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:04.426 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.722965+0000", 2024-04-20T01:48:04.426 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3369e-05 2024-04-20T01:48:04.426 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.426 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.426 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:04.426 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.722974+0000", 2024-04-20T01:48:04.426 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.6740000000000003e-06 2024-04-20T01:48:04.426 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.426 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.426 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:04.426 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.723093+0000", 2024-04-20T01:48:04.427 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00011913000000000001 2024-04-20T01:48:04.427 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.427 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.427 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:04.427 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.723103+0000", 2024-04-20T01:48:04.427 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.8770000000000005e-06 2024-04-20T01:48:04.427 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.427 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.427 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:04.427 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.727939+0000", 2024-04-20T01:48:04.427 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0048359450000000003 2024-04-20T01:48:04.427 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.427 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.427 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:04.427 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.727960+0000", 2024-04-20T01:48:04.427 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1390999999999999e-05 2024-04-20T01:48:04.428 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.428 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.428 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.428 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.428 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.428 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 47/30 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=47'35, trim_to=0'0, roll_forward_to=44'31))", 2024-04-20T01:48:04.428 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.723003+0000", 2024-04-20T01:48:04.428 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 15.710121426000001, 2024-04-20T01:48:04.428 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0049761140000000002, 2024-04-20T01:48:04.428 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:04.428 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:04.428 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:04.428 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.428 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:04.428 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.723003+0000", 2024-04-20T01:48:04.429 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.429 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.429 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.429 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:04.429 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.723003+0000", 2024-04-20T01:48:04.429 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.429 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.429 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.429 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:04.429 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.723003+0000", 2024-04-20T01:48:04.429 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:04.429 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.429 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.429 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:04.429 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.723021+0000", 2024-04-20T01:48:04.430 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8777e-05 2024-04-20T01:48:04.430 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.430 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.430 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:04.430 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.723032+0000", 2024-04-20T01:48:04.430 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0139000000000001e-05 2024-04-20T01:48:04.430 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.430 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.430 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:04.430 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.723038+0000", 2024-04-20T01:48:04.430 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.674e-06 2024-04-20T01:48:04.430 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.430 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.430 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:04.431 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.723269+0000", 2024-04-20T01:48:04.431 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00023070600000000001 2024-04-20T01:48:04.431 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.431 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.431 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:04.431 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.723278+0000", 2024-04-20T01:48:04.431 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.867e-06 2024-04-20T01:48:04.431 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.431 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.431 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:04.431 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.727963+0000", 2024-04-20T01:48:04.431 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0046851009999999997 2024-04-20T01:48:04.431 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.431 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.431 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:04.431 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.727979+0000", 2024-04-20T01:48:04.432 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6243999999999999e-05 2024-04-20T01:48:04.432 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.432 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.432 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.432 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.432 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.432 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 47/30 ECSubWrite(tid=246, reqid=client.4643.0:388, at_version=47'38, trim_to=0'0, roll_forward_to=47'35))", 2024-04-20T01:48:04.432 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.745219+0000", 2024-04-20T01:48:04.432 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 15.687905058, 2024-04-20T01:48:04.432 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0073977050000000001, 2024-04-20T01:48:04.432 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:04.432 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:04.432 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:04.432 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.432 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:04.433 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745219+0000", 2024-04-20T01:48:04.433 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.433 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.433 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.433 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:04.433 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745219+0000", 2024-04-20T01:48:04.433 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.433 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.433 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.433 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:04.433 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745218+0000", 2024-04-20T01:48:04.433 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:04.433 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.433 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.433 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:04.433 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745266+0000", 2024-04-20T01:48:04.434 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.7012999999999999e-05 2024-04-20T01:48:04.434 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.434 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.434 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:04.434 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745277+0000", 2024-04-20T01:48:04.434 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1348999999999999e-05 2024-04-20T01:48:04.434 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.434 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.434 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:04.434 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745284+0000", 2024-04-20T01:48:04.434 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0060000000000003e-06 2024-04-20T01:48:04.434 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.434 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.434 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:04.434 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745340+0000", 2024-04-20T01:48:04.435 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6023000000000002e-05 2024-04-20T01:48:04.435 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.435 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.435 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:04.435 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745361+0000", 2024-04-20T01:48:04.435 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1356999999999999e-05 2024-04-20T01:48:04.435 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.435 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.435 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:04.435 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.752597+0000", 2024-04-20T01:48:04.435 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0072354910000000001 2024-04-20T01:48:04.435 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.435 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.435 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:04.435 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.752617+0000", 2024-04-20T01:48:04.436 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0341000000000001e-05 2024-04-20T01:48:04.436 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.436 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.436 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.436 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.436 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.436 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 47/30 ECSubWrite(tid=247, reqid=client.4643.0:389, at_version=47'39, trim_to=0'0, roll_forward_to=47'35))", 2024-04-20T01:48:04.436 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.746671+0000", 2024-04-20T01:48:04.436 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 15.686453395000001, 2024-04-20T01:48:04.436 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0059586839999999997, 2024-04-20T01:48:04.436 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:04.436 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:04.436 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:04.436 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.436 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:04.436 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.746671+0000", 2024-04-20T01:48:04.437 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.437 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.437 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.437 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:04.437 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.746671+0000", 2024-04-20T01:48:04.437 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.437 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.437 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.437 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:04.437 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.746670+0000", 2024-04-20T01:48:04.437 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:04.437 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.437 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.437 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:04.437 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.746736+0000", 2024-04-20T01:48:04.438 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.5914999999999995e-05 2024-04-20T01:48:04.438 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.438 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.438 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:04.438 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.746749+0000", 2024-04-20T01:48:04.438 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2245999999999999e-05 2024-04-20T01:48:04.438 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.438 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.438 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:04.438 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.746758+0000", 2024-04-20T01:48:04.438 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4590000000000004e-06 2024-04-20T01:48:04.438 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.438 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.438 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:04.438 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.747358+0000", 2024-04-20T01:48:04.439 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00060028299999999996 2024-04-20T01:48:04.439 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.439 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.439 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:04.439 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.747367+0000", 2024-04-20T01:48:04.439 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.2670000000000006e-06 2024-04-20T01:48:04.439 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.439 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.439 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:04.439 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.752621+0000", 2024-04-20T01:48:04.439 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0052539919999999999 2024-04-20T01:48:04.439 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.439 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.439 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:04.439 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.752630+0000", 2024-04-20T01:48:04.440 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.1940000000000004e-06 2024-04-20T01:48:04.440 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.440 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.440 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.440 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.440 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.440 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 47/30 ECSubWrite(tid=248, reqid=client.4643.0:390, at_version=47'40, trim_to=0'0, roll_forward_to=47'35))", 2024-04-20T01:48:04.440 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.747763+0000", 2024-04-20T01:48:04.440 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 15.685361404, 2024-04-20T01:48:04.440 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.004878088, 2024-04-20T01:48:04.440 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:04.440 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:04.440 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:04.440 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.440 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:04.441 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.747763+0000", 2024-04-20T01:48:04.441 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.441 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.441 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.441 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:04.441 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.747763+0000", 2024-04-20T01:48:04.441 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.441 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.441 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.441 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:04.441 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.747762+0000", 2024-04-20T01:48:04.441 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:04.441 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.441 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.441 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:04.442 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.747779+0000", 2024-04-20T01:48:04.442 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7272e-05 2024-04-20T01:48:04.442 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.442 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.442 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:04.442 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.747792+0000", 2024-04-20T01:48:04.442 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2323999999999999e-05 2024-04-20T01:48:04.442 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.442 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.442 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:04.442 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.747801+0000", 2024-04-20T01:48:04.442 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8089999999999999e-06 2024-04-20T01:48:04.442 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.442 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.442 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:04.443 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.747843+0000", 2024-04-20T01:48:04.443 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2876000000000001e-05 2024-04-20T01:48:04.443 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.443 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.443 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:04.443 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.747853+0000", 2024-04-20T01:48:04.443 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4930000000000004e-06 2024-04-20T01:48:04.443 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.443 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.443 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:04.443 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.752632+0000", 2024-04-20T01:48:04.443 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0047787890000000003 2024-04-20T01:48:04.443 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.443 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.443 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:04.444 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.752641+0000", 2024-04-20T01:48:04.444 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3449999999999995e-06 2024-04-20T01:48:04.444 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.444 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.444 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.444 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.444 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.444 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 48/30 ECSubWrite(tid=84, reqid=osd.8.0:84, at_version=48'69, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:04.444 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.709204+0000", 2024-04-20T01:48:04.444 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.723920494, 2024-04-20T01:48:04.444 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0059724779999999998, 2024-04-20T01:48:04.444 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:04.444 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:04.444 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:04.444 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.445 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:04.445 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.709204+0000", 2024-04-20T01:48:04.445 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.445 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.445 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.445 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:04.445 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.709204+0000", 2024-04-20T01:48:04.445 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.445 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.445 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.445 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:04.445 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.709203+0000", 2024-04-20T01:48:04.445 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:04.445 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.445 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.446 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:04.446 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.709222+0000", 2024-04-20T01:48:04.446 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8780999999999999e-05 2024-04-20T01:48:04.446 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.446 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.446 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:04.446 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.709233+0000", 2024-04-20T01:48:04.446 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.045e-05 2024-04-20T01:48:04.446 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.446 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.446 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:04.447 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.709243+0000", 2024-04-20T01:48:04.447 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0111e-05 2024-04-20T01:48:04.447 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.447 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.447 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:04.447 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.709284+0000", 2024-04-20T01:48:04.447 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.1495999999999997e-05 2024-04-20T01:48:04.447 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.447 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.447 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:04.447 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.709294+0000", 2024-04-20T01:48:04.447 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3689999999999996e-06 2024-04-20T01:48:04.447 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.447 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.448 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:04.448 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715152+0000", 2024-04-20T01:48:04.448 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0058586300000000001 2024-04-20T01:48:04.448 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.448 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.448 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:04.448 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715176+0000", 2024-04-20T01:48:04.448 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4139999999999999e-05 2024-04-20T01:48:04.448 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.448 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.448 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.448 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.448 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.448 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 48/30 ECSubWrite(tid=252, reqid=osd.13.0:252, at_version=48'42, trim_to=0'0, roll_forward_to=47'40))", 2024-04-20T01:48:04.448 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.710137+0000", 2024-04-20T01:48:04.448 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.722987209999999, 2024-04-20T01:48:04.449 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0082943410000000002, 2024-04-20T01:48:04.449 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:04.449 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:04.449 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:04.449 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.449 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:04.449 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710137+0000", 2024-04-20T01:48:04.449 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.449 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.449 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.449 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:04.449 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710137+0000", 2024-04-20T01:48:04.449 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.449 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.449 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.450 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:04.450 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710137+0000", 2024-04-20T01:48:04.450 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:04.450 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.450 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.450 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:04.450 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710162+0000", 2024-04-20T01:48:04.450 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4768000000000001e-05 2024-04-20T01:48:04.450 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.450 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.450 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:04.450 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710174+0000", 2024-04-20T01:48:04.450 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2150999999999999e-05 2024-04-20T01:48:04.450 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.450 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.451 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:04.451 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710183+0000", 2024-04-20T01:48:04.451 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8200000000000003e-06 2024-04-20T01:48:04.451 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.451 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.451 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:04.451 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710238+0000", 2024-04-20T01:48:04.451 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.4954999999999998e-05 2024-04-20T01:48:04.451 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.451 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.451 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:04.451 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710248+0000", 2024-04-20T01:48:04.451 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0222e-05 2024-04-20T01:48:04.451 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.451 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.451 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:04.452 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718408+0000", 2024-04-20T01:48:04.452 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0081605570000000006 2024-04-20T01:48:04.452 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.452 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.452 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:04.452 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718432+0000", 2024-04-20T01:48:04.452 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3129000000000002e-05 2024-04-20T01:48:04.452 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.452 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.452 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.452 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.452 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.452 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 48/30 ECSubWrite(tid=253, reqid=osd.13.0:253, at_version=48'44, trim_to=0'0, roll_forward_to=47'40))", 2024-04-20T01:48:04.452 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.711047+0000", 2024-04-20T01:48:04.452 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.722077792, 2024-04-20T01:48:04.453 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0093898609999999993, 2024-04-20T01:48:04.453 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:04.453 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:04.453 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:04.453 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.453 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:04.453 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711047+0000", 2024-04-20T01:48:04.453 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.453 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.453 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.453 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:04.453 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711047+0000", 2024-04-20T01:48:04.453 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.453 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.453 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.454 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:04.454 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711046+0000", 2024-04-20T01:48:04.454 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:04.454 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.454 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.454 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:04.454 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711066+0000", 2024-04-20T01:48:04.454 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0421999999999999e-05 2024-04-20T01:48:04.454 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.454 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.454 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:04.454 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711082+0000", 2024-04-20T01:48:04.454 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5231999999999999e-05 2024-04-20T01:48:04.454 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.454 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.455 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:04.455 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711091+0000", 2024-04-20T01:48:04.455 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.1679999999999993e-06 2024-04-20T01:48:04.455 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.455 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.455 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:04.455 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.712543+0000", 2024-04-20T01:48:04.455 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001452545 2024-04-20T01:48:04.455 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.455 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.455 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:04.455 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.712549+0000", 2024-04-20T01:48:04.455 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.7450000000000001e-06 2024-04-20T01:48:04.455 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.455 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.455 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:04.456 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720419+0000", 2024-04-20T01:48:04.456 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0078696630000000007 2024-04-20T01:48:04.456 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.456 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.456 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:04.456 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720436+0000", 2024-04-20T01:48:04.456 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7869000000000001e-05 2024-04-20T01:48:04.456 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.456 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.456 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.456 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.456 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.456 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 48/30 ECSubWrite(tid=85, reqid=client.4643.0:399, at_version=48'71, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:04.456 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.711873+0000", 2024-04-20T01:48:04.456 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.721250989, 2024-04-20T01:48:04.457 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0060631469999999996, 2024-04-20T01:48:04.457 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:04.457 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:04.457 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:04.457 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.457 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:04.457 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711873+0000", 2024-04-20T01:48:04.457 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.457 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.457 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.457 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:04.457 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711873+0000", 2024-04-20T01:48:04.457 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.457 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.457 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.457 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:04.458 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711873+0000", 2024-04-20T01:48:04.458 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:04.458 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.458 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.458 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:04.458 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711898+0000", 2024-04-20T01:48:04.458 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5327999999999999e-05 2024-04-20T01:48:04.458 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.458 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.458 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:04.458 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711907+0000", 2024-04-20T01:48:04.458 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.7839999999999992e-06 2024-04-20T01:48:04.458 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.458 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.458 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:04.459 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711913+0000", 2024-04-20T01:48:04.459 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.0599999999999996e-06 2024-04-20T01:48:04.459 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.459 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.459 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:04.459 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711972+0000", 2024-04-20T01:48:04.459 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.9212000000000002e-05 2024-04-20T01:48:04.459 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.459 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.459 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:04.459 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711983+0000", 2024-04-20T01:48:04.459 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1126e-05 2024-04-20T01:48:04.459 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.459 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.459 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:04.460 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.717902+0000", 2024-04-20T01:48:04.460 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0059184529999999997 2024-04-20T01:48:04.460 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.460 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.460 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:04.460 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.717937+0000", 2024-04-20T01:48:04.460 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.4909000000000001e-05 2024-04-20T01:48:04.460 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.460 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.460 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.460 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.460 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.460 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 48/30 ECSubWrite(tid=86, reqid=client.4643.0:400, at_version=48'72, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:04.460 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.713378+0000", 2024-04-20T01:48:04.460 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.719746309, 2024-04-20T01:48:04.461 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0070552389999999996, 2024-04-20T01:48:04.461 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:04.461 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:04.461 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:04.461 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.461 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:04.461 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713378+0000", 2024-04-20T01:48:04.461 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.461 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.461 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.461 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:04.461 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713378+0000", 2024-04-20T01:48:04.461 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.461 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.461 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.462 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:04.462 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713378+0000", 2024-04-20T01:48:04.462 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:04.462 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.462 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.462 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:04.462 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713397+0000", 2024-04-20T01:48:04.462 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9165999999999999e-05 2024-04-20T01:48:04.462 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.462 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.462 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:04.462 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713408+0000", 2024-04-20T01:48:04.462 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0854e-05 2024-04-20T01:48:04.462 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.462 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.463 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:04.463 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713414+0000", 2024-04-20T01:48:04.463 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.1850000000000001e-06 2024-04-20T01:48:04.463 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.463 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.463 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:04.463 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713458+0000", 2024-04-20T01:48:04.463 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.4162999999999997e-05 2024-04-20T01:48:04.463 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.463 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.463 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:04.463 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713469+0000", 2024-04-20T01:48:04.463 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0789e-05 2024-04-20T01:48:04.463 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.463 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.464 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:04.464 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720404+0000", 2024-04-20T01:48:04.464 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0069350109999999996 2024-04-20T01:48:04.464 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.464 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.464 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:04.464 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720433+0000", 2024-04-20T01:48:04.464 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9660000000000001e-05 2024-04-20T01:48:04.464 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.464 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.464 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.464 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.464 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.464 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 48/30 ECSubWrite(tid=87, reqid=client.4643.0:401, at_version=48'73, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:04.464 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.713964+0000", 2024-04-20T01:48:04.464 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.719160548, 2024-04-20T01:48:04.465 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0065393960000000003, 2024-04-20T01:48:04.465 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:04.465 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:04.465 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:04.465 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.465 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:04.465 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713964+0000", 2024-04-20T01:48:04.465 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.465 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.465 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.465 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:04.465 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713964+0000", 2024-04-20T01:48:04.465 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.465 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.465 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.466 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:04.466 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713963+0000", 2024-04-20T01:48:04.466 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:04.466 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.466 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.466 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:04.466 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713977+0000", 2024-04-20T01:48:04.466 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3828999999999999e-05 2024-04-20T01:48:04.466 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.466 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.466 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:04.466 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713988+0000", 2024-04-20T01:48:04.466 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0728e-05 2024-04-20T01:48:04.466 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.466 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.467 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:04.467 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713994+0000", 2024-04-20T01:48:04.467 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.2949999999999999e-06 2024-04-20T01:48:04.467 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.467 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.467 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:04.467 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715264+0000", 2024-04-20T01:48:04.467 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00126958 2024-04-20T01:48:04.467 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.467 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.467 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:04.467 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715273+0000", 2024-04-20T01:48:04.467 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2720000000000003e-06 2024-04-20T01:48:04.467 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.467 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.467 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:04.468 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720484+0000", 2024-04-20T01:48:04.468 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0052106820000000003 2024-04-20T01:48:04.468 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.468 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.468 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:04.468 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720503+0000", 2024-04-20T01:48:04.468 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9706000000000001e-05 2024-04-20T01:48:04.468 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.468 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.468 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.468 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.468 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.468 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.9s1 48/30 ECSubWrite(tid=17, reqid=osd.7.0:17, at_version=48'12, trim_to=0'0, roll_forward_to=47'10))", 2024-04-20T01:48:04.468 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.715248+0000", 2024-04-20T01:48:04.468 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.717876267999999, 2024-04-20T01:48:04.469 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.005232039, 2024-04-20T01:48:04.469 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:04.469 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:04.469 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:04.469 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.469 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:04.469 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715248+0000", 2024-04-20T01:48:04.469 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.469 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.469 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.469 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:04.469 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715248+0000", 2024-04-20T01:48:04.469 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.469 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.469 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.469 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:04.470 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715248+0000", 2024-04-20T01:48:04.470 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:48:04.470 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.470 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.470 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:04.470 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715269+0000", 2024-04-20T01:48:04.470 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1322999999999999e-05 2024-04-20T01:48:04.470 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.470 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.470 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:04.470 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715278+0000", 2024-04-20T01:48:04.470 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.0869999999999996e-06 2024-04-20T01:48:04.470 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.470 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.470 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:04.471 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715286+0000", 2024-04-20T01:48:04.471 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.1000000000000004e-06 2024-04-20T01:48:04.471 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.471 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.471 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:04.471 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715336+0000", 2024-04-20T01:48:04.471 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.9503999999999999e-05 2024-04-20T01:48:04.471 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.471 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.471 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:04.471 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715347+0000", 2024-04-20T01:48:04.471 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0577000000000001e-05 2024-04-20T01:48:04.471 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.471 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.471 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:04.472 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720458+0000", 2024-04-20T01:48:04.472 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0051115600000000002 2024-04-20T01:48:04.472 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.472 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.472 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:04.472 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720480+0000", 2024-04-20T01:48:04.472 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2113e-05 2024-04-20T01:48:04.472 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.472 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.472 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.472 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.472 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.472 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.as2 48/30 ECSubWrite(tid=33, reqid=osd.6.0:33, at_version=48'14, trim_to=0'0, roll_forward_to=47'12))", 2024-04-20T01:48:04.472 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.716701+0000", 2024-04-20T01:48:04.472 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.716423133999999, 2024-04-20T01:48:04.473 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.006490778, 2024-04-20T01:48:04.473 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:04.473 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:04.473 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:04.473 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.473 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:04.473 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.716701+0000", 2024-04-20T01:48:04.473 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.473 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.473 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.473 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:04.473 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.716701+0000", 2024-04-20T01:48:04.473 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.473 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.473 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.473 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:04.474 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.716701+0000", 2024-04-20T01:48:04.474 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:48:04.474 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.474 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.474 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:04.474 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.716719+0000", 2024-04-20T01:48:04.474 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7574000000000001e-05 2024-04-20T01:48:04.474 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.474 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.474 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:04.474 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.716731+0000", 2024-04-20T01:48:04.474 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1949e-05 2024-04-20T01:48:04.474 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.474 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.474 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:04.475 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.716739+0000", 2024-04-20T01:48:04.475 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4190000000000002e-06 2024-04-20T01:48:04.475 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.475 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.475 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:04.475 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.716782+0000", 2024-04-20T01:48:04.475 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2803000000000003e-05 2024-04-20T01:48:04.475 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.475 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.475 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:04.475 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.716789+0000", 2024-04-20T01:48:04.475 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.3100000000000003e-06 2024-04-20T01:48:04.475 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.475 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.475 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:04.476 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723173+0000", 2024-04-20T01:48:04.476 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0063837140000000004 2024-04-20T01:48:04.476 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.476 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.476 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:04.476 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723192+0000", 2024-04-20T01:48:04.476 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9154999999999999e-05 2024-04-20T01:48:04.476 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.476 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.476 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.476 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.476 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.476 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 48/30 ECSubWrite(tid=120, reqid=osd.4.0:120, at_version=48'51, trim_to=0'0, roll_forward_to=47'49))", 2024-04-20T01:48:04.476 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.717383+0000", 2024-04-20T01:48:04.476 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.715741716, 2024-04-20T01:48:04.477 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0058028560000000003, 2024-04-20T01:48:04.477 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:04.477 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:04.477 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:04.477 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.477 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:04.477 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.717383+0000", 2024-04-20T01:48:04.477 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.477 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.477 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.477 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:04.477 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.717383+0000", 2024-04-20T01:48:04.477 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.477 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.477 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.477 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:04.478 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.717382+0000", 2024-04-20T01:48:04.478 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:04.478 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.478 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.478 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:04.478 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.717404+0000", 2024-04-20T01:48:04.478 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1586999999999998e-05 2024-04-20T01:48:04.478 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.478 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.478 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:04.478 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.717419+0000", 2024-04-20T01:48:04.478 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5180999999999999e-05 2024-04-20T01:48:04.478 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.478 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.478 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:04.479 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.717429+0000", 2024-04-20T01:48:04.479 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4290000000000007e-06 2024-04-20T01:48:04.479 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.479 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.479 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:04.479 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.717502+0000", 2024-04-20T01:48:04.479 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.3638000000000002e-05 2024-04-20T01:48:04.479 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.479 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.479 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:04.479 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.717511+0000", 2024-04-20T01:48:04.479 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.0729999999999994e-06 2024-04-20T01:48:04.479 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.479 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.479 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:04.479 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723164+0000", 2024-04-20T01:48:04.480 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0056525639999999997 2024-04-20T01:48:04.480 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.480 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.480 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:04.480 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723186+0000", 2024-04-20T01:48:04.480 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1637999999999998e-05 2024-04-20T01:48:04.480 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.480 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.480 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.480 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.480 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.480 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 48/30 ECSubWrite(tid=121, reqid=osd.4.0:121, at_version=48'53, trim_to=0'0, roll_forward_to=47'49))", 2024-04-20T01:48:04.480 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.718319+0000", 2024-04-20T01:48:04.480 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.714805837, 2024-04-20T01:48:04.480 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0075908039999999996, 2024-04-20T01:48:04.481 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:04.481 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:04.481 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:04.481 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.481 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:04.481 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718319+0000", 2024-04-20T01:48:04.481 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.481 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.481 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.481 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:04.481 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718319+0000", 2024-04-20T01:48:04.481 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.481 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.481 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.481 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:04.481 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718318+0000", 2024-04-20T01:48:04.482 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:04.482 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.482 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.482 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:04.482 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718351+0000", 2024-04-20T01:48:04.482 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.3179000000000002e-05 2024-04-20T01:48:04.482 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.482 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.482 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:04.482 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718362+0000", 2024-04-20T01:48:04.482 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0689999999999999e-05 2024-04-20T01:48:04.482 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.482 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.482 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:04.482 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718370+0000", 2024-04-20T01:48:04.483 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4610000000000008e-06 2024-04-20T01:48:04.483 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.483 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.483 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:04.483 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718703+0000", 2024-04-20T01:48:04.483 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00033314100000000001 2024-04-20T01:48:04.483 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.483 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.483 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:04.483 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718713+0000", 2024-04-20T01:48:04.483 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.5459999999999997e-06 2024-04-20T01:48:04.483 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.483 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.483 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:04.483 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.725887+0000", 2024-04-20T01:48:04.483 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0071743909999999996 2024-04-20T01:48:04.484 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.484 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.484 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:04.484 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.725909+0000", 2024-04-20T01:48:04.484 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1965999999999999e-05 2024-04-20T01:48:04.484 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.484 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.484 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.484 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.484 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.484 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.9s1 48/30 ECSubWrite(tid=18, reqid=osd.7.0:17, at_version=0'0, trim_to=0'0, roll_forward_to=48'12))", 2024-04-20T01:48:04.484 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.723162+0000", 2024-04-20T01:48:04.484 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.709962795999999, 2024-04-20T01:48:04.484 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.005487034, 2024-04-20T01:48:04.485 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:04.485 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:04.485 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:04.485 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.485 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:04.485 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723162+0000", 2024-04-20T01:48:04.485 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.485 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.485 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.485 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:04.485 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723162+0000", 2024-04-20T01:48:04.485 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.485 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.485 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.485 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:04.485 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723161+0000", 2024-04-20T01:48:04.486 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:04.486 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.486 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.486 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:04.486 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723171+0000", 2024-04-20T01:48:04.486 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4960000000000002e-06 2024-04-20T01:48:04.486 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.486 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.486 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:04.486 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723179+0000", 2024-04-20T01:48:04.486 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.3999999999999992e-06 2024-04-20T01:48:04.486 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.486 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.486 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:04.486 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723191+0000", 2024-04-20T01:48:04.487 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1878999999999999e-05 2024-04-20T01:48:04.487 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.487 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.487 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:04.487 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723236+0000", 2024-04-20T01:48:04.487 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5611000000000001e-05 2024-04-20T01:48:04.487 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.487 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.487 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:04.487 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.723246+0000", 2024-04-20T01:48:04.487 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.9019999999999994e-06 2024-04-20T01:48:04.487 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.487 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.487 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:04.488 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.728625+0000", 2024-04-20T01:48:04.488 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0053787369999999998 2024-04-20T01:48:04.488 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.488 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.488 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:04.488 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.728649+0000", 2024-04-20T01:48:04.488 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3589000000000001e-05 2024-04-20T01:48:04.488 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.488 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.488 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.488 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.488 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.488 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.as2 48/30 ECSubWrite(tid=34, reqid=osd.6.0:33, at_version=0'0, trim_to=0'0, roll_forward_to=48'14))", 2024-04-20T01:48:04.488 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.726124+0000", 2024-04-20T01:48:04.488 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.707000009, 2024-04-20T01:48:04.489 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0051488189999999998, 2024-04-20T01:48:04.489 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:04.489 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:04.489 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:04.489 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.489 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:04.489 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.726124+0000", 2024-04-20T01:48:04.489 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.489 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.489 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.489 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:04.489 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.726124+0000", 2024-04-20T01:48:04.489 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.489 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.489 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.490 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:04.490 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.726124+0000", 2024-04-20T01:48:04.490 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:04.490 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.490 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.490 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:04.490 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.726134+0000", 2024-04-20T01:48:04.490 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.9040000000000004e-06 2024-04-20T01:48:04.490 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.490 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.490 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:04.490 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.726147+0000", 2024-04-20T01:48:04.490 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3366e-05 2024-04-20T01:48:04.490 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.490 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.491 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:04.491 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.726158+0000", 2024-04-20T01:48:04.491 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.06e-05 2024-04-20T01:48:04.491 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.491 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.491 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:04.491 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.726207+0000", 2024-04-20T01:48:04.491 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.9153000000000002e-05 2024-04-20T01:48:04.491 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.491 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.491 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:04.491 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.726214+0000", 2024-04-20T01:48:04.491 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.9630000000000001e-06 2024-04-20T01:48:04.491 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.491 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.491 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:04.492 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.731248+0000", 2024-04-20T01:48:04.492 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0050344830000000002 2024-04-20T01:48:04.492 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.492 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.492 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:04.492 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.731273+0000", 2024-04-20T01:48:04.492 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5040000000000001e-05 2024-04-20T01:48:04.492 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.492 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.492 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.492 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.492 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.492 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s3 53/30 ECSubWrite(tid=32, reqid=client.4643.0:443, at_version=53'21, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:04.492 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:54.718764+0000", 2024-04-20T01:48:04.492 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 9.7143601999999998, 2024-04-20T01:48:04.493 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0048456610000000002, 2024-04-20T01:48:04.493 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:04.493 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:04.493 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:04.493 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.493 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:04.493 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.718764+0000", 2024-04-20T01:48:04.493 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.493 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.493 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.493 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:04.493 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.718764+0000", 2024-04-20T01:48:04.493 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.493 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.493 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.494 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:04.494 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.718764+0000", 2024-04-20T01:48:04.494 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:04.494 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.494 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.494 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:04.494 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.719094+0000", 2024-04-20T01:48:04.494 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000330534 2024-04-20T01:48:04.494 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.494 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.494 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:04.494 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.719111+0000", 2024-04-20T01:48:04.494 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7027000000000001e-05 2024-04-20T01:48:04.494 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.494 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.495 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:04.495 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.719125+0000", 2024-04-20T01:48:04.495 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3675e-05 2024-04-20T01:48:04.495 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.495 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.495 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:04.495 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.719231+0000", 2024-04-20T01:48:04.495 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0001058 2024-04-20T01:48:04.495 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.495 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.495 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:04.495 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.719251+0000", 2024-04-20T01:48:04.495 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9698999999999999e-05 2024-04-20T01:48:04.495 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.495 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.496 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:04.496 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723583+0000", 2024-04-20T01:48:04.496 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.004331957 2024-04-20T01:48:04.496 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.496 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.496 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:04.496 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723610+0000", 2024-04-20T01:48:04.496 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7285e-05 2024-04-20T01:48:04.496 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.496 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.496 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.496 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.496 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.496 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s3 53/30 ECSubWrite(tid=34, reqid=client.4643.0:445, at_version=53'23, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:04.496 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:54.722837+0000", 2024-04-20T01:48:04.496 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 9.7102874000000003, 2024-04-20T01:48:04.497 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0058785790000000001, 2024-04-20T01:48:04.497 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:04.497 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:04.497 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:04.497 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.497 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:04.497 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.722837+0000", 2024-04-20T01:48:04.497 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.497 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.497 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.497 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:04.497 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.722837+0000", 2024-04-20T01:48:04.497 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:04.497 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.497 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.498 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:04.498 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.722836+0000", 2024-04-20T01:48:04.498 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:04.498 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.498 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.498 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:04.498 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723118+0000", 2024-04-20T01:48:04.498 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00028123299999999998 2024-04-20T01:48:04.498 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.498 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.498 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:04.498 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723136+0000", 2024-04-20T01:48:04.498 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8400999999999999e-05 2024-04-20T01:48:04.498 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.498 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.499 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:04.499 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723145+0000", 2024-04-20T01:48:04.499 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.5140000000000001e-06 2024-04-20T01:48:04.499 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.499 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.499 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:04.499 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723204+0000", 2024-04-20T01:48:04.499 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.9077999999999999e-05 2024-04-20T01:48:04.499 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.499 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.499 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:04.499 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723213+0000", 2024-04-20T01:48:04.499 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3980000000000005e-06 2024-04-20T01:48:04.499 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.499 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.499 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:04.500 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.728692+0000", 2024-04-20T01:48:04.500 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0054791789999999998 2024-04-20T01:48:04.500 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:04.500 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:04.500 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:04.500 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.728716+0000", 2024-04-20T01:48:04.500 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3388e-05 2024-04-20T01:48:04.500 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.500 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.500 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.500 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:04.500 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:04.500 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:04.501 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:48:04.502 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:04.516+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:04.502 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-20T01:48:04.603 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:04.617+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:04.622 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:48:04.626 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T01:48:04.630 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:04.630 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:04.630 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 0 2024-04-20T01:48:04.630 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:04.636 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-04-20T01:48:03.706993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":63095328,"num_objects":76,"num_object_clones":31,"num_object_copies":522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":4,"num_read":413,"num_read_kb":45830,"num_write":708,"num_write_kb":72676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":24199168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":642,"ondisk_log_size":642,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":553028,"kb_used_data":120616,"kb_used_omap":24,"kb_used_meta":432103,"kb_avail":1509396412,"statfs":{"total":1546188226560,"available":1545621925888,"internally_reserved":0,"allocated":123510784,"data_stored":146783636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25507,"internal_metadata":442473565},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11730944,"num_objects":11,"num_object_clones":11,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":21,"num_read_kb":6717,"num_write":31,"num_write_kb":11456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":13697024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.981480"},"pg_stats":[{"pgid":"3.f","version":"55'26","reported_seq":120,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-20T01:47:59.766338+0000","last_change":"2024-04-20T01:47:56.747163+0000","last_active":"2024-04-20T01:47:59.766338+0000","last_peered":"2024-04-20T01:47:59.766338+0000","last_clean":"2024-04-20T01:47:59.766338+0000","last_became_active":"2024-04-20T01:47:32.867228+0000","last_became_peered":"2024-04-20T01:47:32.867228+0000","last_unstale":"2024-04-20T01:47:59.766338+0000","last_undegraded":"2024-04-20T01:47:59.766338+0000","last_fullsized":"2024-04-20T01:47:59.766338+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.328818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018547972999999999,"stat_sum":{"num_bytes":2686976,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":20,"num_read_kb":5143,"num_write":54,"num_write_kb":2912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,15,13,8,11,0],"acting":[2,3,15,13,8,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"2"},{"start":"11","length":"1"},{"start":"13","length":"1"},{"start":"15","length":"1"}]},{"pgid":"3.e","version":"55'27","reported_seq":126,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-20T01:47:59.779956+0000","last_change":"2024-04-20T01:47:56.735821+0000","last_active":"2024-04-20T01:47:59.779956+0000","last_peered":"2024-04-20T01:47:59.779956+0000","last_clean":"2024-04-20T01:47:59.779956+0000","last_became_active":"2024-04-20T01:47:32.867409+0000","last_became_peered":"2024-04-20T01:47:32.867409+0000","last_unstale":"2024-04-20T01:47:59.779956+0000","last_undegraded":"2024-04-20T01:47:59.779956+0000","last_fullsized":"2024-04-20T01:47:59.779956+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:42.581217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049757239999999999,"stat_sum":{"num_bytes":802816,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":2312,"num_write":15,"num_write_kb":3280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,1,10,8,3,12],"acting":[11,9,1,10,8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"2"},{"start":"11","length":"1"},{"start":"13","length":"1"},{"start":"15","length":"1"}]},{"pgid":"3.d","version":"41'12","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-20T01:47:59.731516+0000","last_change":"2024-04-20T01:47:56.728645+0000","last_active":"2024-04-20T01:47:59.731516+0000","last_peered":"2024-04-20T01:47:59.731516+0000","last_clean":"2024-04-20T01:47:59.731516+0000","last_became_active":"2024-04-20T01:47:32.867954+0000","last_became_peered":"2024-04-20T01:47:32.867954+0000","last_unstale":"2024-04-20T01:47:59.731516+0000","last_undegraded":"2024-04-20T01:47:59.731516+0000","last_fullsized":"2024-04-20T01:47:59.731516+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.083554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054290199999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,11,15,7,8,4],"acting":[6,2,11,15,7,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"2"},{"start":"11","length":"1"},{"start":"13","length":"1"},{"start":"15","length":"1"}]},{"pgid":"3.c","version":"55'41","reported_seq":126,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-20T01:47:59.766083+0000","last_change":"2024-04-20T01:47:56.745337+0000","last_active":"2024-04-20T01:47:59.766083+0000","last_peered":"2024-04-20T01:47:59.766083+0000","last_clean":"2024-04-20T01:47:59.766083+0000","last_became_active":"2024-04-20T01:47:32.867926+0000","last_became_peered":"2024-04-20T01:47:32.867926+0000","last_unstale":"2024-04-20T01:47:59.766083+0000","last_undegraded":"2024-04-20T01:47:59.766083+0000","last_fullsized":"2024-04-20T01:47:59.766083+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:54.374528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045182600000000001,"stat_sum":{"num_bytes":7208960,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":44,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,8,6,3,7,12,5],"acting":[14,8,6,3,7,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"2"},{"start":"11","length":"1"},{"start":"13","length":"1"},{"start":"15","length":"1"}]},{"pgid":"3.b","version":"55'60","reported_seq":162,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-20T01:47:59.775138+0000","last_change":"2024-04-20T01:47:56.739603+0000","last_active":"2024-04-20T01:47:59.775138+0000","last_peered":"2024-04-20T01:47:59.775138+0000","last_clean":"2024-04-20T01:47:59.775138+0000","last_became_active":"2024-04-20T01:47:32.867309+0000","last_became_peered":"2024-04-20T01:47:32.867309+0000","last_unstale":"2024-04-20T01:47:59.775138+0000","last_undegraded":"2024-04-20T01:47:59.775138+0000","last_fullsized":"2024-04-20T01:47:59.775138+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:01.227735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010873048999999999,"stat_sum":{"num_bytes":8273920,"num_objects":9,"num_object_clones":3,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":37,"num_read_kb":5391,"num_write":97,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,11,10,9],"acting":[4,0,1,12,11,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"2"},{"start":"11","length":"1"},{"start":"13","length":"1"},{"start":"15","length":"1"}]},{"pgid":"3.a","version":"48'14","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-20T01:47:59.731747+0000","last_change":"2024-04-20T01:47:56.728691+0000","last_active":"2024-04-20T01:47:59.731747+0000","last_peered":"2024-04-20T01:47:59.731747+0000","last_clean":"2024-04-20T01:47:59.731747+0000","last_became_active":"2024-04-20T01:47:32.868002+0000","last_became_peered":"2024-04-20T01:47:32.868002+0000","last_unstale":"2024-04-20T01:47:59.731747+0000","last_undegraded":"2024-04-20T01:47:59.731747+0000","last_fullsized":"2024-04-20T01:47:59.731747+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:35.716198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052242300000000005,"stat_sum":{"num_bytes":1736704,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":8,"num_read_kb":1971,"num_write":6,"num_write_kb":1696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,1,14,3,15,2],"acting":[6,10,1,14,3,15,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"2"},{"start":"11","length":"1"},{"start":"13","length":"1"},{"start":"15","length":"1"}]},{"pgid":"3.9","version":"52'15","reported_seq":103,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-20T01:47:59.731237+0000","last_change":"2024-04-20T01:47:56.733226+0000","last_active":"2024-04-20T01:47:59.731237+0000","last_peered":"2024-04-20T01:47:59.731237+0000","last_clean":"2024-04-20T01:47:59.731237+0000","last_became_active":"2024-04-20T01:47:32.866706+0000","last_became_peered":"2024-04-20T01:47:32.866706+0000","last_unstale":"2024-04-20T01:47:59.731237+0000","last_undegraded":"2024-04-20T01:47:59.731237+0000","last_fullsized":"2024-04-20T01:47:59.731237+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:42.776206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064997600000000003,"stat_sum":{"num_bytes":1097728,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":10,"num_read_kb":1251,"num_write":27,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0,11,14,8,3],"acting":[7,1,0,11,14,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"2"},{"start":"11","length":"1"},{"start":"13","length":"1"},{"start":"15","length":"1"}]},{"pgid":"3.8","version":"58'42","reported_seq":134,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-20T01:47:59.780966+0000","last_change":"2024-04-20T01:47:56.715908+0000","last_active":"2024-04-20T01:47:59.780966+0000","last_peered":"2024-04-20T01:47:59.780966+0000","last_clean":"2024-04-20T01:47:59.780966+0000","last_became_active":"2024-04-20T01:47:32.718069+0000","last_became_peered":"2024-04-20T01:47:32.718069+0000","last_unstale":"2024-04-20T01:47:59.780966+0000","last_undegraded":"2024-04-20T01:47:59.780966+0000","last_fullsized":"2024-04-20T01:47:59.780966+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:58.489885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033964200000000003,"stat_sum":{"num_bytes":6340608,"num_objects":7,"num_object_clones":3,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":18,"num_read_kb":537,"num_write":27,"num_write_kb":6800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,11,3,14,2,7],"acting":[5,0,11,3,14,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"2"},{"start":"11","length":"1"},{"start":"13","length":"1"},{"start":"15","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-20T01:47:59.776597+0000","last_change":"2024-04-20T01:47:20.071988+0000","last_active":"2024-04-20T01:47:59.776597+0000","last_peered":"2024-04-20T01:47:59.776597+0000","last_clean":"2024-04-20T01:47:59.776597+0000","last_became_active":"2024-04-20T01:47:20.071615+0000","last_became_peered":"2024-04-20T01:47:20.071615+0000","last_unstale":"2024-04-20T01:47:59.776597+0000","last_undegraded":"2024-04-20T01:47:59.776597+0000","last_fullsized":"2024-04-20T01:47:59.776597+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:40.106777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"55'9","reported_seq":96,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-20T01:47:59.731485+0000","last_change":"2024-04-20T01:47:56.736763+0000","last_active":"2024-04-20T01:47:59.731485+0000","last_peered":"2024-04-20T01:47:59.731485+0000","last_clean":"2024-04-20T01:47:59.731485+0000","last_became_active":"2024-04-20T01:47:32.867180+0000","last_became_peered":"2024-04-20T01:47:32.867180+0000","last_unstale":"2024-04-20T01:47:59.731485+0000","last_undegraded":"2024-04-20T01:47:59.731485+0000","last_fullsized":"2024-04-20T01:47:59.731485+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:43.358783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040875800000000001,"stat_sum":{"num_bytes":360448,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":1059,"num_write":7,"num_write_kb":416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,4,14,12,0,11],"acting":[10,8,4,14,12,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"2"},{"start":"11","length":"1"},{"start":"13","length":"1"},{"start":"15","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-20T01:47:59.766225+0000","last_change":"2024-04-20T01:47:20.075613+0000","last_active":"2024-04-20T01:47:59.766225+0000","last_peered":"2024-04-20T01:47:59.766225+0000","last_clean":"2024-04-20T01:47:59.766225+0000","last_became_active":"2024-04-20T01:47:20.075314+0000","last_became_peered":"2024-04-20T01:47:20.075314+0000","last_unstale":"2024-04-20T01:47:59.766225+0000","last_undegraded":"2024-04-20T01:47:59.766225+0000","last_fullsized":"2024-04-20T01:47:59.766225+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:35.956173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,12],"acting":[2,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"48'22","reported_seq":115,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T01:48:01.749848+0000","last_change":"2024-04-20T01:48:01.750225+0000","last_active":"2024-04-20T01:48:01.749848+0000","last_peered":"2024-04-20T01:48:01.749848+0000","last_clean":"2024-04-20T01:48:01.749848+0000","last_became_active":"2024-04-20T01:47:32.719136+0000","last_became_peered":"2024-04-20T01:47:32.719136+0000","last_unstale":"2024-04-20T01:48:01.749848+0000","last_undegraded":"2024-04-20T01:48:01.749848+0000","last_fullsized":"2024-04-20T01:48:01.749848+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:46.940575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000522964,"stat_sum":{"num_bytes":1540096,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":3,"num_write":30,"num_write_kb":2128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,4,14,11,5,10],"acting":[13,2,4,14,11,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"2"},{"start":"11","length":"1"},{"start":"13","length":"1"},{"start":"15","length":"1"},{"start":"18","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T01:48:00.739522+0000","last_change":"2024-04-20T01:47:20.039221+0000","last_active":"2024-04-20T01:48:00.739522+0000","last_peered":"2024-04-20T01:48:00.739522+0000","last_clean":"2024-04-20T01:48:00.739522+0000","last_became_active":"2024-04-20T01:47:20.038972+0000","last_became_peered":"2024-04-20T01:47:20.038972+0000","last_unstale":"2024-04-20T01:48:00.739522+0000","last_undegraded":"2024-04-20T01:48:00.739522+0000","last_fullsized":"2024-04-20T01:48:00.739522+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:24.167832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"55'33","reported_seq":127,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T01:48:00.742555+0000","last_change":"2024-04-20T01:47:56.748275+0000","last_active":"2024-04-20T01:48:00.742555+0000","last_peered":"2024-04-20T01:48:00.742555+0000","last_clean":"2024-04-20T01:48:00.742555+0000","last_became_active":"2024-04-20T01:47:32.868365+0000","last_became_peered":"2024-04-20T01:47:32.868365+0000","last_unstale":"2024-04-20T01:48:00.742555+0000","last_undegraded":"2024-04-20T01:48:00.742555+0000","last_fullsized":"2024-04-20T01:48:00.742555+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:41.198793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024124432000000001,"stat_sum":{"num_bytes":3850240,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":1928,"num_write":37,"num_write_kb":4240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,1,5,15,4],"acting":[3,7,6,1,5,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"2"},{"start":"11","length":"1"},{"start":"13","length":"1"},{"start":"15","length":"1"}]},{"pgid":"2.3","version":"0'0","reported_seq":42,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-20T01:47:59.763022+0000","last_change":"2024-04-20T01:47:20.039014+0000","last_active":"2024-04-20T01:47:59.763022+0000","last_peered":"2024-04-20T01:47:59.763022+0000","last_clean":"2024-04-20T01:47:59.763022+0000","last_became_active":"2024-04-20T01:47:20.038764+0000","last_became_peered":"2024-04-20T01:47:20.038764+0000","last_unstale":"2024-04-20T01:47:59.763022+0000","last_undegraded":"2024-04-20T01:47:59.763022+0000","last_fullsized":"2024-04-20T01:47:59.763022+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:26.112165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"25'157","reported_seq":240,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T01:48:00.739405+0000","last_change":"2024-04-20T01:47:17.303029+0000","last_active":"2024-04-20T01:48:00.739405+0000","last_peered":"2024-04-20T01:48:00.739405+0000","last_clean":"2024-04-20T01:48:00.739405+0000","last_became_active":"2024-04-20T01:47:17.302105+0000","last_became_peered":"2024-04-20T01:47:17.302105+0000","last_unstale":"2024-04-20T01:48:00.739405+0000","last_undegraded":"2024-04-20T01:48:00.739405+0000","last_fullsized":"2024-04-20T01:48:00.739405+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_clean_scrub_stamp":"2024-04-20T01:47:15.986754+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:30.481506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,15],"acting":[13,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.2","version":"60'69","reported_seq":174,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-20T01:48:01.849758+0000","last_change":"2024-04-20T01:48:01.760898+0000","last_active":"2024-04-20T01:48:01.849758+0000","last_peered":"2024-04-20T01:48:01.849758+0000","last_clean":"2024-04-20T01:48:01.849758+0000","last_became_active":"2024-04-20T01:47:32.866509+0000","last_became_peered":"2024-04-20T01:47:32.866509+0000","last_unstale":"2024-04-20T01:48:01.849758+0000","last_undegraded":"2024-04-20T01:48:01.849758+0000","last_fullsized":"2024-04-20T01:48:01.849758+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.395078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0111784,"stat_sum":{"num_bytes":13008896,"num_objects":12,"num_object_clones":6,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":42,"num_read_kb":9376,"num_write":58,"num_write_kb":13440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,6,10,14,1,7],"acting":[13,3,6,10,14,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"2"},{"start":"11","length":"1"},{"start":"13","length":"1"},{"start":"15","length":"1"},{"start":"18","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":41,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-20T01:47:59.766186+0000","last_change":"2024-04-20T01:47:20.069279+0000","last_active":"2024-04-20T01:47:59.766186+0000","last_peered":"2024-04-20T01:47:59.766186+0000","last_clean":"2024-04-20T01:47:59.766186+0000","last_became_active":"2024-04-20T01:47:20.069005+0000","last_became_peered":"2024-04-20T01:47:20.069005+0000","last_unstale":"2024-04-20T01:47:59.766186+0000","last_undegraded":"2024-04-20T01:47:59.766186+0000","last_fullsized":"2024-04-20T01:47:59.766186+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:22.648743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"58'16","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-20T01:47:59.787342+0000","last_change":"2024-04-20T01:47:56.729093+0000","last_active":"2024-04-20T01:47:59.787342+0000","last_peered":"2024-04-20T01:47:59.787342+0000","last_clean":"2024-04-20T01:47:59.787342+0000","last_became_active":"2024-04-20T01:47:32.722189+0000","last_became_peered":"2024-04-20T01:47:32.722189+0000","last_unstale":"2024-04-20T01:47:59.787342+0000","last_undegraded":"2024-04-20T01:47:59.787342+0000","last_fullsized":"2024-04-20T01:47:59.787342+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:43.443403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053983799999999995,"stat_sum":{"num_bytes":1081344,"num_objects":3,"num_object_clones":2,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":1011,"num_write":7,"num_write_kb":1488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,14,9,5,6,3,13],"acting":[4,14,9,5,6,3,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"2"},{"start":"11","length":"1"},{"start":"13","length":"1"},{"start":"15","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-20T01:47:59.734616+0000","last_change":"2024-04-20T01:47:20.038409+0000","last_active":"2024-04-20T01:47:59.734616+0000","last_peered":"2024-04-20T01:47:59.734616+0000","last_clean":"2024-04-20T01:47:59.734616+0000","last_became_active":"2024-04-20T01:47:20.038107+0000","last_became_peered":"2024-04-20T01:47:20.038107+0000","last_unstale":"2024-04-20T01:47:59.734616+0000","last_undegraded":"2024-04-20T01:47:59.734616+0000","last_fullsized":"2024-04-20T01:47:59.734616+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:44.125577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"58'81","reported_seq":383,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-20T01:47:59.924051+0000","last_change":"2024-04-20T01:47:59.924051+0000","last_active":"2024-04-20T01:47:59.924051+0000","last_peered":"2024-04-20T01:47:59.924051+0000","last_clean":"2024-04-20T01:47:59.924051+0000","last_became_active":"2024-04-20T01:47:59.783367+0000","last_became_peered":"2024-04-20T01:47:59.783367+0000","last_unstale":"2024-04-20T01:47:59.924051+0000","last_undegraded":"2024-04-20T01:47:59.924051+0000","last_fullsized":"2024-04-20T01:47:59.924051+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:51.089820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057439399999999999,"stat_sum":{"num_bytes":11993088,"num_objects":11,"num_object_clones":5,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":38,"num_read_kb":10277,"num_write":59,"num_write_kb":12496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":24199168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,15,4,1,0,3,13],"acting":[8,15,4,1,0,3,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"2"},{"start":"11","length":"1"},{"start":"13","length":"1"},{"start":"15","length":"1"}]},{"pgid":"3.7","version":"41'12","reported_seq":101,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-20T01:47:59.775161+0000","last_change":"2024-04-20T01:47:56.729297+0000","last_active":"2024-04-20T01:47:59.775161+0000","last_peered":"2024-04-20T01:47:59.775161+0000","last_clean":"2024-04-20T01:47:59.775161+0000","last_became_active":"2024-04-20T01:47:32.722002+0000","last_became_peered":"2024-04-20T01:47:32.722002+0000","last_unstale":"2024-04-20T01:47:59.775161+0000","last_undegraded":"2024-04-20T01:47:59.775161+0000","last_fullsized":"2024-04-20T01:47:59.775161+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:43.990113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074629099999999995,"stat_sum":{"num_bytes":2523136,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":5058,"num_write":28,"num_write_kb":2464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,0,8,5,14],"acting":[4,11,9,0,8,5,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"2"},{"start":"11","length":"1"},{"start":"13","length":"1"},{"start":"15","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-20T01:47:58.723826+0000","last_change":"2024-04-20T01:47:20.110600+0000","last_active":"2024-04-20T01:47:58.723826+0000","last_peered":"2024-04-20T01:47:58.723826+0000","last_clean":"2024-04-20T01:47:58.723826+0000","last_became_active":"2024-04-20T01:47:20.110260+0000","last_became_peered":"2024-04-20T01:47:20.110260+0000","last_unstale":"2024-04-20T01:47:58.723826+0000","last_undegraded":"2024-04-20T01:47:58.723826+0000","last_fullsized":"2024-04-20T01:47:58.723826+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:47.718290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":42,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-20T01:47:59.731595+0000","last_change":"2024-04-20T01:47:20.036634+0000","last_active":"2024-04-20T01:47:59.731595+0000","last_peered":"2024-04-20T01:47:59.731595+0000","last_clean":"2024-04-20T01:47:59.731595+0000","last_became_active":"2024-04-20T01:47:20.036328+0000","last_became_peered":"2024-04-20T01:47:20.036328+0000","last_unstale":"2024-04-20T01:47:59.731595+0000","last_undegraded":"2024-04-20T01:47:59.731595+0000","last_fullsized":"2024-04-20T01:47:59.731595+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.385250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,13],"acting":[6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"47'6","reported_seq":93,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-20T01:47:58.728047+0000","last_change":"2024-04-20T01:47:56.735407+0000","last_active":"2024-04-20T01:47:58.728047+0000","last_peered":"2024-04-20T01:47:58.728047+0000","last_clean":"2024-04-20T01:47:58.728047+0000","last_became_active":"2024-04-20T01:47:32.866834+0000","last_became_peered":"2024-04-20T01:47:32.866834+0000","last_unstale":"2024-04-20T01:47:58.728047+0000","last_undegraded":"2024-04-20T01:47:58.728047+0000","last_fullsized":"2024-04-20T01:47:58.728047+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:54.424527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052822299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":386,"num_write":3,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,3,6,2,9],"acting":[0,13,1,3,6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"2"},{"start":"11","length":"1"},{"start":"13","length":"1"},{"start":"15","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":62504960,"num_objects":74,"num_object_clones":31,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":4,"num_read":277,"num_read_kb":45712,"num_write":505,"num_write_kb":69216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":24199168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110104576,"data_stored":139919360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":485,"ondisk_log_size":485,"up":112,"acting":112,"num_store_stats":16},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280524,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36112,"kb_used_data":9092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335728,"statfs":{"total":96636764160,"available":96599785472,"internally_reserved":0,"allocated":9310208,"data_stored":10563525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]}]},{"osd":15,"up_from":21,"seq":90194313228,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34072,"kb_used_data":7044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337768,"statfs":{"total":96636764160,"available":96601874432,"internally_reserved":0,"allocated":7213056,"data_stored":7809575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]},{"osd":14,"up_from":21,"seq":90194313229,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35912,"kb_used_data":8892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335928,"statfs":{"total":96636764160,"available":96599990272,"internally_reserved":0,"allocated":9105408,"data_stored":10894757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,4,5,6,7,8,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]}]},{"osd":12,"up_from":21,"seq":90194313228,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":4856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":4972544,"data_stored":6246400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]},{"osd":11,"up_from":21,"seq":90194313227,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33580,"kb_used_data":6560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338260,"statfs":{"total":96636764160,"available":96602378240,"internally_reserved":0,"allocated":6717440,"data_stored":7761920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,8,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]}]},{"osd":10,"up_from":21,"seq":90194313229,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33668,"kb_used_data":6640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338172,"statfs":{"total":96636764160,"available":96602288128,"internally_reserved":0,"allocated":6799360,"data_stored":7895047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,4,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":9,"up_from":21,"seq":90194313229,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30900,"kb_used_data":3872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340940,"statfs":{"total":96636764160,"available":96605122560,"internally_reserved":0,"allocated":3964928,"data_stored":4290567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,4,6,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]}]},{"osd":8,"up_from":21,"seq":90194313228,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34316,"kb_used_data":7288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337524,"statfs":{"total":96636764160,"available":96601624576,"internally_reserved":0,"allocated":7462912,"data_stored":9967623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9630000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]}]},{"osd":7,"up_from":21,"seq":90194313229,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35080,"kb_used_data":8052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336760,"statfs":{"total":96636764160,"available":96600842240,"internally_reserved":0,"allocated":8245248,"data_stored":9852935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,3,4,6,8,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":6,"up_from":21,"seq":90194313227,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33952,"kb_used_data":6924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337888,"statfs":{"total":96636764160,"available":96601997312,"internally_reserved":0,"allocated":7090176,"data_stored":8177671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]}]},{"osd":1,"up_from":21,"seq":90194313228,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37340,"kb_used_data":10312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334500,"statfs":{"total":96636764160,"available":96598528000,"internally_reserved":0,"allocated":10559488,"data_stored":13232135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]}]},{"osd":0,"up_from":21,"seq":90194313229,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36724,"kb_used_data":9696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335116,"statfs":{"total":96636764160,"available":96599158784,"internally_reserved":0,"allocated":9928704,"data_stored":11658685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]}]},{"osd":2,"up_from":21,"seq":90194313227,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30804,"kb_used_data":3780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341036,"statfs":{"total":96636764160,"available":96605220864,"internally_reserved":0,"allocated":3870720,"data_stored":4143111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]}]},{"osd":3,"up_from":21,"seq":90194313229,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39548,"kb_used_data":12520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94332292,"statfs":{"total":96636764160,"available":96596267008,"internally_reserved":0,"allocated":12820480,"data_stored":16340999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,4,5,6,7,8,9,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":4,"up_from":21,"seq":90194313229,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35836,"kb_used_data":8808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336004,"statfs":{"total":96636764160,"available":96600068096,"internally_reserved":0,"allocated":9019392,"data_stored":10360839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,5,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]}]},{"osd":5,"up_from":21,"seq":90194313228,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":6280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":6430720,"data_stored":7587847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9138176,"data_stored":11227136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9764864,"data_stored":12800000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3076096,"data_stored":3710976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12025856,"data_stored":15908864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8224768,"data_stored":9928704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5636096,"data_stored":7155712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6295552,"data_stored":7745536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7450624,"data_stored":9420800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":6668288,"data_stored":9535488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":3170304,"data_stored":3858432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":6004736,"data_stored":7462912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":5906432,"data_stored":7319552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":4161536,"data_stored":5804032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":8396800,"data_stored":10739712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":8298496,"data_stored":10452992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":5885952,"data_stored":6848512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T01:48:04.639 INFO:tasks.thrashosds.thrasher:Setting 2.4 to [3, 10, 14, 7] 2024-04-20T01:48:04.639 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.4', '3', '10', '14', '7'] 2024-04-20T01:48:04.639 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.4 3 10 14 7 2024-04-20T01:48:04.640 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-20T01:48:04.643 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:48:04.667 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:48:04.688 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:48:04.703 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:04.718+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:04.710 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:48:04.720 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-20T01:48:04.731 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:48:04.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:48:04.746+0000 7f510f7fe700 0 -- 172.21.15.44:0/1812583664 >> [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] conn(0x7f5110221910 msgr2=0x7f5110223d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-20T01:48:04.734 INFO:tasks.rados.rados.0.smithi165.stdout:223: done (7 left) 2024-04-20T01:48:04.734 INFO:tasks.rados.rados.0.smithi165.stdout:224: done (6 left) 2024-04-20T01:48:04.734 INFO:tasks.rados.rados.0.smithi165.stdout:225: done (5 left) 2024-04-20T01:48:04.734 INFO:tasks.rados.rados.0.smithi165.stdout:226: done (4 left) 2024-04-20T01:48:04.734 INFO:tasks.rados.rados.0.smithi165.stdout:227: done (3 left) 2024-04-20T01:48:04.734 INFO:tasks.rados.rados.0.smithi165.stdout:228: done (2 left) 2024-04-20T01:48:04.734 INFO:tasks.rados.rados.0.smithi165.stdout:229: done (1 left) 2024-04-20T01:48:04.734 INFO:tasks.rados.rados.0.smithi165.stdout:230: done (0 left) 2024-04-20T01:48:04.734 INFO:tasks.rados.rados.0.smithi165.stdout:231: read oid 8 snap 14 2024-04-20T01:48:04.734 INFO:tasks.rados.rados.0.smithi165.stdout:231: expect deleted 2024-04-20T01:48:04.734 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 31 current snap is 17 2024-04-20T01:48:04.734 INFO:tasks.rados.rados.0.smithi165.stdout:232: seq_num 107 ranges {507904=638976,1146880=458752,1605632=704512} 2024-04-20T01:48:04.740 INFO:tasks.rados.rados.0.smithi165.stdout:232: writing smithi16579500-31 from 507904 to 1146880 tid 1 2024-04-20T01:48:04.742 INFO:tasks.rados.rados.0.smithi165.stdout:232: writing smithi16579500-31 from 1146880 to 1605632 tid 2 2024-04-20T01:48:04.747 INFO:tasks.rados.rados.0.smithi165.stdout:232: writing smithi16579500-31 from 1605632 to 2310144 tid 3 2024-04-20T01:48:04.747 INFO:tasks.rados.rados.0.smithi165.stdout:233: read oid 16 snap 16 2024-04-20T01:48:04.747 INFO:tasks.rados.rados.0.smithi165.stdout:233: expect (ObjNum 15 snap 0 seq_num 15) 2024-04-20T01:48:04.747 INFO:tasks.rados.rados.0.smithi165.stdout:231: done (2 left) 2024-04-20T01:48:04.747 INFO:tasks.rados.rados.0.smithi165.stdout:234: setattr oid 8 current snap is 17 2024-04-20T01:48:04.749 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 38 current snap is 17 2024-04-20T01:48:04.749 INFO:tasks.rados.rados.0.smithi165.stdout:235: seq_num 108 ranges {4587520=655360,5242880=606208,5849088=475136,6324224=81920} 2024-04-20T01:48:04.777 INFO:tasks.rados.rados.0.smithi165.stdout:235: writing smithi16579500-38 from 4587520 to 5242880 tid 1 2024-04-20T01:48:04.825 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:04.825 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:04.825 INFO:teuthology.orchestra.run.smithi121.stdout: "complaint_time": 30, 2024-04-20T01:48:04.825 INFO:teuthology.orchestra.run.smithi121.stdout: "num_blocked_ops": 0 2024-04-20T01:48:04.825 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:04.826 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:04.839+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:04.827 INFO:tasks.rados.rados.0.smithi165.stdout:235: writing smithi16579500-38 from 5242880 to 5849088 tid 2 2024-04-20T01:48:04.827 INFO:tasks.rados.rados.0.smithi165.stdout:235: writing smithi16579500-38 from 5849088 to 6324224 tid 3 2024-04-20T01:48:04.827 INFO:tasks.rados.rados.0.smithi165.stdout:235: writing smithi16579500-38 from 6324224 to 6406144 tid 4 2024-04-20T01:48:04.827 INFO:tasks.rados.rados.0.smithi165.stdout:236: delete oid 19 current snap is 17 2024-04-20T01:48:04.827 INFO:tasks.rados.rados.0.smithi165.stdout:232: finishing write tid 1 to smithi16579500-31 2024-04-20T01:48:04.827 INFO:tasks.rados.rados.0.smithi165.stdout:232: finishing write tid 2 to smithi16579500-31 2024-04-20T01:48:04.828 INFO:tasks.rados.rados.0.smithi165.stdout:233: expect (ObjNum 15 snap 0 seq_num 15) 2024-04-20T01:48:04.828 INFO:tasks.rados.rados.0.smithi165.stdout:232: finishing write tid 3 to smithi16579500-31 2024-04-20T01:48:04.828 INFO:tasks.rados.rados.0.smithi165.stdout:232: finishing write tid 4 to smithi16579500-31 2024-04-20T01:48:04.828 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 8 v 7 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-20T01:48:04.828 INFO:tasks.rados.rados.0.smithi165.stdout:232: finishing write tid 5 to smithi16579500-31 2024-04-20T01:48:04.828 INFO:tasks.rados.rados.0.smithi165.stdout:232: oid 31 updating version 0 to 84 2024-04-20T01:48:04.828 INFO:tasks.rados.rados.0.smithi165.stdout:232: oid 31 version 84 is already newer than 83 2024-04-20T01:48:04.828 INFO:tasks.rados.rados.0.smithi165.stdout:232: oid 31 version 84 is already newer than 81 2024-04-20T01:48:04.828 INFO:tasks.rados.rados.0.smithi165.stdout:232: oid 31 version 84 is already newer than 82 2024-04-20T01:48:04.828 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 31 v 84 (ObjNum 107 snap 17 seq_num 107) dirty exists 2024-04-20T01:48:04.828 INFO:tasks.rados.rados.0.smithi165.stdout:232: left oid 31 (ObjNum 107 snap 17 seq_num 107) 2024-04-20T01:48:04.828 INFO:tasks.rados.rados.0.smithi165.stdout:235: finishing write tid 1 to smithi16579500-38 2024-04-20T01:48:04.828 INFO:tasks.rados.rados.0.smithi165.stdout:235: finishing write tid 2 to smithi16579500-38 2024-04-20T01:48:04.828 INFO:tasks.rados.rados.0.smithi165.stdout:235: finishing write tid 3 to smithi16579500-38 2024-04-20T01:48:04.828 INFO:tasks.rados.rados.0.smithi165.stdout:235: finishing write tid 4 to smithi16579500-38 2024-04-20T01:48:04.828 INFO:tasks.rados.rados.0.smithi165.stdout:235: finishing write tid 5 to smithi16579500-38 2024-04-20T01:48:04.829 INFO:tasks.rados.rados.0.smithi165.stdout:232: done (4 left) 2024-04-20T01:48:04.829 INFO:tasks.rados.rados.0.smithi165.stdout:233: done (3 left) 2024-04-20T01:48:04.829 INFO:tasks.rados.rados.0.smithi165.stdout:234: done (2 left) 2024-04-20T01:48:04.829 INFO:tasks.rados.rados.0.smithi165.stdout:236: done (1 left) 2024-04-20T01:48:04.829 INFO:tasks.rados.rados.0.smithi165.stdout:237: snap_create 2024-04-20T01:48:04.829 INFO:tasks.rados.rados.0.smithi165.stdout:235: finishing write tid 6 to smithi16579500-38 2024-04-20T01:48:04.829 INFO:tasks.rados.rados.0.smithi165.stdout:235: oid 38 updating version 0 to 81 2024-04-20T01:48:04.829 INFO:tasks.rados.rados.0.smithi165.stdout:235: oid 38 updating version 81 to 82 2024-04-20T01:48:04.829 INFO:tasks.rados.rados.0.smithi165.stdout:235: oid 38 updating version 82 to 83 2024-04-20T01:48:04.829 INFO:tasks.rados.rados.0.smithi165.stdout:235: oid 38 version 83 is already newer than 79 2024-04-20T01:48:04.829 INFO:tasks.rados.rados.0.smithi165.stdout:235: oid 38 version 83 is already newer than 80 2024-04-20T01:48:04.829 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 38 v 83 (ObjNum 108 snap 17 seq_num 108) dirty exists 2024-04-20T01:48:04.829 INFO:tasks.rados.rados.0.smithi165.stdout:235: left oid 38 (ObjNum 108 snap 17 seq_num 108) 2024-04-20T01:48:04.830 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-20T01:48:04.926 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:04.940+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:04.953 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:48:04.955 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:48:04.955 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:48:04.955 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:48:04.955 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:48:04.955 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:48:04.955 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:48:04.955 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:48:04.955 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:48:04.955 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:48:04.956 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:48:04.956 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:48:04.956 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:48:04.956 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:48:04.956 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:48:04.956 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:48:04.956 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:48:05.001 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:05.001 INFO:teuthology.orchestra.run.smithi121.stdout: "size": 20, 2024-04-20T01:48:05.001 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 600, 2024-04-20T01:48:05.001 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-04-20T01:48:05.001 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.001 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:285 3.8s0 3:12b7be30:::smithi16579500-11:head [read 0~180224,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:05.002 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.806166+0000", 2024-04-20T01:48:05.002 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.208392019000001, 2024-04-20T01:48:05.002 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0077061849999999999, 2024-04-20T01:48:05.002 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.002 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:05.002 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:05.002 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:05.002 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:05.002 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 285 2024-04-20T01:48:05.002 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.002 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.002 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.002 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.002 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806166+0000", 2024-04-20T01:48:05.003 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.003 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.003 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.003 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.003 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806166+0000", 2024-04-20T01:48:05.003 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.003 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.003 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.003 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.003 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806164+0000", 2024-04-20T01:48:05.003 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:05.003 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.003 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.003 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.004 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806177+0000", 2024-04-20T01:48:05.004 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2343e-05 2024-04-20T01:48:05.004 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.004 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.004 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.004 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806195+0000", 2024-04-20T01:48:05.004 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8879999999999999e-05 2024-04-20T01:48:05.004 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.004 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.004 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.004 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806207+0000", 2024-04-20T01:48:05.004 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1105e-05 2024-04-20T01:48:05.004 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.004 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.005 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.005 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806263+0000", 2024-04-20T01:48:05.005 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6357999999999998e-05 2024-04-20T01:48:05.005 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.005 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.005 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op must wait for map", 2024-04-20T01:48:05.005 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806270+0000", 2024-04-20T01:48:05.005 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.7390000000000002e-06 2024-04-20T01:48:05.005 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.005 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.005 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.005 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808279+0000", 2024-04-20T01:48:05.005 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0020090680000000001 2024-04-20T01:48:05.005 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.006 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.006 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:05.006 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808392+0000", 2024-04-20T01:48:05.006 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000113669 2024-04-20T01:48:05.006 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.006 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.006 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.006 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.813872+0000", 2024-04-20T01:48:05.006 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0054798920000000001 2024-04-20T01:48:05.006 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.006 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.006 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.006 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.006 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.006 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:286 3.8s0 3:12b7be30:::smithi16579500-11:head [read 0~180224,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:05.007 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.806216+0000", 2024-04-20T01:48:05.007 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.208342072000001, 2024-04-20T01:48:05.007 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0081155800000000007, 2024-04-20T01:48:05.007 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.007 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:05.007 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:05.007 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:05.007 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:05.007 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 286 2024-04-20T01:48:05.007 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.007 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.007 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.007 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.007 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806216+0000", 2024-04-20T01:48:05.007 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.008 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.008 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.008 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.008 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806216+0000", 2024-04-20T01:48:05.008 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.008 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.008 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.008 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.008 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806216+0000", 2024-04-20T01:48:05.008 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.008 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.008 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.008 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.008 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806219+0000", 2024-04-20T01:48:05.009 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.4860000000000002e-06 2024-04-20T01:48:05.009 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.009 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.009 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.009 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806224+0000", 2024-04-20T01:48:05.009 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.3429999999999998e-06 2024-04-20T01:48:05.009 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.009 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.009 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.009 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806227+0000", 2024-04-20T01:48:05.009 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.8e-06 2024-04-20T01:48:05.009 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.009 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.009 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.009 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806338+0000", 2024-04-20T01:48:05.010 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000110938 2024-04-20T01:48:05.010 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.010 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.010 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:05.010 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806343+0000", 2024-04-20T01:48:05.010 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.9649999999999999e-06 2024-04-20T01:48:05.010 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.010 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.010 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.010 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.810311+0000", 2024-04-20T01:48:05.010 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0039677250000000001 2024-04-20T01:48:05.010 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.010 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.010 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:05.011 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.810407+0000", 2024-04-20T01:48:05.011 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6135e-05 2024-04-20T01:48:05.011 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.011 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.011 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.011 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.814332+0000", 2024-04-20T01:48:05.011 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0039245169999999998 2024-04-20T01:48:05.011 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.011 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.011 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.011 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.011 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.011 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:287 3.8s0 3:12b7be30:::smithi16579500-11:head [read 0~180224,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:05.011 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.806248+0000", 2024-04-20T01:48:05.011 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.208310479000001, 2024-04-20T01:48:05.012 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0084388380000000006, 2024-04-20T01:48:05.012 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.012 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:05.012 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:05.012 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:05.012 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:05.012 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 287 2024-04-20T01:48:05.012 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.012 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.012 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.012 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.012 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806248+0000", 2024-04-20T01:48:05.012 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.012 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.012 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.012 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.013 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806248+0000", 2024-04-20T01:48:05.013 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.013 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.013 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.013 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.013 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806247+0000", 2024-04-20T01:48:05.013 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.013 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.013 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.013 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.013 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806251+0000", 2024-04-20T01:48:05.013 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7019999999999999e-06 2024-04-20T01:48:05.013 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.013 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.013 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.014 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806261+0000", 2024-04-20T01:48:05.014 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7839999999999998e-06 2024-04-20T01:48:05.014 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.014 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.014 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.014 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806265+0000", 2024-04-20T01:48:05.014 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.3599999999999998e-06 2024-04-20T01:48:05.014 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.014 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.014 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.014 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806370+0000", 2024-04-20T01:48:05.014 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00010491 2024-04-20T01:48:05.014 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.014 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.015 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:05.015 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806375+0000", 2024-04-20T01:48:05.015 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.8110000000000002e-06 2024-04-20T01:48:05.015 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.015 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.015 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.015 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.810963+0000", 2024-04-20T01:48:05.015 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0045881790000000004 2024-04-20T01:48:05.015 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.015 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.015 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:05.015 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.811042+0000", 2024-04-20T01:48:05.015 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.8571000000000004e-05 2024-04-20T01:48:05.015 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.015 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.015 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.016 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.814686+0000", 2024-04-20T01:48:05.016 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0036447900000000002 2024-04-20T01:48:05.016 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.016 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.016 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.016 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.016 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.016 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:307 3.8s0 3:16aac79f:::smithi16579500-14:head [copy-from ver 2] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:05.016 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.823600+0000", 2024-04-20T01:48:05.016 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.190957702999999, 2024-04-20T01:48:05.016 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.016644029000000001, 2024-04-20T01:48:05.016 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.016 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:05.016 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:05.017 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:05.017 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:05.017 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 307 2024-04-20T01:48:05.017 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.017 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.017 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.017 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.017 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823600+0000", 2024-04-20T01:48:05.017 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.017 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.017 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.017 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.017 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823600+0000", 2024-04-20T01:48:05.017 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.017 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.017 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.018 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.018 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823598+0000", 2024-04-20T01:48:05.018 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:05.018 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.018 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.018 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.018 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823606+0000", 2024-04-20T01:48:05.018 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.2629999999999997e-06 2024-04-20T01:48:05.018 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.018 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.018 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.018 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823621+0000", 2024-04-20T01:48:05.018 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5373999999999999e-05 2024-04-20T01:48:05.018 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.019 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.019 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.019 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823635+0000", 2024-04-20T01:48:05.019 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3888e-05 2024-04-20T01:48:05.019 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.019 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.019 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.019 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823683+0000", 2024-04-20T01:48:05.019 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.8455000000000003e-05 2024-04-20T01:48:05.019 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.019 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.019 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:05.019 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823792+0000", 2024-04-20T01:48:05.019 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000108156 2024-04-20T01:48:05.019 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.020 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.020 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.020 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.830048+0000", 2024-04-20T01:48:05.020 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0062565729999999996 2024-04-20T01:48:05.020 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.020 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.020 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.020 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.831286+0000", 2024-04-20T01:48:05.020 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0012377829999999999 2024-04-20T01:48:05.020 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.020 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.020 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:05.020 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.840196+0000", 2024-04-20T01:48:05.020 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0089103129999999996 2024-04-20T01:48:05.021 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.021 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.021 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.021 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.840244+0000", 2024-04-20T01:48:05.021 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.8154e-05 2024-04-20T01:48:05.021 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.021 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.021 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.021 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.021 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.021 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:308 3.8s0 3:16aac79f:::smithi16579500-14:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:05.021 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.823649+0000", 2024-04-20T01:48:05.021 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.190909544, 2024-04-20T01:48:05.021 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0073110809999999997, 2024-04-20T01:48:05.022 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.022 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:05.022 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:05.022 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:05.022 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:05.022 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 308 2024-04-20T01:48:05.022 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.022 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.022 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.022 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.022 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823649+0000", 2024-04-20T01:48:05.022 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.022 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.022 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.023 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.023 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823649+0000", 2024-04-20T01:48:05.023 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.023 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.023 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.023 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.023 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823648+0000", 2024-04-20T01:48:05.023 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.023 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.023 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.023 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.023 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823651+0000", 2024-04-20T01:48:05.023 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.948e-06 2024-04-20T01:48:05.023 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.023 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.024 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.024 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823660+0000", 2024-04-20T01:48:05.024 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3829999999999998e-06 2024-04-20T01:48:05.024 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.024 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.024 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.024 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823666+0000", 2024-04-20T01:48:05.024 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.9710000000000001e-06 2024-04-20T01:48:05.024 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.024 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.024 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.024 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823993+0000", 2024-04-20T01:48:05.024 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00032648600000000002 2024-04-20T01:48:05.024 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.024 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.025 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:05.025 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.824061+0000", 2024-04-20T01:48:05.025 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.8360000000000003e-05 2024-04-20T01:48:05.025 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.025 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.025 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.025 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.830763+0000", 2024-04-20T01:48:05.025 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0067017450000000003 2024-04-20T01:48:05.025 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.025 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.025 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:05.025 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.830849+0000", 2024-04-20T01:48:05.025 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.5649999999999995e-05 2024-04-20T01:48:05.025 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.025 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.026 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.026 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.830960+0000", 2024-04-20T01:48:05.026 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000111079 2024-04-20T01:48:05.026 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.026 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.026 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.026 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.026 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.026 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:311 3.8s0 3:1d169b1d:::smithi16579500-9:head [append 0~376832] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:05.026 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.920919+0000", 2024-04-20T01:48:05.026 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.093638624, 2024-04-20T01:48:05.026 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.009305476, 2024-04-20T01:48:05.026 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.026 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:05.027 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:05.027 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:05.027 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:05.027 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 311 2024-04-20T01:48:05.027 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.027 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.027 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.027 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.027 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.920919+0000", 2024-04-20T01:48:05.027 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.027 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.027 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.027 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.027 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.920919+0000", 2024-04-20T01:48:05.027 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.028 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.028 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.028 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.028 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.920918+0000", 2024-04-20T01:48:05.028 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:05.028 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.028 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.028 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.028 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921528+0000", 2024-04-20T01:48:05.028 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00060985100000000001 2024-04-20T01:48:05.028 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.028 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.028 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.028 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921543+0000", 2024-04-20T01:48:05.029 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4302e-05 2024-04-20T01:48:05.029 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.029 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.029 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.029 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921557+0000", 2024-04-20T01:48:05.029 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4416999999999999e-05 2024-04-20T01:48:05.029 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.029 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.029 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.029 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921979+0000", 2024-04-20T01:48:05.029 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00042189099999999998 2024-04-20T01:48:05.029 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.029 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.029 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:05.029 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.922074+0000", 2024-04-20T01:48:05.030 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4798999999999998e-05 2024-04-20T01:48:05.030 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.030 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.030 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.030 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.922634+0000", 2024-04-20T01:48:05.030 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00056005300000000003 2024-04-20T01:48:05.030 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.030 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.030 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.030 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923907+0000", 2024-04-20T01:48:05.030 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0012733239999999999 2024-04-20T01:48:05.030 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.030 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.030 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:05.031 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930149+0000", 2024-04-20T01:48:05.031 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00624202 2024-04-20T01:48:05.031 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.031 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.031 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.031 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930225+0000", 2024-04-20T01:48:05.031 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.5860999999999998e-05 2024-04-20T01:48:05.031 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.031 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.031 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.031 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.031 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.031 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:312 3.8s0 3:1d169b1d:::smithi16579500-9:head [setxattr _header (55) in=7b] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:05.031 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.921568+0000", 2024-04-20T01:48:05.031 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.092990180000001, 2024-04-20T01:48:05.032 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0088119310000000003, 2024-04-20T01:48:05.032 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.032 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:05.032 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:05.032 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:05.032 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:05.032 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 312 2024-04-20T01:48:05.032 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.032 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.032 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.032 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.032 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921568+0000", 2024-04-20T01:48:05.032 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.032 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.032 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.033 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.033 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921568+0000", 2024-04-20T01:48:05.033 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.033 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.033 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.033 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.033 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921567+0000", 2024-04-20T01:48:05.033 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.033 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.033 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.033 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.033 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921571+0000", 2024-04-20T01:48:05.033 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.6389999999999998e-06 2024-04-20T01:48:05.033 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.033 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.034 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.034 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921577+0000", 2024-04-20T01:48:05.034 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.7189999999999998e-06 2024-04-20T01:48:05.034 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.034 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.034 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.034 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921580+0000", 2024-04-20T01:48:05.034 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.5810000000000001e-06 2024-04-20T01:48:05.034 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.034 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.034 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.034 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923124+0000", 2024-04-20T01:48:05.034 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0015439360000000001 2024-04-20T01:48:05.034 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.034 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.035 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:05.035 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923191+0000", 2024-04-20T01:48:05.035 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.6507000000000005e-05 2024-04-20T01:48:05.035 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.035 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.035 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.035 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923488+0000", 2024-04-20T01:48:05.035 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00029770699999999997 2024-04-20T01:48:05.035 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.035 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.035 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.035 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.924427+0000", 2024-04-20T01:48:05.035 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000938225 2024-04-20T01:48:05.035 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.035 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.037 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:05.037 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930309+0000", 2024-04-20T01:48:05.037 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0058824990000000002 2024-04-20T01:48:05.037 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.037 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.037 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.037 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930380+0000", 2024-04-20T01:48:05.037 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0779999999999997e-05 2024-04-20T01:48:05.037 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.038 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.038 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.038 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.038 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.038 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:313 3.8s0 3:1d169b1d:::smithi16579500-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:05.038 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.921593+0000", 2024-04-20T01:48:05.038 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.092965045, 2024-04-20T01:48:05.038 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0095502929999999996, 2024-04-20T01:48:05.038 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.038 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:05.038 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:05.038 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:05.038 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:05.038 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 313 2024-04-20T01:48:05.038 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.039 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.039 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.039 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.039 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921593+0000", 2024-04-20T01:48:05.039 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.039 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.039 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.039 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.039 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921593+0000", 2024-04-20T01:48:05.039 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.039 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.039 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.039 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.039 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921593+0000", 2024-04-20T01:48:05.039 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.040 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.040 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.040 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.040 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921596+0000", 2024-04-20T01:48:05.040 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.089e-06 2024-04-20T01:48:05.040 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.040 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.040 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.040 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921601+0000", 2024-04-20T01:48:05.040 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.8799999999999999e-06 2024-04-20T01:48:05.040 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.040 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.040 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.040 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921604+0000", 2024-04-20T01:48:05.040 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.36e-06 2024-04-20T01:48:05.041 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.041 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.041 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.041 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923831+0000", 2024-04-20T01:48:05.041 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0022266479999999999 2024-04-20T01:48:05.041 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.041 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.041 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:05.041 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923898+0000", 2024-04-20T01:48:05.041 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.7391000000000006e-05 2024-04-20T01:48:05.041 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.041 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.041 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.041 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930413+0000", 2024-04-20T01:48:05.041 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.006514382 2024-04-20T01:48:05.042 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.042 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.042 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:05.042 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930468+0000", 2024-04-20T01:48:05.042 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.5923e-05 2024-04-20T01:48:05.042 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.042 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.042 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.042 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.931143+0000", 2024-04-20T01:48:05.042 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00067489999999999998 2024-04-20T01:48:05.042 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.042 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.042 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.042 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.042 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.043 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:314 3.8s0 3:1746945e:::smithi16579500-24:head [copy-from ver 4] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:05.043 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.921607+0000", 2024-04-20T01:48:05.043 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.092951450000001, 2024-04-20T01:48:05.043 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013830053, 2024-04-20T01:48:05.043 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.043 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:05.043 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:05.043 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:05.043 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:05.043 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 314 2024-04-20T01:48:05.043 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.043 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.043 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.043 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.044 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921607+0000", 2024-04-20T01:48:05.044 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.044 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.044 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.044 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.044 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921607+0000", 2024-04-20T01:48:05.044 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.044 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.044 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.044 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.044 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921606+0000", 2024-04-20T01:48:05.044 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.044 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.044 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.044 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.045 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921609+0000", 2024-04-20T01:48:05.045 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0999999999999998e-06 2024-04-20T01:48:05.045 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.045 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.045 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.045 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921612+0000", 2024-04-20T01:48:05.045 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.4089999999999999e-06 2024-04-20T01:48:05.045 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.045 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.045 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.045 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921614+0000", 2024-04-20T01:48:05.045 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5610000000000001e-06 2024-04-20T01:48:05.045 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.045 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.045 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.046 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923919+0000", 2024-04-20T01:48:05.046 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0023049809999999998 2024-04-20T01:48:05.046 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.046 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.046 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:05.046 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923973+0000", 2024-04-20T01:48:05.046 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3532999999999997e-05 2024-04-20T01:48:05.046 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.046 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.046 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.046 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.928709+0000", 2024-04-20T01:48:05.046 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0047359979999999999 2024-04-20T01:48:05.046 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.046 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.046 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.047 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930393+0000", 2024-04-20T01:48:05.047 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0016838459999999999 2024-04-20T01:48:05.047 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.047 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.047 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:05.047 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.935394+0000", 2024-04-20T01:48:05.047 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0050015779999999996 2024-04-20T01:48:05.047 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.047 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.047 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.047 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.935437+0000", 2024-04-20T01:48:05.047 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2289000000000001e-05 2024-04-20T01:48:05.047 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.047 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.047 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.048 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.048 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.048 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:315 3.8s0 3:1746945e:::smithi16579500-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:05.048 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.921619+0000", 2024-04-20T01:48:05.048 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.092939134000002, 2024-04-20T01:48:05.048 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0084014469999999994, 2024-04-20T01:48:05.048 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.048 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:05.048 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:05.048 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:05.048 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:05.048 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 315 2024-04-20T01:48:05.048 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.048 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.048 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.049 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.049 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921619+0000", 2024-04-20T01:48:05.049 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.049 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.049 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.049 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.049 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921619+0000", 2024-04-20T01:48:05.049 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.049 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.049 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.049 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.049 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921617+0000", 2024-04-20T01:48:05.049 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:05.049 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.049 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.050 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.050 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921620+0000", 2024-04-20T01:48:05.050 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.703e-06 2024-04-20T01:48:05.050 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.050 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.050 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.050 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921624+0000", 2024-04-20T01:48:05.050 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.2650000000000001e-06 2024-04-20T01:48:05.050 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.050 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.050 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.050 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921626+0000", 2024-04-20T01:48:05.050 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.666e-06 2024-04-20T01:48:05.051 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.051 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.051 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.051 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.924144+0000", 2024-04-20T01:48:05.051 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0025175000000000002 2024-04-20T01:48:05.051 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.051 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.051 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:05.051 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.924187+0000", 2024-04-20T01:48:05.051 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.3652000000000002e-05 2024-04-20T01:48:05.051 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.051 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.051 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.051 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.929879+0000", 2024-04-20T01:48:05.052 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0056911849999999996 2024-04-20T01:48:05.052 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.052 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.052 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:05.052 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.929945+0000", 2024-04-20T01:48:05.052 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.6908000000000004e-05 2024-04-20T01:48:05.052 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.052 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.052 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.052 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930020+0000", 2024-04-20T01:48:05.052 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.4988999999999996e-05 2024-04-20T01:48:05.052 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.052 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.052 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.052 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.053 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.053 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:334 3.8s0 3:1746945e:::smithi16579500-24:head [append 0~147456] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:05.053 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.972944+0000", 2024-04-20T01:48:05.053 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.041614359, 2024-04-20T01:48:05.053 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0072223890000000001, 2024-04-20T01:48:05.053 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.053 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:05.053 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:05.053 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:05.053 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:05.053 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 334 2024-04-20T01:48:05.053 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.053 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.053 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.053 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.054 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.972944+0000", 2024-04-20T01:48:05.054 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.054 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.054 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.054 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.054 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.972944+0000", 2024-04-20T01:48:05.054 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.054 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.054 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.054 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.054 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.972943+0000", 2024-04-20T01:48:05.054 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:05.054 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.054 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.055 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.055 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973105+0000", 2024-04-20T01:48:05.055 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00016233999999999999 2024-04-20T01:48:05.055 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.055 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.055 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.055 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973113+0000", 2024-04-20T01:48:05.055 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.1100000000000003e-06 2024-04-20T01:48:05.055 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.055 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.055 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.055 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973118+0000", 2024-04-20T01:48:05.055 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.3490000000000002e-06 2024-04-20T01:48:05.055 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.055 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.056 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.056 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.974308+0000", 2024-04-20T01:48:05.056 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0011903759999999999 2024-04-20T01:48:05.056 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.056 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.056 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:05.056 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.974373+0000", 2024-04-20T01:48:05.056 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.5295000000000004e-05 2024-04-20T01:48:05.056 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.056 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.056 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.056 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.974771+0000", 2024-04-20T01:48:05.056 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00039771199999999999 2024-04-20T01:48:05.056 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.056 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.057 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.057 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.980089+0000", 2024-04-20T01:48:05.057 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0053181290000000004 2024-04-20T01:48:05.057 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.057 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.057 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:05.057 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.980127+0000", 2024-04-20T01:48:05.057 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7768000000000002e-05 2024-04-20T01:48:05.057 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.057 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.057 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.057 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.980166+0000", 2024-04-20T01:48:05.057 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.9131999999999997e-05 2024-04-20T01:48:05.057 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.057 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.058 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.058 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.058 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.058 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:335 3.8s0 3:1746945e:::smithi16579500-24:head [setxattr _header (56) in=7b] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:05.058 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.973122+0000", 2024-04-20T01:48:05.058 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.041436339000001, 2024-04-20T01:48:05.058 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0075025550000000002, 2024-04-20T01:48:05.058 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.058 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:05.058 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:05.058 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:05.058 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:05.058 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 335 2024-04-20T01:48:05.058 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.058 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.059 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.059 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.059 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973122+0000", 2024-04-20T01:48:05.059 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.059 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.059 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.059 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.059 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973122+0000", 2024-04-20T01:48:05.059 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.059 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.059 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.059 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.059 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973121+0000", 2024-04-20T01:48:05.059 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.059 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.060 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.060 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.060 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973124+0000", 2024-04-20T01:48:05.060 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3089999999999998e-06 2024-04-20T01:48:05.060 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.060 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.060 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.060 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973128+0000", 2024-04-20T01:48:05.060 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.3109999999999999e-06 2024-04-20T01:48:05.060 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.060 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.060 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.060 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973131+0000", 2024-04-20T01:48:05.060 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.791e-06 2024-04-20T01:48:05.060 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.061 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.061 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.061 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.977166+0000", 2024-04-20T01:48:05.061 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.004034948 2024-04-20T01:48:05.061 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.061 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.061 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:05.061 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.977220+0000", 2024-04-20T01:48:05.061 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.4561999999999997e-05 2024-04-20T01:48:05.061 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.061 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.061 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.061 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.977510+0000", 2024-04-20T01:48:05.061 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000289869 2024-04-20T01:48:05.061 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.062 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.062 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.062 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.980331+0000", 2024-04-20T01:48:05.062 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0028210869999999999 2024-04-20T01:48:05.062 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.062 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.062 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:05.062 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.980365+0000", 2024-04-20T01:48:05.062 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.4020000000000003e-05 2024-04-20T01:48:05.062 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.062 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.062 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.062 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.980624+0000", 2024-04-20T01:48:05.062 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000259017 2024-04-20T01:48:05.063 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.063 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.063 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.063 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.063 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.063 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:336 3.8s0 3:1746945e:::smithi16579500-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:05.063 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.973134+0000", 2024-04-20T01:48:05.063 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.041424482, 2024-04-20T01:48:05.063 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010536856000000001, 2024-04-20T01:48:05.063 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.063 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:05.063 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:05.063 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:05.063 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:05.064 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 336 2024-04-20T01:48:05.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.064 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973134+0000", 2024-04-20T01:48:05.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973134+0000", 2024-04-20T01:48:05.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973133+0000", 2024-04-20T01:48:05.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973135+0000", 2024-04-20T01:48:05.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5269999999999999e-06 2024-04-20T01:48:05.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973139+0000", 2024-04-20T01:48:05.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.3749999999999996e-06 2024-04-20T01:48:05.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.066 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973142+0000", 2024-04-20T01:48:05.066 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7800000000000001e-06 2024-04-20T01:48:05.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.066 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.066 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.066 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.978050+0000", 2024-04-20T01:48:05.066 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0049085049999999996 2024-04-20T01:48:05.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.066 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.066 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:05.066 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.978106+0000", 2024-04-20T01:48:05.066 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.5648e-05 2024-04-20T01:48:05.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.066 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.066 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.067 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.980649+0000", 2024-04-20T01:48:05.067 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0025426630000000001 2024-04-20T01:48:05.067 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.067 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.067 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:05.067 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.980701+0000", 2024-04-20T01:48:05.067 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.1907999999999998e-05 2024-04-20T01:48:05.067 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.067 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.067 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.067 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.983670+0000", 2024-04-20T01:48:05.067 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0029697590000000002 2024-04-20T01:48:05.067 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.067 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.067 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.068 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.068 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.1s3 48/30 ECSubWrite(tid=119, reqid=osd.4.0:119, at_version=48'14, trim_to=0'0, roll_forward_to=47'12))", 2024-04-20T01:48:05.068 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.718478+0000", 2024-04-20T01:48:05.068 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 15.296080551999999, 2024-04-20T01:48:05.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0075494899999999998, 2024-04-20T01:48:05.068 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.068 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.068 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.068 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.068 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.068 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718478+0000", 2024-04-20T01:48:05.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.068 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718478+0000", 2024-04-20T01:48:05.069 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.069 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718477+0000", 2024-04-20T01:48:05.069 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.069 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718498+0000", 2024-04-20T01:48:05.069 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0271999999999998e-05 2024-04-20T01:48:05.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.069 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718508+0000", 2024-04-20T01:48:05.070 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0322999999999999e-05 2024-04-20T01:48:05.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718517+0000", 2024-04-20T01:48:05.070 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8680000000000005e-06 2024-04-20T01:48:05.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718568+0000", 2024-04-20T01:48:05.070 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.1119999999999998e-05 2024-04-20T01:48:05.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.071 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.071 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718580+0000", 2024-04-20T01:48:05.071 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1945e-05 2024-04-20T01:48:05.071 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.071 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.071 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.071 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.726011+0000", 2024-04-20T01:48:05.071 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0074307189999999997 2024-04-20T01:48:05.071 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.071 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.071 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.071 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.726027+0000", 2024-04-20T01:48:05.071 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6481999999999999e-05 2024-04-20T01:48:05.071 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.071 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.072 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.072 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.072 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.072 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s4 53/30 ECSubWrite(tid=34, reqid=client.4643.0:445, at_version=53'23, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:05.072 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:54.722907+0000", 2024-04-20T01:48:05.072 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 10.291651168, 2024-04-20T01:48:05.072 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0084068279999999999, 2024-04-20T01:48:05.072 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.072 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.072 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.072 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.072 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.072 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.722907+0000", 2024-04-20T01:48:05.072 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.072 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.073 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.073 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.722907+0000", 2024-04-20T01:48:05.073 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.073 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.073 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.073 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.722896+0000", 2024-04-20T01:48:05.073 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999895 2024-04-20T01:48:05.073 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.073 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.073 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723127+0000", 2024-04-20T01:48:05.073 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00023036199999999999 2024-04-20T01:48:05.073 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.074 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723141+0000", 2024-04-20T01:48:05.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4174e-05 2024-04-20T01:48:05.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.074 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723150+0000", 2024-04-20T01:48:05.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3090000000000001e-06 2024-04-20T01:48:05.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.074 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723203+0000", 2024-04-20T01:48:05.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.2431e-05 2024-04-20T01:48:05.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.075 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723212+0000", 2024-04-20T01:48:05.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3239999999999992e-06 2024-04-20T01:48:05.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.075 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.731289+0000", 2024-04-20T01:48:05.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0080767059999999995 2024-04-20T01:48:05.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.075 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.731314+0000", 2024-04-20T01:48:05.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5088000000000001e-05 2024-04-20T01:48:05.075 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.075 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.076 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.076 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.076 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.076 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s4 53/30 ECSubWrite(tid=35, reqid=client.4643.0:446, at_version=53'24, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:05.076 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:54.723916+0000", 2024-04-20T01:48:05.076 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 10.29064236, 2024-04-20T01:48:05.076 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0074288590000000003, 2024-04-20T01:48:05.076 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.076 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.076 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.076 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.076 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.076 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723916+0000", 2024-04-20T01:48:05.076 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.076 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.077 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.077 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723916+0000", 2024-04-20T01:48:05.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.077 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.077 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723915+0000", 2024-04-20T01:48:05.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:05.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.077 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.077 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723931+0000", 2024-04-20T01:48:05.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6379999999999999e-05 2024-04-20T01:48:05.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.078 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.078 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723943+0000", 2024-04-20T01:48:05.078 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1904e-05 2024-04-20T01:48:05.078 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.078 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.078 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.078 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723951+0000", 2024-04-20T01:48:05.078 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.4420000000000001e-06 2024-04-20T01:48:05.078 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.078 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.078 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.078 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.725888+0000", 2024-04-20T01:48:05.078 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0019375760000000001 2024-04-20T01:48:05.078 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.078 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.079 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.725901+0000", 2024-04-20T01:48:05.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2855e-05 2024-04-20T01:48:05.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.079 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.731335+0000", 2024-04-20T01:48:05.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0054342440000000004 2024-04-20T01:48:05.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.079 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.731345+0000", 2024-04-20T01:48:05.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2949999999999999e-06 2024-04-20T01:48:05.079 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.079 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.080 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.080 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:487 3.8s0 3:16aac79f:::smithi16579500-14:head [append 0~425984] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:05.080 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:59.760614+0000", 2024-04-20T01:48:05.080 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.2539444660000001, 2024-04-20T01:48:05.080 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010465621, 2024-04-20T01:48:05.080 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.080 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:05.080 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:05.080 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:05.080 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:05.080 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 487 2024-04-20T01:48:05.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.080 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.081 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.081 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.760614+0000", 2024-04-20T01:48:05.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.081 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.081 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.760614+0000", 2024-04-20T01:48:05.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.081 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.081 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.760611+0000", 2024-04-20T01:48:05.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999971 2024-04-20T01:48:05.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.082 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.082 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.082 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.761151+0000", 2024-04-20T01:48:05.082 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00054006400000000004 2024-04-20T01:48:05.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.082 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.082 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.082 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.761173+0000", 2024-04-20T01:48:05.082 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1926999999999999e-05 2024-04-20T01:48:05.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.082 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.082 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.082 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.761202+0000", 2024-04-20T01:48:05.082 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9286e-05 2024-04-20T01:48:05.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.083 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.761276+0000", 2024-04-20T01:48:05.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.4319000000000005e-05 2024-04-20T01:48:05.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.083 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op must wait for map", 2024-04-20T01:48:05.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.761289+0000", 2024-04-20T01:48:05.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.239e-05 2024-04-20T01:48:05.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.083 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.763247+0000", 2024-04-20T01:48:05.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0019584989999999998 2024-04-20T01:48:05.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:05.084 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.763331+0000", 2024-04-20T01:48:05.084 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.3320999999999997e-05 2024-04-20T01:48:05.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.084 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.764297+0000", 2024-04-20T01:48:05.084 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00096590700000000001 2024-04-20T01:48:05.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.084 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768005+0000", 2024-04-20T01:48:05.084 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0037084940000000001 2024-04-20T01:48:05.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.085 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:05.085 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.770983+0000", 2024-04-20T01:48:05.085 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0029780129999999998 2024-04-20T01:48:05.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.085 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.085 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.085 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.771079+0000", 2024-04-20T01:48:05.085 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6250999999999997e-05 2024-04-20T01:48:05.085 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.085 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.085 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.085 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.085 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:489 3.8s0 3:16aac79f:::smithi16579500-14:head [append 0~376832] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:05.086 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:59.767481+0000", 2024-04-20T01:48:05.086 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.2470767949999999, 2024-04-20T01:48:05.086 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011576091, 2024-04-20T01:48:05.086 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.086 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:05.086 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:05.086 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:05.086 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:05.086 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 489 2024-04-20T01:48:05.086 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.086 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.086 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.086 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.086 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.767481+0000", 2024-04-20T01:48:05.086 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.087 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.087 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.087 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.767481+0000", 2024-04-20T01:48:05.087 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.087 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.087 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.087 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.767480+0000", 2024-04-20T01:48:05.087 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:05.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.087 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.087 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.087 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768144+0000", 2024-04-20T01:48:05.087 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00066418499999999997 2024-04-20T01:48:05.088 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.088 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.088 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.088 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768157+0000", 2024-04-20T01:48:05.088 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2281e-05 2024-04-20T01:48:05.088 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.088 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.088 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.088 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768166+0000", 2024-04-20T01:48:05.088 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2629999999999992e-06 2024-04-20T01:48:05.088 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.088 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.088 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.088 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768215+0000", 2024-04-20T01:48:05.088 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.9292000000000002e-05 2024-04-20T01:48:05.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.089 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.089 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:05.089 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768322+0000", 2024-04-20T01:48:05.089 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000106461 2024-04-20T01:48:05.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.089 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.089 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.089 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.769186+0000", 2024-04-20T01:48:05.089 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00086420900000000001 2024-04-20T01:48:05.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.089 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.089 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.089 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.771140+0000", 2024-04-20T01:48:05.089 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0019545890000000001 2024-04-20T01:48:05.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.090 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.090 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:05.090 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779011+0000", 2024-04-20T01:48:05.090 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0078710150000000003 2024-04-20T01:48:05.090 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.090 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.090 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.090 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779057+0000", 2024-04-20T01:48:05.090 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5918000000000003e-05 2024-04-20T01:48:05.090 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.090 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.090 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.090 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.090 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.090 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:490 3.8s0 3:16aac79f:::smithi16579500-14:head [setxattr _header (57) in=7b] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:05.091 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:59.768179+0000", 2024-04-20T01:48:05.091 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.246378902, 2024-04-20T01:48:05.091 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011076167, 2024-04-20T01:48:05.091 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.091 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:05.091 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:05.091 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:05.091 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:05.091 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 490 2024-04-20T01:48:05.091 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.091 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.091 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.091 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.091 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768179+0000", 2024-04-20T01:48:05.091 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.092 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.092 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.092 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768179+0000", 2024-04-20T01:48:05.092 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.092 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.092 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.092 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768178+0000", 2024-04-20T01:48:05.092 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:05.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.092 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.092 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.092 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768183+0000", 2024-04-20T01:48:05.092 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.288e-06 2024-04-20T01:48:05.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.093 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.093 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.093 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768191+0000", 2024-04-20T01:48:05.093 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.5850000000000002e-06 2024-04-20T01:48:05.093 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.093 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.093 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.093 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768197+0000", 2024-04-20T01:48:05.093 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.9270000000000002e-06 2024-04-20T01:48:05.093 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.093 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.093 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.093 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.769809+0000", 2024-04-20T01:48:05.093 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001611913 2024-04-20T01:48:05.093 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.094 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.094 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:05.094 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.769930+0000", 2024-04-20T01:48:05.094 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000121304 2024-04-20T01:48:05.094 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.094 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.094 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.094 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.770462+0000", 2024-04-20T01:48:05.094 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00053196700000000005 2024-04-20T01:48:05.094 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.094 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.094 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.094 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.771310+0000", 2024-04-20T01:48:05.094 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00084813200000000003 2024-04-20T01:48:05.095 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.095 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.095 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:05.095 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779156+0000", 2024-04-20T01:48:05.095 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0078461580000000006 2024-04-20T01:48:05.095 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.095 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.095 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.095 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779255+0000", 2024-04-20T01:48:05.095 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.8944999999999995e-05 2024-04-20T01:48:05.095 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.095 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.095 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.095 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.095 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.095 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:491 3.8s0 3:16aac79f:::smithi16579500-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:05.096 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:59.768204+0000", 2024-04-20T01:48:05.096 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.2463538129999998, 2024-04-20T01:48:05.096 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.012786562, 2024-04-20T01:48:05.096 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.096 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:05.096 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:05.096 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:05.096 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:05.096 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 491 2024-04-20T01:48:05.096 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.096 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.096 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.096 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.096 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768204+0000", 2024-04-20T01:48:05.096 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.097 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.097 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.097 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.097 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768204+0000", 2024-04-20T01:48:05.097 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.097 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.097 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.097 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.097 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768204+0000", 2024-04-20T01:48:05.097 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.097 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.097 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.097 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.097 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768207+0000", 2024-04-20T01:48:05.097 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.4180000000000001e-06 2024-04-20T01:48:05.097 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.098 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.098 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.098 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768215+0000", 2024-04-20T01:48:05.098 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.6669999999999996e-06 2024-04-20T01:48:05.098 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.098 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.098 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.098 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768220+0000", 2024-04-20T01:48:05.098 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3149999999999999e-06 2024-04-20T01:48:05.098 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.098 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.098 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.098 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.771203+0000", 2024-04-20T01:48:05.098 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.002982434 2024-04-20T01:48:05.098 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.099 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.099 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:05.099 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.771299+0000", 2024-04-20T01:48:05.099 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6111000000000002e-05 2024-04-20T01:48:05.099 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.099 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.099 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.099 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779629+0000", 2024-04-20T01:48:05.099 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0083302529999999993 2024-04-20T01:48:05.099 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.099 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.099 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:05.099 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779693+0000", 2024-04-20T01:48:05.099 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.4424999999999994e-05 2024-04-20T01:48:05.099 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.100 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.100 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.100 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.780991+0000", 2024-04-20T01:48:05.100 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0012974390000000001 2024-04-20T01:48:05.100 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.100 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.100 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.100 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.100 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.100 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:05.101 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:05.116+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:05.102 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-20T01:48:05.203 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:05.217+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:05.230 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:05.230 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:05.230 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 0 2024-04-20T01:48:05.230 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:05.240 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-20T01:48:05.304 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:05.318+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:05.368 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:05.369 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:05.369 INFO:teuthology.orchestra.run.smithi121.stdout: "complaint_time": 30, 2024-04-20T01:48:05.369 INFO:teuthology.orchestra.run.smithi121.stdout: "num_blocked_ops": 0 2024-04-20T01:48:05.369 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:05.379 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-20T01:48:05.404 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:05.419+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:05.505 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:05.520+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:05.508 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:05.508 INFO:teuthology.orchestra.run.smithi121.stdout: "size": 20, 2024-04-20T01:48:05.508 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 600, 2024-04-20T01:48:05.509 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-04-20T01:48:05.509 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.509 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.6s6 33/30 ECSubWrite(tid=0, reqid=client.4643.0:22, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:48:05.509 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.759301+0000", 2024-04-20T01:48:05.509 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 30.762813478999998, 2024-04-20T01:48:05.509 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0067130840000000002, 2024-04-20T01:48:05.509 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.509 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.509 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.509 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.509 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.509 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.759301+0000", 2024-04-20T01:48:05.509 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.510 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.510 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.510 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.510 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.759301+0000", 2024-04-20T01:48:05.510 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.510 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.510 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.510 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.510 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.759300+0000", 2024-04-20T01:48:05.510 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.510 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.510 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.510 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.510 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.759499+0000", 2024-04-20T01:48:05.511 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000198471 2024-04-20T01:48:05.511 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.511 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.511 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.511 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.759538+0000", 2024-04-20T01:48:05.511 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.9181999999999999e-05 2024-04-20T01:48:05.511 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.511 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.511 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.511 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.759554+0000", 2024-04-20T01:48:05.511 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.592e-05 2024-04-20T01:48:05.511 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.511 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.511 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.511 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.759728+0000", 2024-04-20T01:48:05.512 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00017346399999999999 2024-04-20T01:48:05.512 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.512 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.512 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.512 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.759745+0000", 2024-04-20T01:48:05.512 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7442999999999999e-05 2024-04-20T01:48:05.512 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.512 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.512 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.512 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.765987+0000", 2024-04-20T01:48:05.512 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0062419279999999999 2024-04-20T01:48:05.512 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.513 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.513 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.513 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766014+0000", 2024-04-20T01:48:05.513 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6971e-05 2024-04-20T01:48:05.513 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.513 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.513 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.513 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.513 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.513 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 33/30 ECSubWrite(tid=0, reqid=client.4643.0:37, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:48:05.513 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.765612+0000", 2024-04-20T01:48:05.513 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 30.756501791000002, 2024-04-20T01:48:05.513 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.006884584, 2024-04-20T01:48:05.513 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.514 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.514 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.514 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.514 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.514 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.765612+0000", 2024-04-20T01:48:05.514 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.514 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.514 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.514 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.514 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.765612+0000", 2024-04-20T01:48:05.514 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.514 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.514 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.514 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.515 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.765612+0000", 2024-04-20T01:48:05.515 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.515 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.515 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.515 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.515 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766107+0000", 2024-04-20T01:48:05.515 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00049509399999999996 2024-04-20T01:48:05.515 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.515 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.516 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.516 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766119+0000", 2024-04-20T01:48:05.516 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.168e-05 2024-04-20T01:48:05.516 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.516 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.516 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.516 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766128+0000", 2024-04-20T01:48:05.516 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.1169999999999993e-06 2024-04-20T01:48:05.516 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.516 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.516 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.517 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766186+0000", 2024-04-20T01:48:05.517 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.8236000000000003e-05 2024-04-20T01:48:05.517 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.517 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.517 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.517 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766196+0000", 2024-04-20T01:48:05.517 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3680000000000008e-06 2024-04-20T01:48:05.517 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.517 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.517 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.517 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.772468+0000", 2024-04-20T01:48:05.517 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0062727529999999998 2024-04-20T01:48:05.517 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.517 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.518 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.518 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.772497+0000", 2024-04-20T01:48:05.518 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8759999999999999e-05 2024-04-20T01:48:05.518 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.518 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.518 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.518 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.518 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.518 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 33/30 ECSubWrite(tid=1, reqid=client.4643.0:38, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:48:05.518 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.767018+0000", 2024-04-20T01:48:05.518 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 30.755096743999999, 2024-04-20T01:48:05.518 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0055026670000000001, 2024-04-20T01:48:05.518 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.518 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.518 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.519 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.519 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.519 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.767018+0000", 2024-04-20T01:48:05.519 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.519 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.519 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.519 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.519 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.767018+0000", 2024-04-20T01:48:05.519 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.519 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.519 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.519 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.519 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.767017+0000", 2024-04-20T01:48:05.520 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:05.520 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.520 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.520 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.520 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.767046+0000", 2024-04-20T01:48:05.520 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9116999999999999e-05 2024-04-20T01:48:05.520 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.520 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.520 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.520 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.767065+0000", 2024-04-20T01:48:05.520 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8819000000000001e-05 2024-04-20T01:48:05.520 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.520 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.520 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.521 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.767080+0000", 2024-04-20T01:48:05.521 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5055000000000001e-05 2024-04-20T01:48:05.521 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.521 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.521 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.521 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.767136+0000", 2024-04-20T01:48:05.521 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6174000000000001e-05 2024-04-20T01:48:05.521 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.521 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.521 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.521 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.767146+0000", 2024-04-20T01:48:05.521 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0003999999999999e-05 2024-04-20T01:48:05.521 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.521 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.521 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.522 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.772506+0000", 2024-04-20T01:48:05.522 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0053601109999999999 2024-04-20T01:48:05.522 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.522 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.522 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.522 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.772520+0000", 2024-04-20T01:48:05.522 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4250000000000001e-05 2024-04-20T01:48:05.522 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.522 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.522 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.522 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.523 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.523 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 37/30 ECSubWrite(tid=64, reqid=client.4643.0:225, at_version=37'16, trim_to=0'0, roll_forward_to=36'14))", 2024-04-20T01:48:05.523 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.791284+0000", 2024-04-20T01:48:05.523 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 26.730830449999999, 2024-04-20T01:48:05.523 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0047182170000000002, 2024-04-20T01:48:05.523 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.523 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.523 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.523 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.523 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.523 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791284+0000", 2024-04-20T01:48:05.523 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.524 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.524 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.524 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.524 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791284+0000", 2024-04-20T01:48:05.524 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.524 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.524 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.524 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.524 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791284+0000", 2024-04-20T01:48:05.524 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:48:05.524 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.524 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.524 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.524 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791637+0000", 2024-04-20T01:48:05.525 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000353075 2024-04-20T01:48:05.525 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.525 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.525 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.525 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791652+0000", 2024-04-20T01:48:05.525 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5625e-05 2024-04-20T01:48:05.525 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.525 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.525 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.525 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791665+0000", 2024-04-20T01:48:05.525 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2814e-05 2024-04-20T01:48:05.526 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.526 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.526 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.526 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791738+0000", 2024-04-20T01:48:05.526 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.2700999999999994e-05 2024-04-20T01:48:05.526 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.526 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.526 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.526 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791748+0000", 2024-04-20T01:48:05.526 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.9899999999999992e-06 2024-04-20T01:48:05.526 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.526 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.527 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.527 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.795975+0000", 2024-04-20T01:48:05.527 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0042274510000000001 2024-04-20T01:48:05.527 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.527 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.527 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.527 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796002+0000", 2024-04-20T01:48:05.527 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6766000000000001e-05 2024-04-20T01:48:05.527 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.527 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.527 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.527 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.527 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.527 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 37/30 ECSubWrite(tid=65, reqid=client.4643.0:226, at_version=37'17, trim_to=0'0, roll_forward_to=36'14))", 2024-04-20T01:48:05.528 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.791683+0000", 2024-04-20T01:48:05.528 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 26.730431115000002, 2024-04-20T01:48:05.528 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0043386420000000002, 2024-04-20T01:48:05.528 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.528 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.529 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.529 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.529 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.529 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791683+0000", 2024-04-20T01:48:05.530 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.530 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.530 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.530 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.530 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791683+0000", 2024-04-20T01:48:05.530 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.531 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.531 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.531 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.531 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791683+0000", 2024-04-20T01:48:05.531 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.531 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.531 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.531 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.531 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791807+0000", 2024-04-20T01:48:05.532 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00012426399999999999 2024-04-20T01:48:05.532 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.532 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.532 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.532 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791817+0000", 2024-04-20T01:48:05.532 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0079e-05 2024-04-20T01:48:05.532 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.532 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.532 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.532 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791823+0000", 2024-04-20T01:48:05.532 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.3609999999999997e-06 2024-04-20T01:48:05.532 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.532 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.532 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.532 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793245+0000", 2024-04-20T01:48:05.533 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0014213730000000001 2024-04-20T01:48:05.533 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.533 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.533 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.533 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793255+0000", 2024-04-20T01:48:05.534 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0278999999999999e-05 2024-04-20T01:48:05.534 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.534 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.534 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.534 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796010+0000", 2024-04-20T01:48:05.534 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0027550259999999998 2024-04-20T01:48:05.534 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.534 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.534 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.534 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796022+0000", 2024-04-20T01:48:05.535 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1754e-05 2024-04-20T01:48:05.535 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.535 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.535 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.535 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.535 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.535 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 37/30 ECSubWrite(tid=66, reqid=client.4643.0:227, at_version=37'18, trim_to=0'0, roll_forward_to=36'14))", 2024-04-20T01:48:05.535 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.791832+0000", 2024-04-20T01:48:05.535 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 26.730281956999999, 2024-04-20T01:48:05.535 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0042145630000000002, 2024-04-20T01:48:05.535 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.535 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.535 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.535 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.536 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.536 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791832+0000", 2024-04-20T01:48:05.536 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.536 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.536 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.536 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.536 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791832+0000", 2024-04-20T01:48:05.536 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.536 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.536 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.536 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.536 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791832+0000", 2024-04-20T01:48:05.536 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.536 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.536 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.537 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.537 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791856+0000", 2024-04-20T01:48:05.537 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4837999999999998e-05 2024-04-20T01:48:05.537 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.537 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.537 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.537 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791866+0000", 2024-04-20T01:48:05.537 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.0110000000000006e-06 2024-04-20T01:48:05.537 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.537 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.537 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.537 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.791890+0000", 2024-04-20T01:48:05.537 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4936999999999999e-05 2024-04-20T01:48:05.537 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.537 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.538 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.538 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793767+0000", 2024-04-20T01:48:05.538 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001876827 2024-04-20T01:48:05.538 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.538 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.538 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.538 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.793779+0000", 2024-04-20T01:48:05.538 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1620000000000001e-05 2024-04-20T01:48:05.538 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.538 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.538 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.538 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796029+0000", 2024-04-20T01:48:05.538 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00225012 2024-04-20T01:48:05.538 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.539 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.539 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.539 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.796047+0000", 2024-04-20T01:48:05.539 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7844000000000001e-05 2024-04-20T01:48:05.539 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.539 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.539 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.539 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.539 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.539 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 38/30 ECSubWrite(tid=73, reqid=osd.4.0:73, at_version=38'22, trim_to=0'0, roll_forward_to=37'18))", 2024-04-20T01:48:05.539 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:39.792847+0000", 2024-04-20T01:48:05.539 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 25.729267710999999, 2024-04-20T01:48:05.539 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0061022330000000003, 2024-04-20T01:48:05.539 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.540 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.540 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.540 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.540 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.540 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.792847+0000", 2024-04-20T01:48:05.540 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.540 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.540 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.540 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.540 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.792847+0000", 2024-04-20T01:48:05.541 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.541 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.541 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.541 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.541 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.792846+0000", 2024-04-20T01:48:05.541 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.541 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.541 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.541 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.541 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.792867+0000", 2024-04-20T01:48:05.541 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1022999999999998e-05 2024-04-20T01:48:05.541 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.541 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.541 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.542 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.792879+0000", 2024-04-20T01:48:05.542 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2242000000000001e-05 2024-04-20T01:48:05.542 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.542 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.542 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.542 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.792894+0000", 2024-04-20T01:48:05.542 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4515000000000001e-05 2024-04-20T01:48:05.542 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.542 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.542 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.542 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.792940+0000", 2024-04-20T01:48:05.542 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.6592999999999999e-05 2024-04-20T01:48:05.542 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.542 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.543 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.543 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.792956+0000", 2024-04-20T01:48:05.543 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5962999999999999e-05 2024-04-20T01:48:05.543 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.543 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.543 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.543 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.798920+0000", 2024-04-20T01:48:05.543 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0059632720000000004 2024-04-20T01:48:05.543 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.543 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.543 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.543 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.798949+0000", 2024-04-20T01:48:05.544 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9196999999999998e-05 2024-04-20T01:48:05.544 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.544 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.544 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.544 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.544 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.544 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 38/30 ECSubWrite(tid=74, reqid=osd.4.0:73, at_version=0'0, trim_to=0'0, roll_forward_to=38'22))", 2024-04-20T01:48:05.544 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:39.799475+0000", 2024-04-20T01:48:05.544 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 25.722639320999999, 2024-04-20T01:48:05.544 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.004862295, 2024-04-20T01:48:05.544 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.545 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.545 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.545 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.545 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.545 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.799475+0000", 2024-04-20T01:48:05.545 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.545 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.545 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.545 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.545 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.799475+0000", 2024-04-20T01:48:05.545 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.545 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.545 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.545 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.546 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.799474+0000", 2024-04-20T01:48:05.546 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.546 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.546 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.546 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.546 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.799483+0000", 2024-04-20T01:48:05.546 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.1619999999999997e-06 2024-04-20T01:48:05.546 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.546 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.546 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.546 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.799498+0000", 2024-04-20T01:48:05.546 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4168999999999999e-05 2024-04-20T01:48:05.546 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.546 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.547 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.547 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.799507+0000", 2024-04-20T01:48:05.547 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2499999999999995e-06 2024-04-20T01:48:05.547 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.547 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.547 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.547 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.799544+0000", 2024-04-20T01:48:05.547 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7020999999999997e-05 2024-04-20T01:48:05.547 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.547 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.547 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.547 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.799553+0000", 2024-04-20T01:48:05.547 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2089999999999994e-06 2024-04-20T01:48:05.547 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.548 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.548 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.548 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.804307+0000", 2024-04-20T01:48:05.548 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0047538260000000001 2024-04-20T01:48:05.548 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.548 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.548 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.548 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:39.804337+0000", 2024-04-20T01:48:05.548 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0272000000000001e-05 2024-04-20T01:48:05.549 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.549 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.549 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.549 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.549 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.549 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 42/30 ECSubWrite(tid=92, reqid=client.4643.0:297, at_version=42'33, trim_to=0'0, roll_forward_to=41'31))", 2024-04-20T01:48:05.549 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.810386+0000", 2024-04-20T01:48:05.549 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.711727943, 2024-04-20T01:48:05.549 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0049351079999999997, 2024-04-20T01:48:05.549 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.549 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.550 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.550 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.550 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.550 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.810386+0000", 2024-04-20T01:48:05.550 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.550 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.550 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.550 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.551 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.810386+0000", 2024-04-20T01:48:05.551 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.551 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.551 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.551 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.551 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.810385+0000", 2024-04-20T01:48:05.551 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:05.551 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.551 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.551 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.551 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.810471+0000", 2024-04-20T01:48:05.551 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.5270000000000002e-05 2024-04-20T01:48:05.552 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.552 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.552 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.552 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.810497+0000", 2024-04-20T01:48:05.552 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6508000000000001e-05 2024-04-20T01:48:05.552 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.552 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.552 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.552 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.810507+0000", 2024-04-20T01:48:05.552 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0399e-05 2024-04-20T01:48:05.552 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.553 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.553 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.553 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.811310+0000", 2024-04-20T01:48:05.553 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00080240200000000002 2024-04-20T01:48:05.553 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.553 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.553 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.553 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.811320+0000", 2024-04-20T01:48:05.553 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0536999999999999e-05 2024-04-20T01:48:05.553 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.553 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.553 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.553 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.815282+0000", 2024-04-20T01:48:05.553 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0039620690000000004 2024-04-20T01:48:05.553 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.554 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.554 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.554 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.815321+0000", 2024-04-20T01:48:05.554 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.8971999999999999e-05 2024-04-20T01:48:05.554 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.555 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.555 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.555 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.555 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.555 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 47/30 ECSubWrite(tid=110, reqid=osd.4.0:110, at_version=47'47, trim_to=0'0, roll_forward_to=44'43))", 2024-04-20T01:48:05.555 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.708619+0000", 2024-04-20T01:48:05.555 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 16.813495750000001, 2024-04-20T01:48:05.555 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.003833787, 2024-04-20T01:48:05.555 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.555 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.555 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.555 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.556 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.556 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.708619+0000", 2024-04-20T01:48:05.556 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.556 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.556 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.556 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.556 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.708619+0000", 2024-04-20T01:48:05.556 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.556 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.556 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.557 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.557 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.708618+0000", 2024-04-20T01:48:05.557 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:05.557 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.557 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.557 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.557 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.708635+0000", 2024-04-20T01:48:05.557 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7050000000000001e-05 2024-04-20T01:48:05.557 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.557 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.557 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.557 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.708645+0000", 2024-04-20T01:48:05.557 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0516000000000001e-05 2024-04-20T01:48:05.558 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.558 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.558 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.558 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.708653+0000", 2024-04-20T01:48:05.558 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.3309999999999998e-06 2024-04-20T01:48:05.558 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.558 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.558 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.558 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.708692+0000", 2024-04-20T01:48:05.558 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.9539000000000002e-05 2024-04-20T01:48:05.558 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.558 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.558 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.558 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.708704+0000", 2024-04-20T01:48:05.559 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.145e-05 2024-04-20T01:48:05.559 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.559 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.559 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.559 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.712427+0000", 2024-04-20T01:48:05.560 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0037235789999999999 2024-04-20T01:48:05.560 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.560 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.560 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.560 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.712452+0000", 2024-04-20T01:48:05.560 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5091000000000001e-05 2024-04-20T01:48:05.560 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.560 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.560 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.560 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.560 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.561 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 47/30 ECSubWrite(tid=111, reqid=osd.4.0:110, at_version=0'0, trim_to=0'0, roll_forward_to=47'47))", 2024-04-20T01:48:05.561 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.713446+0000", 2024-04-20T01:48:05.561 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 16.808668487999999, 2024-04-20T01:48:05.561 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0037974179999999999, 2024-04-20T01:48:05.561 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.561 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.561 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.561 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.561 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.561 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.713446+0000", 2024-04-20T01:48:05.561 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.561 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.561 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.561 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.562 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.713446+0000", 2024-04-20T01:48:05.562 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.562 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.562 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.562 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.562 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.713445+0000", 2024-04-20T01:48:05.562 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:05.562 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.562 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.562 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.562 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.713456+0000", 2024-04-20T01:48:05.562 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1705999999999999e-05 2024-04-20T01:48:05.562 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.563 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.563 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.563 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.713467+0000", 2024-04-20T01:48:05.563 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0471e-05 2024-04-20T01:48:05.563 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.563 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.563 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.563 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.713473+0000", 2024-04-20T01:48:05.563 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.1510000000000001e-06 2024-04-20T01:48:05.563 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.563 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.563 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.563 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.713512+0000", 2024-04-20T01:48:05.563 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.8642e-05 2024-04-20T01:48:05.563 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.564 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.564 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.564 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.713520+0000", 2024-04-20T01:48:05.564 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8359999999999998e-06 2024-04-20T01:48:05.564 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.564 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.564 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.564 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.717218+0000", 2024-04-20T01:48:05.564 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0036978079999999999 2024-04-20T01:48:05.564 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.564 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.564 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.564 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.717243+0000", 2024-04-20T01:48:05.564 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4890999999999999e-05 2024-04-20T01:48:05.564 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.565 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.565 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.565 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.565 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.565 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.1s2 47/30 ECSubWrite(tid=112, reqid=client.4643.0:385, at_version=47'12, trim_to=0'0, roll_forward_to=41'10))", 2024-04-20T01:48:05.565 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.719776+0000", 2024-04-20T01:48:05.565 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 16.802337782999999, 2024-04-20T01:48:05.565 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0060583920000000001, 2024-04-20T01:48:05.565 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.565 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.565 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.565 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.565 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.565 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.719776+0000", 2024-04-20T01:48:05.565 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.566 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.566 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.566 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.566 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.719776+0000", 2024-04-20T01:48:05.566 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.566 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.566 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.566 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.566 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.719776+0000", 2024-04-20T01:48:05.567 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:05.567 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.567 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.567 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.567 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.719877+0000", 2024-04-20T01:48:05.567 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000101585 2024-04-20T01:48:05.567 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.567 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.567 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.567 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.719893+0000", 2024-04-20T01:48:05.567 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5641000000000002e-05 2024-04-20T01:48:05.567 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.567 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.567 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.568 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.719909+0000", 2024-04-20T01:48:05.568 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6291e-05 2024-04-20T01:48:05.568 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.568 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.568 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.568 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.719977+0000", 2024-04-20T01:48:05.568 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.7668000000000003e-05 2024-04-20T01:48:05.568 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.568 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.568 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.568 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.719987+0000", 2024-04-20T01:48:05.568 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.978e-06 2024-04-20T01:48:05.568 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.568 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.568 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.569 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.725806+0000", 2024-04-20T01:48:05.569 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0058187029999999997 2024-04-20T01:48:05.569 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.569 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.569 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.569 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.725835+0000", 2024-04-20T01:48:05.569 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9346e-05 2024-04-20T01:48:05.569 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.569 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.569 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.569 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.569 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.569 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.es1 47/30 ECSubWrite(tid=40, reqid=osd.11.0:39, at_version=0'0, trim_to=0'0, roll_forward_to=47'23))", 2024-04-20T01:48:05.570 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.721119+0000", 2024-04-20T01:48:05.570 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 16.800995042, 2024-04-20T01:48:05.570 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0069945759999999997, 2024-04-20T01:48:05.570 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.570 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.570 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.570 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.570 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.571 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721119+0000", 2024-04-20T01:48:05.571 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.571 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.571 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.571 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.572 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721119+0000", 2024-04-20T01:48:05.572 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.572 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.572 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.572 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.572 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721119+0000", 2024-04-20T01:48:05.572 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.572 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.572 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.572 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.572 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721128+0000", 2024-04-20T01:48:05.572 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.0620000000000007e-06 2024-04-20T01:48:05.572 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.572 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.573 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.573 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721138+0000", 2024-04-20T01:48:05.573 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0771000000000001e-05 2024-04-20T01:48:05.573 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.573 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.573 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.573 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721146+0000", 2024-04-20T01:48:05.573 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.3370000000000002e-06 2024-04-20T01:48:05.573 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.573 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.573 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.573 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721183+0000", 2024-04-20T01:48:05.573 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7283000000000002e-05 2024-04-20T01:48:05.573 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.574 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.574 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.574 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721194+0000", 2024-04-20T01:48:05.574 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1093e-05 2024-04-20T01:48:05.574 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.574 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.574 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.575 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.728091+0000", 2024-04-20T01:48:05.575 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0068972499999999997 2024-04-20T01:48:05.575 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.575 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.575 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.575 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.728114+0000", 2024-04-20T01:48:05.575 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2410000000000001e-05 2024-04-20T01:48:05.575 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.575 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.575 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.575 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.575 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.576 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.6s6 47/30 ECSubWrite(tid=11, reqid=osd.0.0:10, at_version=0'0, trim_to=0'0, roll_forward_to=47'6))", 2024-04-20T01:48:05.576 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.724263+0000", 2024-04-20T01:48:05.576 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 16.797851138999999, 2024-04-20T01:48:05.576 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.003868509, 2024-04-20T01:48:05.576 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.576 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.576 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.577 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.577 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.577 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.724263+0000", 2024-04-20T01:48:05.577 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.577 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.577 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.577 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.577 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.724263+0000", 2024-04-20T01:48:05.577 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.577 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.577 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.578 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.578 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.724262+0000", 2024-04-20T01:48:05.578 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.578 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.578 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.578 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.578 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.724272+0000", 2024-04-20T01:48:05.578 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3230000000000004e-06 2024-04-20T01:48:05.578 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.578 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.579 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.579 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.724282+0000", 2024-04-20T01:48:05.579 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0494e-05 2024-04-20T01:48:05.579 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.579 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.579 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.579 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.724289+0000", 2024-04-20T01:48:05.579 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.725e-06 2024-04-20T01:48:05.579 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.579 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.579 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.579 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.724340+0000", 2024-04-20T01:48:05.579 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.0997e-05 2024-04-20T01:48:05.579 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.580 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.580 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.580 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.724349+0000", 2024-04-20T01:48:05.580 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.0319999999999992e-06 2024-04-20T01:48:05.580 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.580 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.580 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.580 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.728116+0000", 2024-04-20T01:48:05.580 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.003767154 2024-04-20T01:48:05.580 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.580 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.580 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.580 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.728132+0000", 2024-04-20T01:48:05.580 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5432000000000001e-05 2024-04-20T01:48:05.581 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.581 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.581 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.581 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.581 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.581 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.1s2 47/30 ECSubWrite(tid=113, reqid=client.4643.0:385, at_version=0'0, trim_to=0'0, roll_forward_to=47'12))", 2024-04-20T01:48:05.581 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.726763+0000", 2024-04-20T01:48:05.581 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 16.795351371999999, 2024-04-20T01:48:05.581 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.003897934, 2024-04-20T01:48:05.581 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.581 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.581 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.581 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.581 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.582 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.726763+0000", 2024-04-20T01:48:05.582 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.582 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.582 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.582 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.582 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.726763+0000", 2024-04-20T01:48:05.582 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.582 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.582 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.582 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.582 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.726762+0000", 2024-04-20T01:48:05.582 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.582 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.582 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.583 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.583 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.726771+0000", 2024-04-20T01:48:05.583 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.1920000000000005e-06 2024-04-20T01:48:05.583 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.583 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.583 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.583 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.726780+0000", 2024-04-20T01:48:05.583 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7850000000000003e-06 2024-04-20T01:48:05.583 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.583 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.583 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.583 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.726792+0000", 2024-04-20T01:48:05.583 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1268e-05 2024-04-20T01:48:05.583 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.584 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.584 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.584 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.726830+0000", 2024-04-20T01:48:05.584 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7944000000000001e-05 2024-04-20T01:48:05.584 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.584 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.584 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.584 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.726835+0000", 2024-04-20T01:48:05.584 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.959e-06 2024-04-20T01:48:05.584 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.584 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.584 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.584 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.730634+0000", 2024-04-20T01:48:05.584 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0037988060000000001 2024-04-20T01:48:05.584 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.585 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.585 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.585 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.730661+0000", 2024-04-20T01:48:05.585 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6520999999999999e-05 2024-04-20T01:48:05.585 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.585 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.585 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.585 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.585 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.585 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 48/30 ECSubWrite(tid=123, reqid=osd.4.0:121, at_version=0'0, trim_to=0'0, roll_forward_to=48'53))", 2024-04-20T01:48:05.585 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.733335+0000", 2024-04-20T01:48:05.585 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 15.788779353000001, 2024-04-20T01:48:05.585 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0044011809999999997, 2024-04-20T01:48:05.586 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.586 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.586 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.586 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.586 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.586 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.733335+0000", 2024-04-20T01:48:05.586 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.586 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.586 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.586 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.586 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.733335+0000", 2024-04-20T01:48:05.586 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.586 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.586 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.587 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.587 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.733334+0000", 2024-04-20T01:48:05.587 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.587 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.587 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.587 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.587 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.733344+0000", 2024-04-20T01:48:05.587 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4469999999999995e-06 2024-04-20T01:48:05.587 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.587 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.587 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.587 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.733357+0000", 2024-04-20T01:48:05.587 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3004e-05 2024-04-20T01:48:05.587 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.588 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.588 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.588 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.733365+0000", 2024-04-20T01:48:05.588 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.3489999999999992e-06 2024-04-20T01:48:05.588 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.588 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.588 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.588 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.733417+0000", 2024-04-20T01:48:05.588 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.1547e-05 2024-04-20T01:48:05.588 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.588 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.588 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.588 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.733427+0000", 2024-04-20T01:48:05.588 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0779e-05 2024-04-20T01:48:05.588 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.589 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.589 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.589 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.737713+0000", 2024-04-20T01:48:05.589 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0042858050000000002 2024-04-20T01:48:05.589 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.589 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.589 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.589 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.737736+0000", 2024-04-20T01:48:05.589 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2858e-05 2024-04-20T01:48:05.589 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.589 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.589 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.589 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.589 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.590 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 63/30 ECSubWrite(tid=164, reqid=client.4643.0:565, at_version=63'81, trim_to=0'0, roll_forward_to=62'79))", 2024-04-20T01:48:05.590 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:04.757834+0000", 2024-04-20T01:48:05.590 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.764280403, 2024-04-20T01:48:05.590 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0039371160000000001, 2024-04-20T01:48:05.590 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.590 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.590 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.590 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.590 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.590 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.757834+0000", 2024-04-20T01:48:05.590 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.590 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.590 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.591 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.591 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.757834+0000", 2024-04-20T01:48:05.591 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.591 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.591 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.591 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.591 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.757834+0000", 2024-04-20T01:48:05.591 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:48:05.591 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.591 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.591 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.591 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.758175+0000", 2024-04-20T01:48:05.591 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000341761 2024-04-20T01:48:05.591 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.591 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.591 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.592 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.758190+0000", 2024-04-20T01:48:05.592 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.411e-05 2024-04-20T01:48:05.592 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.592 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.592 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.592 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.758206+0000", 2024-04-20T01:48:05.592 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6392000000000002e-05 2024-04-20T01:48:05.592 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.592 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.592 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.592 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.758267+0000", 2024-04-20T01:48:05.592 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.0723000000000001e-05 2024-04-20T01:48:05.592 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.593 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.593 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.593 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.758280+0000", 2024-04-20T01:48:05.593 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2975e-05 2024-04-20T01:48:05.593 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.593 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.593 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.593 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.761748+0000", 2024-04-20T01:48:05.593 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0034686500000000002 2024-04-20T01:48:05.593 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.593 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.593 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.593 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.761771+0000", 2024-04-20T01:48:05.593 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2633000000000002e-05 2024-04-20T01:48:05.594 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.594 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.594 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.594 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.594 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.594 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 63/30 ECSubWrite(tid=165, reqid=client.4643.0:566, at_version=63'82, trim_to=0'0, roll_forward_to=62'79))", 2024-04-20T01:48:05.594 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:04.760038+0000", 2024-04-20T01:48:05.594 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.762075897, 2024-04-20T01:48:05.594 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0046125869999999996, 2024-04-20T01:48:05.594 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.594 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.594 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.594 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.594 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.594 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.760038+0000", 2024-04-20T01:48:05.595 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.595 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.595 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.595 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.595 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.760038+0000", 2024-04-20T01:48:05.595 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.595 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.595 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.595 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.595 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.760038+0000", 2024-04-20T01:48:05.595 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.595 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.595 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.595 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.595 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.760331+0000", 2024-04-20T01:48:05.596 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00029345 2024-04-20T01:48:05.596 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.596 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.596 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.596 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.760341+0000", 2024-04-20T01:48:05.596 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0285000000000001e-05 2024-04-20T01:48:05.596 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.596 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.596 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.596 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.760349+0000", 2024-04-20T01:48:05.596 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.9500000000000001e-06 2024-04-20T01:48:05.596 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.596 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.597 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.597 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.760382+0000", 2024-04-20T01:48:05.597 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.2236000000000001e-05 2024-04-20T01:48:05.597 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.597 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.597 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.597 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.760388+0000", 2024-04-20T01:48:05.597 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.7329999999999997e-06 2024-04-20T01:48:05.597 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.597 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.597 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.597 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764633+0000", 2024-04-20T01:48:05.597 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0042450020000000003 2024-04-20T01:48:05.597 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.598 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.598 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.598 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764651+0000", 2024-04-20T01:48:05.598 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7589e-05 2024-04-20T01:48:05.598 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.598 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.598 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.598 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.598 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.598 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 63/30 ECSubWrite(tid=166, reqid=client.4643.0:567, at_version=63'83, trim_to=0'0, roll_forward_to=62'79))", 2024-04-20T01:48:05.598 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:04.764460+0000", 2024-04-20T01:48:05.598 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.757654465, 2024-04-20T01:48:05.599 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0060601029999999998, 2024-04-20T01:48:05.599 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.599 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.599 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.599 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.599 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.599 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764460+0000", 2024-04-20T01:48:05.599 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.599 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.599 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.599 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.599 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764460+0000", 2024-04-20T01:48:05.599 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.599 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.600 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.600 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.600 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764459+0000", 2024-04-20T01:48:05.600 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:05.600 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.600 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.600 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.600 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764867+0000", 2024-04-20T01:48:05.600 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000407546 2024-04-20T01:48:05.600 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.600 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.600 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.600 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764881+0000", 2024-04-20T01:48:05.600 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4352000000000001e-05 2024-04-20T01:48:05.601 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.601 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.601 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.601 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764894+0000", 2024-04-20T01:48:05.601 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3268000000000001e-05 2024-04-20T01:48:05.601 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.601 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.601 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.601 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764939+0000", 2024-04-20T01:48:05.601 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.4583000000000003e-05 2024-04-20T01:48:05.601 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.601 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.601 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.601 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764949+0000", 2024-04-20T01:48:05.601 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.798e-06 2024-04-20T01:48:05.602 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.602 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.602 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.602 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.770499+0000", 2024-04-20T01:48:05.602 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0055503669999999996 2024-04-20T01:48:05.602 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.602 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.602 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.602 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.770520+0000", 2024-04-20T01:48:05.602 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0951999999999999e-05 2024-04-20T01:48:05.602 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.602 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.602 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.603 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.603 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.603 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 63/30 ECSubWrite(tid=167, reqid=client.4643.0:568, at_version=63'84, trim_to=0'0, roll_forward_to=63'81))", 2024-04-20T01:48:05.603 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:04.766814+0000", 2024-04-20T01:48:05.603 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.75530067999999995, 2024-04-20T01:48:05.603 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0037432820000000001, 2024-04-20T01:48:05.603 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:05.603 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:05.603 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:05.603 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.603 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:05.603 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.766814+0000", 2024-04-20T01:48:05.603 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.603 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.603 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.604 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:05.604 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.766814+0000", 2024-04-20T01:48:05.604 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:05.604 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.604 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.604 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:05.604 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.766813+0000", 2024-04-20T01:48:05.604 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:05.604 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.604 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.604 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:05.604 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.766830+0000", 2024-04-20T01:48:05.604 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6818999999999999e-05 2024-04-20T01:48:05.604 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.604 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.605 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:05.605 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.766843+0000", 2024-04-20T01:48:05.605 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3529e-05 2024-04-20T01:48:05.605 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.605 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.605 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:05.605 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.766852+0000", 2024-04-20T01:48:05.605 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.7779999999999996e-06 2024-04-20T01:48:05.605 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.605 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.605 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:05.605 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.766904+0000", 2024-04-20T01:48:05.605 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.1480000000000002e-05 2024-04-20T01:48:05.605 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.605 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.605 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:05.606 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.766914+0000", 2024-04-20T01:48:05.606 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0553e-05 2024-04-20T01:48:05.606 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.606 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.606 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:05.606 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.770544+0000", 2024-04-20T01:48:05.606 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0036299959999999999 2024-04-20T01:48:05.606 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:05.606 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:05.606 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:05.606 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.770557+0000", 2024-04-20T01:48:05.606 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2588e-05 2024-04-20T01:48:05.606 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.606 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.606 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.607 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:05.607 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:05.607 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:05.608 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-20T01:48:05.609 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:05.622+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:05.709 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:05.724+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:05.737 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:05.737 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:05.737 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 0 2024-04-20T01:48:05.737 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:05.742 INFO:tasks.rados.rados.0.smithi165.stdout:235: done (1 left) 2024-04-20T01:48:05.743 INFO:tasks.rados.rados.0.smithi165.stdout:237: done (0 left) 2024-04-20T01:48:05.743 INFO:tasks.rados.rados.0.smithi165.stdout:238: read oid 26 snap 16 2024-04-20T01:48:05.743 INFO:tasks.rados.rados.0.smithi165.stdout:238: expect (ObjNum 96 snap 16 seq_num 96) 2024-04-20T01:48:05.743 INFO:tasks.rados.rados.0.smithi165.stdout:239: snap_remove snap 15 2024-04-20T01:48:05.748 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-20T01:48:05.749 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is set 2024-04-20T01:48:05.751 INFO:teuthology.orchestra.run.smithi044.stderr:set 2.4 pg_upmap_items mapping to [3->10,14->7] 2024-04-20T01:48:05.810 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:05.825+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:05.877 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:05.877 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:05.878 INFO:teuthology.orchestra.run.smithi121.stdout: "complaint_time": 30, 2024-04-20T01:48:05.878 INFO:teuthology.orchestra.run.smithi121.stdout: "num_blocked_ops": 0 2024-04-20T01:48:05.878 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:05.888 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-20T01:48:05.911 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:05.926+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:06.012 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:06.026+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:06.018 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:06.019 INFO:teuthology.orchestra.run.smithi121.stdout: "size": 20, 2024-04-20T01:48:06.019 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 600, 2024-04-20T01:48:06.019 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-04-20T01:48:06.019 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.019 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-20T01:48:06.019 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:17.004323+0000", 2024-04-20T01:48:06.019 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 49.027734909000003, 2024-04-20T01:48:06.019 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.29931750600000001, 2024-04-20T01:48:06.019 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:06.019 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:06.019 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:06.019 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:48:06.019 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:48:06.020 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 1 2024-04-20T01:48:06.020 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.020 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:06.020 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.020 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:06.020 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004323+0000", 2024-04-20T01:48:06.020 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.020 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.020 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.020 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:06.020 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004323+0000", 2024-04-20T01:48:06.020 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.020 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.020 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.021 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:06.021 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004321+0000", 2024-04-20T01:48:06.021 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:06.021 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.021 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.021 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:06.021 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004337+0000", 2024-04-20T01:48:06.021 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5866999999999999e-05 2024-04-20T01:48:06.021 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.021 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.021 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:06.021 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004349+0000", 2024-04-20T01:48:06.021 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1802e-05 2024-04-20T01:48:06.021 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.021 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.022 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.022 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004360+0000", 2024-04-20T01:48:06.022 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1348999999999999e-05 2024-04-20T01:48:06.022 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.022 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.022 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.022 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004411+0000", 2024-04-20T01:48:06.022 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.0385e-05 2024-04-20T01:48:06.022 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.022 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.022 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for peered", 2024-04-20T01:48:06.022 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004419+0000", 2024-04-20T01:48:06.022 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4430000000000003e-06 2024-04-20T01:48:06.022 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.023 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.023 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.023 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.302330+0000", 2024-04-20T01:48:06.023 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.29791090199999998 2024-04-20T01:48:06.023 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.023 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.023 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:06.023 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.302488+0000", 2024-04-20T01:48:06.023 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00015830799999999999 2024-04-20T01:48:06.023 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.023 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.023 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:48:06.024 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.302650+0000", 2024-04-20T01:48:06.024 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00016196900000000001 2024-04-20T01:48:06.024 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.024 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.024 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:48:06.024 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.303204+0000", 2024-04-20T01:48:06.024 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00055385000000000005 2024-04-20T01:48:06.024 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.024 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.024 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:48:06.024 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.303574+0000", 2024-04-20T01:48:06.024 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00037030800000000002 2024-04-20T01:48:06.024 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.025 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.025 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:06.025 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.303614+0000", 2024-04-20T01:48:06.025 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.9161e-05 2024-04-20T01:48:06.025 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.025 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.025 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:06.025 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.303640+0000", 2024-04-20T01:48:06.025 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6752999999999999e-05 2024-04-20T01:48:06.025 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.025 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.025 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.025 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.025 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.026 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:18 3.3s0 3:cb777d78:::smithi16579500-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:06.026 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.756072+0000", 2024-04-20T01:48:06.026 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 31.275985502000001, 2024-04-20T01:48:06.026 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013705281, 2024-04-20T01:48:06.026 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:06.026 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:06.026 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:06.026 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:06.026 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.026 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 18 2024-04-20T01:48:06.026 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.026 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:06.026 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.026 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:06.027 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756072+0000", 2024-04-20T01:48:06.027 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.027 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.027 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.027 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:06.027 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756072+0000", 2024-04-20T01:48:06.027 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.027 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.027 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.027 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:06.027 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756072+0000", 2024-04-20T01:48:06.027 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:06.027 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.027 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.027 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:06.028 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756074+0000", 2024-04-20T01:48:06.028 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.148e-06 2024-04-20T01:48:06.028 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.028 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.028 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:06.028 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756079+0000", 2024-04-20T01:48:06.028 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.4790000000000004e-06 2024-04-20T01:48:06.028 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.028 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.028 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.028 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756083+0000", 2024-04-20T01:48:06.028 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.4910000000000001e-06 2024-04-20T01:48:06.028 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.028 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.029 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.029 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.758130+0000", 2024-04-20T01:48:06.029 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00204725 2024-04-20T01:48:06.029 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.029 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.029 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:06.029 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.758200+0000", 2024-04-20T01:48:06.029 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0246000000000005e-05 2024-04-20T01:48:06.029 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.029 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.029 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.029 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766738+0000", 2024-04-20T01:48:06.029 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0085377939999999996 2024-04-20T01:48:06.029 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.029 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.029 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:06.029 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766828+0000", 2024-04-20T01:48:06.030 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.9677000000000002e-05 2024-04-20T01:48:06.030 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.030 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.030 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:06.030 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.769777+0000", 2024-04-20T01:48:06.030 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0029495369999999999 2024-04-20T01:48:06.030 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.030 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.030 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.030 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.030 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.030 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:48 3.2s0 3:4efae9ea:::smithi16579500-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:06.030 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.768007+0000", 2024-04-20T01:48:06.031 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 31.264050357999999, 2024-04-20T01:48:06.031 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010953058, 2024-04-20T01:48:06.031 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:06.031 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:06.031 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:06.031 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:06.031 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.031 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 48 2024-04-20T01:48:06.031 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.031 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:06.031 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.031 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:06.031 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768007+0000", 2024-04-20T01:48:06.032 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.032 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.032 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.032 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:06.032 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768007+0000", 2024-04-20T01:48:06.032 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.032 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.032 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.032 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:06.032 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768007+0000", 2024-04-20T01:48:06.032 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:06.032 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.032 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.032 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:06.033 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768011+0000", 2024-04-20T01:48:06.033 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2899999999999996e-06 2024-04-20T01:48:06.033 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.033 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.033 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:06.033 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768018+0000", 2024-04-20T01:48:06.033 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0060000000000003e-06 2024-04-20T01:48:06.033 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.033 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.033 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.033 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768025+0000", 2024-04-20T01:48:06.033 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.8390000000000001e-06 2024-04-20T01:48:06.033 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.033 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.034 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.034 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.772765+0000", 2024-04-20T01:48:06.034 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0047401409999999998 2024-04-20T01:48:06.034 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.034 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.034 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:06.034 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.772861+0000", 2024-04-20T01:48:06.034 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6513000000000002e-05 2024-04-20T01:48:06.034 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.034 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.034 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.034 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.776375+0000", 2024-04-20T01:48:06.034 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0035133679999999998 2024-04-20T01:48:06.034 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.035 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.035 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:06.035 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.776459+0000", 2024-04-20T01:48:06.035 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4368000000000003e-05 2024-04-20T01:48:06.035 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.035 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.035 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:06.035 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.778960+0000", 2024-04-20T01:48:06.035 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0025012910000000001 2024-04-20T01:48:06.035 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.035 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.035 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.035 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.035 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.035 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:67 3.3s0 3:c86e2dd3:::smithi16579500-23:head [append 0~376832,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:06.035 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.783364+0000", 2024-04-20T01:48:06.035 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 31.248693568, 2024-04-20T01:48:06.036 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013527099000000001, 2024-04-20T01:48:06.036 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:06.036 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:06.036 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:06.036 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:06.036 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.036 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 67 2024-04-20T01:48:06.036 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.036 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:06.036 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.036 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:06.036 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783364+0000", 2024-04-20T01:48:06.036 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.036 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.036 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.037 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:06.037 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783364+0000", 2024-04-20T01:48:06.037 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.037 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.037 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.037 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:06.037 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783355+0000", 2024-04-20T01:48:06.037 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999909 2024-04-20T01:48:06.037 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.037 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.037 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:06.037 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783704+0000", 2024-04-20T01:48:06.037 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00034900399999999998 2024-04-20T01:48:06.037 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.037 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.038 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:06.038 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783720+0000", 2024-04-20T01:48:06.038 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5464e-05 2024-04-20T01:48:06.038 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.038 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.038 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.038 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783727+0000", 2024-04-20T01:48:06.038 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.6199999999999999e-06 2024-04-20T01:48:06.038 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.038 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.038 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.038 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783773+0000", 2024-04-20T01:48:06.038 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5237000000000001e-05 2024-04-20T01:48:06.038 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.038 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.039 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:06.039 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783881+0000", 2024-04-20T01:48:06.039 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00010799200000000001 2024-04-20T01:48:06.039 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.039 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.039 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.039 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.784468+0000", 2024-04-20T01:48:06.039 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00058749400000000004 2024-04-20T01:48:06.039 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.039 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.039 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.039 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.785849+0000", 2024-04-20T01:48:06.039 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0013810420000000001 2024-04-20T01:48:06.039 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.039 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.039 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:06.040 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.796778+0000", 2024-04-20T01:48:06.040 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010928955000000001 2024-04-20T01:48:06.040 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.040 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.040 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:06.040 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.796891+0000", 2024-04-20T01:48:06.040 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00011315399999999999 2024-04-20T01:48:06.040 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.040 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.040 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.040 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.040 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.040 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:68 3.3s0 3:c86e2dd3:::smithi16579500-23:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:06.040 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.783739+0000", 2024-04-20T01:48:06.041 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 31.248318294000001, 2024-04-20T01:48:06.041 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013359042, 2024-04-20T01:48:06.041 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:06.041 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:06.041 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:06.041 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:06.041 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.041 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 68 2024-04-20T01:48:06.041 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.041 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:06.041 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.041 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:06.041 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783739+0000", 2024-04-20T01:48:06.041 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.041 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.041 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.042 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:06.042 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783739+0000", 2024-04-20T01:48:06.042 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.042 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.042 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.042 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:06.042 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783739+0000", 2024-04-20T01:48:06.042 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:06.042 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.042 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.042 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:06.042 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783744+0000", 2024-04-20T01:48:06.042 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.7690000000000004e-06 2024-04-20T01:48:06.042 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.043 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.043 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:06.043 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783750+0000", 2024-04-20T01:48:06.043 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.0719999999999996e-06 2024-04-20T01:48:06.043 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.043 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.043 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.043 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783754+0000", 2024-04-20T01:48:06.043 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.6450000000000004e-06 2024-04-20T01:48:06.043 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.043 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.043 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.043 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.784916+0000", 2024-04-20T01:48:06.043 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001161477 2024-04-20T01:48:06.044 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.044 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.044 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:06.044 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.785022+0000", 2024-04-20T01:48:06.044 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000106153 2024-04-20T01:48:06.044 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.044 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.044 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.044 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.785477+0000", 2024-04-20T01:48:06.044 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00045462699999999998 2024-04-20T01:48:06.044 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.044 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.044 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.044 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.787709+0000", 2024-04-20T01:48:06.044 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0022322309999999999 2024-04-20T01:48:06.044 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.045 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.045 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:06.045 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797021+0000", 2024-04-20T01:48:06.045 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0093120730000000006 2024-04-20T01:48:06.045 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.045 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.045 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:06.045 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797098+0000", 2024-04-20T01:48:06.045 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.7526000000000003e-05 2024-04-20T01:48:06.045 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.045 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.045 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.045 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.045 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.046 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:69 3.3s0 3:c86e2dd3:::smithi16579500-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:06.046 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.783758+0000", 2024-04-20T01:48:06.046 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 31.248299747000001, 2024-04-20T01:48:06.046 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.017903170999999999, 2024-04-20T01:48:06.046 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:06.046 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:06.046 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:06.046 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:06.046 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.046 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 69 2024-04-20T01:48:06.046 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.046 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:06.047 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.047 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:06.047 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783758+0000", 2024-04-20T01:48:06.047 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.047 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.047 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.047 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:06.047 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783758+0000", 2024-04-20T01:48:06.047 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.047 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.047 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.047 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:06.047 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783758+0000", 2024-04-20T01:48:06.047 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:06.047 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.047 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.048 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:06.048 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783759+0000", 2024-04-20T01:48:06.048 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.734e-06 2024-04-20T01:48:06.048 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.048 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.048 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:06.048 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783763+0000", 2024-04-20T01:48:06.048 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.5820000000000002e-06 2024-04-20T01:48:06.048 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.048 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.048 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.048 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783766+0000", 2024-04-20T01:48:06.048 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0469999999999998e-06 2024-04-20T01:48:06.048 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.048 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.049 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.049 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.785726+0000", 2024-04-20T01:48:06.049 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0019601079999999999 2024-04-20T01:48:06.049 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.049 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.049 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:06.049 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.785810+0000", 2024-04-20T01:48:06.049 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.3522000000000004e-05 2024-04-20T01:48:06.049 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.049 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.049 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.049 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797485+0000", 2024-04-20T01:48:06.049 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011675207 2024-04-20T01:48:06.049 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.049 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.049 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:06.050 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797573+0000", 2024-04-20T01:48:06.050 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8146000000000006e-05 2024-04-20T01:48:06.050 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.050 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.050 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:06.050 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.801661+0000", 2024-04-20T01:48:06.050 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0040880969999999997 2024-04-20T01:48:06.050 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.050 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.050 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.050 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.050 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.050 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:75 3.2s0 3:48443997:::smithi16579500-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:06.050 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.789051+0000", 2024-04-20T01:48:06.051 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 31.243006212000001, 2024-04-20T01:48:06.051 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013741625, 2024-04-20T01:48:06.051 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:06.051 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:06.051 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:06.051 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:06.051 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.051 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 75 2024-04-20T01:48:06.051 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.051 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:06.051 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.051 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:06.051 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789051+0000", 2024-04-20T01:48:06.051 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.051 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.052 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.052 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:06.052 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789051+0000", 2024-04-20T01:48:06.052 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.052 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.052 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.052 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:06.052 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789051+0000", 2024-04-20T01:48:06.052 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:06.052 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.052 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.052 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:06.052 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789054+0000", 2024-04-20T01:48:06.052 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7499999999999999e-06 2024-04-20T01:48:06.052 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.052 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.053 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:06.053 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789060+0000", 2024-04-20T01:48:06.053 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.567e-06 2024-04-20T01:48:06.053 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.053 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.053 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.053 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789066+0000", 2024-04-20T01:48:06.053 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.1499999999999998e-06 2024-04-20T01:48:06.053 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.053 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.053 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.053 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.790905+0000", 2024-04-20T01:48:06.053 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0018397000000000001 2024-04-20T01:48:06.053 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.053 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.054 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:06.054 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.790994+0000", 2024-04-20T01:48:06.054 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8679999999999998e-05 2024-04-20T01:48:06.054 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.054 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.054 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.054 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797798+0000", 2024-04-20T01:48:06.054 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0068040899999999996 2024-04-20T01:48:06.054 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.054 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.054 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:06.054 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797879+0000", 2024-04-20T01:48:06.054 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.1318999999999998e-05 2024-04-20T01:48:06.054 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.054 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.055 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:06.055 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.802793+0000", 2024-04-20T01:48:06.055 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.004913789 2024-04-20T01:48:06.055 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.055 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.055 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.055 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.055 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.055 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:84 3.3s0 3:cb8bd5ef:::smithi16579500-28:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:06.055 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.797448+0000", 2024-04-20T01:48:06.055 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 31.234609750000001, 2024-04-20T01:48:06.055 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.01109836, 2024-04-20T01:48:06.055 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:06.055 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:06.055 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:06.056 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:06.056 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.056 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 84 2024-04-20T01:48:06.056 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.056 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:06.056 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.056 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:06.056 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797448+0000", 2024-04-20T01:48:06.056 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.056 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.056 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.056 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:06.056 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797448+0000", 2024-04-20T01:48:06.056 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.056 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.057 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.057 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:06.057 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797448+0000", 2024-04-20T01:48:06.057 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:06.057 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.057 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.057 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:06.057 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797450+0000", 2024-04-20T01:48:06.057 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7999999999999999e-06 2024-04-20T01:48:06.057 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.057 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.057 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:06.057 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797458+0000", 2024-04-20T01:48:06.057 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.2590000000000003e-06 2024-04-20T01:48:06.057 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.058 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.058 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.058 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797464+0000", 2024-04-20T01:48:06.058 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.2979999999999997e-06 2024-04-20T01:48:06.058 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.058 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.058 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.058 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.801705+0000", 2024-04-20T01:48:06.058 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0042409819999999999 2024-04-20T01:48:06.058 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.058 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.058 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:06.058 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.801761+0000", 2024-04-20T01:48:06.059 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.5696999999999999e-05 2024-04-20T01:48:06.059 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.059 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.059 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.059 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.808533+0000", 2024-04-20T01:48:06.059 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0067724639999999997 2024-04-20T01:48:06.059 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.059 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.059 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:06.059 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.808546+0000", 2024-04-20T01:48:06.059 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3276e-05 2024-04-20T01:48:06.059 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.059 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.059 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.059 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.059 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.061 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:232 3.3s0 3:cb777d78:::smithi16579500-6:head [copy-from ver 6] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:06.061 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.787663+0000", 2024-04-20T01:48:06.061 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 27.244395145999999, 2024-04-20T01:48:06.061 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.019650832, 2024-04-20T01:48:06.061 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:06.061 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:06.061 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:06.061 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:06.061 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.061 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 232 2024-04-20T01:48:06.062 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.062 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:06.062 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.062 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:06.062 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787663+0000", 2024-04-20T01:48:06.062 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.062 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.062 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.062 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:06.062 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787663+0000", 2024-04-20T01:48:06.062 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.062 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.062 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.062 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:06.063 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787661+0000", 2024-04-20T01:48:06.063 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:06.063 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.063 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.063 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:06.063 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787673+0000", 2024-04-20T01:48:06.063 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.181e-05 2024-04-20T01:48:06.063 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.063 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.063 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:06.063 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787689+0000", 2024-04-20T01:48:06.063 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6731999999999998e-05 2024-04-20T01:48:06.063 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.063 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787706+0000", 2024-04-20T01:48:06.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6739e-05 2024-04-20T01:48:06.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787795+0000", 2024-04-20T01:48:06.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.9226000000000003e-05 2024-04-20T01:48:06.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:06.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787904+0000", 2024-04-20T01:48:06.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000108452 2024-04-20T01:48:06.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.797712+0000", 2024-04-20T01:48:06.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0098079599999999992 2024-04-20T01:48:06.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.799543+0000", 2024-04-20T01:48:06.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001831596 2024-04-20T01:48:06.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:06.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807224+0000", 2024-04-20T01:48:06.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0076805249999999997 2024-04-20T01:48:06.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.066 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.066 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:06.066 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807313+0000", 2024-04-20T01:48:06.066 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.9431999999999993e-05 2024-04-20T01:48:06.066 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.066 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.066 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.066 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.066 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:233 3.3s0 3:cb777d78:::smithi16579500-6:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:06.066 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.787721+0000", 2024-04-20T01:48:06.066 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 27.244336533999999, 2024-04-20T01:48:06.066 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.01143623, 2024-04-20T01:48:06.066 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:06.067 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:06.067 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:06.067 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:06.067 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.067 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 233 2024-04-20T01:48:06.067 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.067 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:06.067 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.067 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:06.067 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787721+0000", 2024-04-20T01:48:06.067 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.067 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.067 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.067 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:06.067 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787721+0000", 2024-04-20T01:48:06.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.068 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.068 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:06.068 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787721+0000", 2024-04-20T01:48:06.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:06.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.068 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.068 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:06.068 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787743+0000", 2024-04-20T01:48:06.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2540000000000001e-05 2024-04-20T01:48:06.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.068 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.068 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:06.068 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787753+0000", 2024-04-20T01:48:06.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0101e-05 2024-04-20T01:48:06.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.069 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787760+0000", 2024-04-20T01:48:06.069 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.3729999999999998e-06 2024-04-20T01:48:06.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.069 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.788051+0000", 2024-04-20T01:48:06.069 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00029180999999999999 2024-04-20T01:48:06.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.069 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:06.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.788092+0000", 2024-04-20T01:48:06.069 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.0309000000000003e-05 2024-04-20T01:48:06.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.798980+0000", 2024-04-20T01:48:06.070 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010888070999999999 2024-04-20T01:48:06.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:06.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.799065+0000", 2024-04-20T01:48:06.070 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.5241999999999995e-05 2024-04-20T01:48:06.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:06.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.799157+0000", 2024-04-20T01:48:06.071 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2388999999999999e-05 2024-04-20T01:48:06.071 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.071 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.071 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.071 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.071 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.071 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:296 3.2s0 3:44d9544b:::smithi16579500-38:head [read 0~1474560,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:06.071 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.807470+0000", 2024-04-20T01:48:06.071 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 22.224587561, 2024-04-20T01:48:06.071 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011565101, 2024-04-20T01:48:06.071 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:06.071 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:06.071 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:06.071 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:06.071 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.072 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 296 2024-04-20T01:48:06.072 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.072 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:06.072 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.072 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:06.072 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807470+0000", 2024-04-20T01:48:06.072 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.072 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.072 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.072 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:06.072 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807470+0000", 2024-04-20T01:48:06.072 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.072 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.072 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.072 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:06.073 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807470+0000", 2024-04-20T01:48:06.073 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:06.073 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.073 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:06.073 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807473+0000", 2024-04-20T01:48:06.073 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9809999999999999e-06 2024-04-20T01:48:06.073 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.073 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:06.073 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807479+0000", 2024-04-20T01:48:06.073 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.1970000000000001e-06 2024-04-20T01:48:06.073 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.073 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.073 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807483+0000", 2024-04-20T01:48:06.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.053e-06 2024-04-20T01:48:06.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.074 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807633+0000", 2024-04-20T01:48:06.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00015004999999999999 2024-04-20T01:48:06.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:06.074 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807643+0000", 2024-04-20T01:48:06.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7049999999999994e-06 2024-04-20T01:48:06.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.074 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.811287+0000", 2024-04-20T01:48:06.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0036436699999999999 2024-04-20T01:48:06.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.075 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:06.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.811372+0000", 2024-04-20T01:48:06.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.5251000000000001e-05 2024-04-20T01:48:06.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.075 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:06.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819035+0000", 2024-04-20T01:48:06.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.007663464 2024-04-20T01:48:06.075 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.075 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.075 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.075 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:298 3.2s0 3:4302a124:::smithi16579500-33:head [append 0~0] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:06.076 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.808446+0000", 2024-04-20T01:48:06.076 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 22.223611480999999, 2024-04-20T01:48:06.076 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.01090315, 2024-04-20T01:48:06.076 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:06.076 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:06.076 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:06.076 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:06.076 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.076 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 298 2024-04-20T01:48:06.076 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.076 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:06.076 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.076 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:06.076 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808446+0000", 2024-04-20T01:48:06.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.077 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:06.077 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808446+0000", 2024-04-20T01:48:06.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.077 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:06.077 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808445+0000", 2024-04-20T01:48:06.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:06.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.077 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:06.078 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808451+0000", 2024-04-20T01:48:06.078 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.9710000000000001e-06 2024-04-20T01:48:06.078 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.078 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.078 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:06.078 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808466+0000", 2024-04-20T01:48:06.078 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4945e-05 2024-04-20T01:48:06.078 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.078 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.078 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.078 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808473+0000", 2024-04-20T01:48:06.078 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.8789999999999997e-06 2024-04-20T01:48:06.078 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.079 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808579+0000", 2024-04-20T01:48:06.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00010635 2024-04-20T01:48:06.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.079 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:06.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808587+0000", 2024-04-20T01:48:06.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.4000000000000003e-06 2024-04-20T01:48:06.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.079 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.813433+0000", 2024-04-20T01:48:06.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0048458549999999996 2024-04-20T01:48:06.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.080 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:06.080 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.813595+0000", 2024-04-20T01:48:06.080 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00016224300000000001 2024-04-20T01:48:06.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.080 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.080 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.814543+0000", 2024-04-20T01:48:06.080 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00094800199999999998 2024-04-20T01:48:06.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.080 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.080 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.818083+0000", 2024-04-20T01:48:06.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.003539824 2024-04-20T01:48:06.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.081 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:06.081 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819295+0000", 2024-04-20T01:48:06.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001212634 2024-04-20T01:48:06.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.081 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:06.081 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819349+0000", 2024-04-20T01:48:06.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3949999999999997e-05 2024-04-20T01:48:06.081 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.081 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.082 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.082 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.082 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:299 3.2s0 3:4302a124:::smithi16579500-33:head [rollback a] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:06.082 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.808485+0000", 2024-04-20T01:48:06.082 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 22.223572789999999, 2024-04-20T01:48:06.082 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011156045, 2024-04-20T01:48:06.082 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:06.082 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:06.082 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:06.082 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:06.082 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.082 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 299 2024-04-20T01:48:06.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.083 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:06.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.083 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:06.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808485+0000", 2024-04-20T01:48:06.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.083 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:06.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808485+0000", 2024-04-20T01:48:06.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.083 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:06.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808484+0000", 2024-04-20T01:48:06.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:06.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:06.084 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808488+0000", 2024-04-20T01:48:06.084 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.3529999999999999e-06 2024-04-20T01:48:06.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:06.084 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808495+0000", 2024-04-20T01:48:06.084 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.4390000000000003e-06 2024-04-20T01:48:06.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.084 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808501+0000", 2024-04-20T01:48:06.084 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.4480000000000002e-06 2024-04-20T01:48:06.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.085 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.085 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.085 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808634+0000", 2024-04-20T01:48:06.085 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000133461 2024-04-20T01:48:06.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.085 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.085 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:06.085 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808650+0000", 2024-04-20T01:48:06.085 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6271999999999999e-05 2024-04-20T01:48:06.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.085 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.085 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.085 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.815215+0000", 2024-04-20T01:48:06.085 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0065644509999999998 2024-04-20T01:48:06.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.085 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.086 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:06.086 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.815328+0000", 2024-04-20T01:48:06.086 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000112754 2024-04-20T01:48:06.086 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.086 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.086 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.086 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.815818+0000", 2024-04-20T01:48:06.086 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00049083000000000004 2024-04-20T01:48:06.086 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.086 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.086 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.086 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819209+0000", 2024-04-20T01:48:06.086 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0033910139999999999 2024-04-20T01:48:06.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.087 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.087 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:06.087 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819596+0000", 2024-04-20T01:48:06.087 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00038603299999999998 2024-04-20T01:48:06.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.087 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.087 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:06.087 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819641+0000", 2024-04-20T01:48:06.087 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5442999999999998e-05 2024-04-20T01:48:06.087 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.087 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.087 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.088 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.088 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:300 3.2s0 3:4302a124:::smithi16579500-33:head [append 0~0] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:06.088 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.808516+0000", 2024-04-20T01:48:06.088 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 22.223541653000002, 2024-04-20T01:48:06.088 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.016176293000000001, 2024-04-20T01:48:06.088 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:06.088 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:06.088 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:06.088 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:06.088 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.088 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 300 2024-04-20T01:48:06.088 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.088 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:06.088 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.088 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:06.088 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808516+0000", 2024-04-20T01:48:06.088 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.089 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.089 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:06.089 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808516+0000", 2024-04-20T01:48:06.089 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.089 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.089 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:06.089 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808515+0000", 2024-04-20T01:48:06.089 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:06.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.089 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.089 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:06.089 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808521+0000", 2024-04-20T01:48:06.089 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.8919999999999997e-06 2024-04-20T01:48:06.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.089 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.090 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:06.090 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808528+0000", 2024-04-20T01:48:06.090 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0190000000000001e-06 2024-04-20T01:48:06.090 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.090 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.090 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.090 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808534+0000", 2024-04-20T01:48:06.090 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.891e-06 2024-04-20T01:48:06.090 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.090 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.090 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.090 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.809751+0000", 2024-04-20T01:48:06.090 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0012163580000000001 2024-04-20T01:48:06.091 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.091 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.091 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:06.091 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.809758+0000", 2024-04-20T01:48:06.091 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.6580000000000002e-06 2024-04-20T01:48:06.091 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.091 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.091 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.091 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819973+0000", 2024-04-20T01:48:06.091 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010214983 2024-04-20T01:48:06.091 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.091 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.091 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:06.091 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.820076+0000", 2024-04-20T01:48:06.091 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00010318 2024-04-20T01:48:06.091 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.092 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.092 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.092 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.820519+0000", 2024-04-20T01:48:06.092 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00044209799999999998 2024-04-20T01:48:06.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.092 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.092 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.092 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.821591+0000", 2024-04-20T01:48:06.092 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001072677 2024-04-20T01:48:06.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.092 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.092 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:06.092 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.824619+0000", 2024-04-20T01:48:06.092 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0030279170000000002 2024-04-20T01:48:06.093 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.093 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.093 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:06.093 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.824692+0000", 2024-04-20T01:48:06.093 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.3220999999999996e-05 2024-04-20T01:48:06.093 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.093 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.093 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.093 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.093 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.093 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:302 3.2s0 3:4cb85a84:::smithi16579500-29:head [copy-from ver 10] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:06.093 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.823551+0000", 2024-04-20T01:48:06.093 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 22.208506597, 2024-04-20T01:48:06.093 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.014229169999999999, 2024-04-20T01:48:06.093 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:06.093 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:06.093 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:06.094 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:06.094 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.094 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 302 2024-04-20T01:48:06.094 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.094 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:06.094 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.094 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:06.094 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823551+0000", 2024-04-20T01:48:06.094 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.094 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.094 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.094 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:06.094 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823551+0000", 2024-04-20T01:48:06.094 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.094 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.095 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.095 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:06.095 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823550+0000", 2024-04-20T01:48:06.095 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:06.095 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.095 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.095 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:06.095 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823556+0000", 2024-04-20T01:48:06.095 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.455e-06 2024-04-20T01:48:06.095 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.095 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.095 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:06.095 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823568+0000", 2024-04-20T01:48:06.095 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1921e-05 2024-04-20T01:48:06.095 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.095 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.095 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.096 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823575+0000", 2024-04-20T01:48:06.096 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.1169999999999998e-06 2024-04-20T01:48:06.096 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.096 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.096 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.096 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823617+0000", 2024-04-20T01:48:06.096 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2012999999999999e-05 2024-04-20T01:48:06.096 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.096 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.096 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:06.096 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823741+0000", 2024-04-20T01:48:06.096 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000123454 2024-04-20T01:48:06.096 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.096 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.096 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.096 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.829454+0000", 2024-04-20T01:48:06.096 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0057129570000000003 2024-04-20T01:48:06.096 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.097 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.097 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.097 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.831037+0000", 2024-04-20T01:48:06.097 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0015832559999999999 2024-04-20T01:48:06.097 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.097 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.097 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:06.097 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.837719+0000", 2024-04-20T01:48:06.097 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.006681953 2024-04-20T01:48:06.097 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.097 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.097 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:06.097 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.837780+0000", 2024-04-20T01:48:06.097 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.1561000000000002e-05 2024-04-20T01:48:06.097 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.097 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.098 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.098 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.098 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.098 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:403 3.2s0 3:44d9544b:::smithi16579500-38:head [append 0~638976] snapc d=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-04-20T01:48:06.098 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.784120+0000", 2024-04-20T01:48:06.098 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 16.247938075, 2024-04-20T01:48:06.098 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.014614749999999999, 2024-04-20T01:48:06.098 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:06.098 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:06.098 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:06.098 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:06.098 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.098 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 403 2024-04-20T01:48:06.099 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.099 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:06.099 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.099 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:06.099 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.784120+0000", 2024-04-20T01:48:06.099 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.099 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.099 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.099 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:06.099 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.784120+0000", 2024-04-20T01:48:06.099 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.099 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.099 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.099 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:06.099 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.784118+0000", 2024-04-20T01:48:06.099 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:06.100 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.100 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.100 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:06.100 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788188+0000", 2024-04-20T01:48:06.100 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0040692860000000001 2024-04-20T01:48:06.100 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.100 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.100 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:06.100 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788201+0000", 2024-04-20T01:48:06.100 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3142999999999999e-05 2024-04-20T01:48:06.100 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.100 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.100 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.100 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788207+0000", 2024-04-20T01:48:06.100 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.4300000000000003e-06 2024-04-20T01:48:06.100 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.101 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.101 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.101 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788261+0000", 2024-04-20T01:48:06.101 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3257000000000002e-05 2024-04-20T01:48:06.101 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.101 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.101 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:06.101 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788365+0000", 2024-04-20T01:48:06.101 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000103928 2024-04-20T01:48:06.101 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.101 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.101 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.101 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.789323+0000", 2024-04-20T01:48:06.101 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00095858900000000003 2024-04-20T01:48:06.101 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.102 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.102 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.102 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.793128+0000", 2024-04-20T01:48:06.102 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.003805307 2024-04-20T01:48:06.102 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.102 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.102 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:06.102 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.798692+0000", 2024-04-20T01:48:06.102 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0055635160000000001 2024-04-20T01:48:06.102 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.102 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.102 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:06.102 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.798734+0000", 2024-04-20T01:48:06.102 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2453999999999997e-05 2024-04-20T01:48:06.102 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.103 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.103 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.103 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.103 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.103 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:404 3.2s0 3:44d9544b:::smithi16579500-38:head [append 0~622592] snapc d=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-04-20T01:48:06.103 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.788218+0000", 2024-04-20T01:48:06.103 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 16.243839775000001, 2024-04-20T01:48:06.103 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.012422041, 2024-04-20T01:48:06.103 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:06.103 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:06.103 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:06.103 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:06.103 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.104 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 404 2024-04-20T01:48:06.104 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.104 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:06.104 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.104 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:06.104 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788218+0000", 2024-04-20T01:48:06.104 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.104 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.104 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.104 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:06.104 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788218+0000", 2024-04-20T01:48:06.104 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.104 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.104 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.105 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:06.105 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788217+0000", 2024-04-20T01:48:06.105 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:06.105 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.105 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.105 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:06.105 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.790736+0000", 2024-04-20T01:48:06.105 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0025187270000000001 2024-04-20T01:48:06.105 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.105 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.105 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:06.105 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.790752+0000", 2024-04-20T01:48:06.105 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5684000000000001e-05 2024-04-20T01:48:06.105 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.105 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.105 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.106 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.790762+0000", 2024-04-20T01:48:06.106 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0693000000000001e-05 2024-04-20T01:48:06.106 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.106 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.106 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.106 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.790813+0000", 2024-04-20T01:48:06.106 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.0235000000000003e-05 2024-04-20T01:48:06.106 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.106 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.106 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:06.106 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.790933+0000", 2024-04-20T01:48:06.106 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000120417 2024-04-20T01:48:06.106 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.106 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.106 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.107 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.791589+0000", 2024-04-20T01:48:06.107 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000655972 2024-04-20T01:48:06.107 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.107 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.107 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.107 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.793913+0000", 2024-04-20T01:48:06.107 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.002323716 2024-04-20T01:48:06.107 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.107 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.107 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:06.107 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.800601+0000", 2024-04-20T01:48:06.107 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0066881880000000003 2024-04-20T01:48:06.107 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.108 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.108 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:06.108 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.800640+0000", 2024-04-20T01:48:06.108 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.9150999999999998e-05 2024-04-20T01:48:06.108 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.108 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.108 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.108 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.108 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.108 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:407 3.2s0 3:44d9544b:::smithi16579500-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e48)", 2024-04-20T01:48:06.108 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.791046+0000", 2024-04-20T01:48:06.108 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 16.241011921999998, 2024-04-20T01:48:06.108 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010908948999999999, 2024-04-20T01:48:06.108 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:06.109 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:06.109 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:06.109 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:06.109 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.109 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 407 2024-04-20T01:48:06.109 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.109 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:06.109 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.109 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:06.109 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.791046+0000", 2024-04-20T01:48:06.109 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.109 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.109 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.109 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:06.109 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.791046+0000", 2024-04-20T01:48:06.110 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.110 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.110 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.110 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:06.110 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.791045+0000", 2024-04-20T01:48:06.110 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:06.110 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.110 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.110 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:06.110 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.791050+0000", 2024-04-20T01:48:06.110 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.1289999999999998e-06 2024-04-20T01:48:06.110 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.111 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.111 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:06.111 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.791060+0000", 2024-04-20T01:48:06.111 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.082e-05 2024-04-20T01:48:06.111 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.111 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.111 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.111 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.791067+0000", 2024-04-20T01:48:06.111 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.2199999999999997e-06 2024-04-20T01:48:06.111 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.111 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.111 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.112 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.793972+0000", 2024-04-20T01:48:06.112 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0029050080000000002 2024-04-20T01:48:06.112 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.112 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.112 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:06.112 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.794067+0000", 2024-04-20T01:48:06.112 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.5637999999999995e-05 2024-04-20T01:48:06.112 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.112 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.112 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.112 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.801223+0000", 2024-04-20T01:48:06.112 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0071558150000000003 2024-04-20T01:48:06.112 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.112 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.113 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:06.113 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.801312+0000", 2024-04-20T01:48:06.113 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.9012999999999996e-05 2024-04-20T01:48:06.113 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.113 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.113 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:06.113 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.801955+0000", 2024-04-20T01:48:06.113 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00064270500000000003 2024-04-20T01:48:06.113 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.113 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.113 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.113 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.113 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.114 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:523 3.2s0 3:44d9544b:::smithi16579500-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-04-20T01:48:06.114 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:01.840230+0000", 2024-04-20T01:48:06.114 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 4.191827569, 2024-04-20T01:48:06.114 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010941562, 2024-04-20T01:48:06.114 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:06.114 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:06.114 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:06.114 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:06.114 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.114 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 523 2024-04-20T01:48:06.114 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.114 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:06.114 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.115 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:06.115 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.840230+0000", 2024-04-20T01:48:06.115 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.115 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.115 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.115 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:06.115 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.840230+0000", 2024-04-20T01:48:06.115 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.115 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.115 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.115 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:06.115 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.840230+0000", 2024-04-20T01:48:06.115 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:06.115 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.115 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.116 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:06.116 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.840237+0000", 2024-04-20T01:48:06.116 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.2309999999999999e-06 2024-04-20T01:48:06.116 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.116 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.116 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:06.116 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.840243+0000", 2024-04-20T01:48:06.116 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.0079999999999999e-06 2024-04-20T01:48:06.116 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.116 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.116 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.116 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.840251+0000", 2024-04-20T01:48:06.116 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.3019999999999995e-06 2024-04-20T01:48:06.116 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.116 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.116 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.116 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.843356+0000", 2024-04-20T01:48:06.117 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0031046419999999999 2024-04-20T01:48:06.117 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.117 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.117 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:06.117 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.843447+0000", 2024-04-20T01:48:06.117 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.1217000000000004e-05 2024-04-20T01:48:06.117 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.117 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.117 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.117 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.850113+0000", 2024-04-20T01:48:06.117 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0066663060000000003 2024-04-20T01:48:06.117 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.117 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.117 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:06.117 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.850210+0000", 2024-04-20T01:48:06.118 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6875999999999999e-05 2024-04-20T01:48:06.118 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.118 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.118 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:06.118 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.851172+0000", 2024-04-20T01:48:06.118 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00096138500000000004 2024-04-20T01:48:06.118 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.118 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.118 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.118 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.118 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.118 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:574 3.2s0 3:44d9544b:::smithi16579500-38:head [append 0~655360] snapc 1f=[1f,1b,1a,b] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-20T01:48:06.118 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:04.793176+0000", 2024-04-20T01:48:06.118 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 1.238881487, 2024-04-20T01:48:06.119 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.012353861000000001, 2024-04-20T01:48:06.119 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:06.119 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:06.119 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:06.119 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:06.119 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.119 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 574 2024-04-20T01:48:06.119 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.119 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:06.119 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.119 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:06.119 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793176+0000", 2024-04-20T01:48:06.119 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.119 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.119 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.120 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:06.120 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793176+0000", 2024-04-20T01:48:06.120 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:06.120 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.120 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.120 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:06.120 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793174+0000", 2024-04-20T01:48:06.120 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:06.120 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.120 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.120 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:06.120 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793851+0000", 2024-04-20T01:48:06.120 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00067681499999999999 2024-04-20T01:48:06.120 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.120 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.121 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:06.121 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793869+0000", 2024-04-20T01:48:06.121 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7569000000000001e-05 2024-04-20T01:48:06.121 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.121 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.121 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.121 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793878+0000", 2024-04-20T01:48:06.121 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8659999999999995e-06 2024-04-20T01:48:06.121 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.121 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.121 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:06.121 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793935+0000", 2024-04-20T01:48:06.121 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6984999999999997e-05 2024-04-20T01:48:06.121 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.122 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.122 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:06.122 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.794017+0000", 2024-04-20T01:48:06.122 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.2719000000000005e-05 2024-04-20T01:48:06.122 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.122 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.122 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.122 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.794790+0000", 2024-04-20T01:48:06.122 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000772495 2024-04-20T01:48:06.122 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.122 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.122 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.122 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.796798+0000", 2024-04-20T01:48:06.122 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0020083980000000002 2024-04-20T01:48:06.122 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.123 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.123 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:06.123 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.805483+0000", 2024-04-20T01:48:06.123 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0086847739999999993 2024-04-20T01:48:06.123 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:06.123 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:06.123 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:06.123 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.805530+0000", 2024-04-20T01:48:06.123 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.7042999999999997e-05 2024-04-20T01:48:06.123 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.123 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.123 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.123 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:06.123 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:06.123 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:06.125 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-20T01:48:06.126 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:06.140+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:06.226 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:06.241+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:06.273 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:06.273 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:06.273 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:48:06.273 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:06.283 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-20T01:48:06.327 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:06.341+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:06.412 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:06.412 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:06.412 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 30, 2024-04-20T01:48:06.412 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-20T01:48:06.412 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:06.422 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-20T01:48:06.428 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:06.443+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:06.529 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:06.544+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:06.551 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:06.551 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-20T01:48:06.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-20T01:48:06.552 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-20T01:48:06.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.552 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:229 3.fs0 3:fc9748e2:::smithi16579500-32:head [read 0~524288,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:06.552 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.787590+0000", 2024-04-20T01:48:06.552 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.777361933000002, 2024-04-20T01:48:06.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047169430000000004, 2024-04-20T01:48:06.552 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:06.552 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:06.552 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:06.552 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:06.552 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.552 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 229 2024-04-20T01:48:06.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.552 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:06.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:06.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787590+0000", 2024-04-20T01:48:06.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:06.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787590+0000", 2024-04-20T01:48:06.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:06.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787589+0000", 2024-04-20T01:48:06.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:06.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:06.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787599+0000", 2024-04-20T01:48:06.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0658e-05 2024-04-20T01:48:06.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:06.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787618+0000", 2024-04-20T01:48:06.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9066999999999999e-05 2024-04-20T01:48:06.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787629+0000", 2024-04-20T01:48:06.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0604999999999999e-05 2024-04-20T01:48:06.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787703+0000", 2024-04-20T01:48:06.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4544999999999999e-05 2024-04-20T01:48:06.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op must wait for map", 2024-04-20T01:48:06.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787713+0000", 2024-04-20T01:48:06.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9930000000000007e-06 2024-04-20T01:48:06.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.789878+0000", 2024-04-20T01:48:06.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021646809999999999 2024-04-20T01:48:06.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:06.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.789941+0000", 2024-04-20T01:48:06.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2558000000000007e-05 2024-04-20T01:48:06.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:06.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.792307+0000", 2024-04-20T01:48:06.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023666920000000001 2024-04-20T01:48:06.557 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.557 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.557 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.557 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:230 3.fs0 3:fc9748e2:::smithi16579500-32:head [read 0~524288,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:06.557 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.787639+0000", 2024-04-20T01:48:06.557 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.77731348, 2024-04-20T01:48:06.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052033440000000004, 2024-04-20T01:48:06.557 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:06.557 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:06.557 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:06.557 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:06.557 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.558 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 230 2024-04-20T01:48:06.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.558 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:06.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:06.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787639+0000", 2024-04-20T01:48:06.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:06.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787639+0000", 2024-04-20T01:48:06.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:06.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787638+0000", 2024-04-20T01:48:06.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:06.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:06.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787642+0000", 2024-04-20T01:48:06.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4539999999999999e-06 2024-04-20T01:48:06.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:06.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787646+0000", 2024-04-20T01:48:06.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1269999999999996e-06 2024-04-20T01:48:06.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787650+0000", 2024-04-20T01:48:06.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.467e-06 2024-04-20T01:48:06.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787796+0000", 2024-04-20T01:48:06.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014651000000000001 2024-04-20T01:48:06.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:06.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787801+0000", 2024-04-20T01:48:06.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5059999999999997e-06 2024-04-20T01:48:06.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.790292+0000", 2024-04-20T01:48:06.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024917329999999999 2024-04-20T01:48:06.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:06.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.790354+0000", 2024-04-20T01:48:06.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1407000000000003e-05 2024-04-20T01:48:06.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:06.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.792842+0000", 2024-04-20T01:48:06.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024885139999999998 2024-04-20T01:48:06.562 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.562 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.562 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.562 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:231 3.fs0 3:fc9748e2:::smithi16579500-32:head [read 0~524288,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:06.562 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.787695+0000", 2024-04-20T01:48:06.562 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.777257248000002, 2024-04-20T01:48:06.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0078083780000000004, 2024-04-20T01:48:06.562 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:06.562 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:06.562 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:06.562 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:06.563 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:06.563 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 231 2024-04-20T01:48:06.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.563 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:06.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:06.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787695+0000", 2024-04-20T01:48:06.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:06.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787695+0000", 2024-04-20T01:48:06.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:06.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787694+0000", 2024-04-20T01:48:06.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:06.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:06.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787699+0000", 2024-04-20T01:48:06.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8439999999999997e-06 2024-04-20T01:48:06.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:06.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787708+0000", 2024-04-20T01:48:06.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2679999999999994e-06 2024-04-20T01:48:06.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787715+0000", 2024-04-20T01:48:06.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8290000000000008e-06 2024-04-20T01:48:06.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787839+0000", 2024-04-20T01:48:06.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012379700000000001 2024-04-20T01:48:06.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:06.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787845+0000", 2024-04-20T01:48:06.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1249999999999998e-06 2024-04-20T01:48:06.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.793431+0000", 2024-04-20T01:48:06.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055859880000000001 2024-04-20T01:48:06.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:06.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.793524+0000", 2024-04-20T01:48:06.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2943999999999996e-05 2024-04-20T01:48:06.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:06.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.795503+0000", 2024-04-20T01:48:06.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019791520000000001 2024-04-20T01:48:06.566 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.566 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.566 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.566 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(osd.13.22:3 3.fs0 3:f3742432:::smithi16579500-42:head [assert-version v6,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:06.567 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.788849+0000", 2024-04-20T01:48:06.567 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.776103628000001, 2024-04-20T01:48:06.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004535295, 2024-04-20T01:48:06.567 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:06.567 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:06.567 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:06.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:06.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788849+0000", 2024-04-20T01:48:06.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:06.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788849+0000", 2024-04-20T01:48:06.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:06.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788847+0000", 2024-04-20T01:48:06.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:06.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:06.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788856+0000", 2024-04-20T01:48:06.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4139999999999994e-06 2024-04-20T01:48:06.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:06.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788868+0000", 2024-04-20T01:48:06.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2235999999999999e-05 2024-04-20T01:48:06.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788880+0000", 2024-04-20T01:48:06.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2122e-05 2024-04-20T01:48:06.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788930+0000", 2024-04-20T01:48:06.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0160000000000001e-05 2024-04-20T01:48:06.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op must wait for map", 2024-04-20T01:48:06.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788938+0000", 2024-04-20T01:48:06.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8269999999999998e-06 2024-04-20T01:48:06.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.791323+0000", 2024-04-20T01:48:06.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002384471 2024-04-20T01:48:06.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:06.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.791386+0000", 2024-04-20T01:48:06.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3707000000000005e-05 2024-04-20T01:48:06.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:06.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.793384+0000", 2024-04-20T01:48:06.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019975800000000001 2024-04-20T01:48:06.571 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.571 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.571 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.571 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.ds1 38/30 ECSubWrite(tid=19, reqid=osd.6.0:18, at_version=0'0, trim_to=0'0, roll_forward_to=38'10))", 2024-04-20T01:48:06.571 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:39.794291+0000", 2024-04-20T01:48:06.571 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.770661119, 2024-04-20T01:48:06.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068904140000000001, 2024-04-20T01:48:06.571 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:06.571 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:06.571 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:06.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:06.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794291+0000", 2024-04-20T01:48:06.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:06.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794291+0000", 2024-04-20T01:48:06.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:06.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794290+0000", 2024-04-20T01:48:06.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:06.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:06.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794317+0000", 2024-04-20T01:48:06.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7328999999999999e-05 2024-04-20T01:48:06.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:06.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794331+0000", 2024-04-20T01:48:06.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4076999999999999e-05 2024-04-20T01:48:06.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794339+0000", 2024-04-20T01:48:06.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6629999999999993e-06 2024-04-20T01:48:06.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794383+0000", 2024-04-20T01:48:06.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4106000000000001e-05 2024-04-20T01:48:06.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794390+0000", 2024-04-20T01:48:06.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5710000000000003e-06 2024-04-20T01:48:06.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.801146+0000", 2024-04-20T01:48:06.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0067565300000000002 2024-04-20T01:48:06.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:06.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.801182+0000", 2024-04-20T01:48:06.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5472999999999997e-05 2024-04-20T01:48:06.575 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.575 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.575 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.575 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 42/30 ECSubWrite(tid=17, reqid=client.4643.0:307, at_version=42'10, trim_to=0'0, roll_forward_to=33'8))", 2024-04-20T01:48:06.576 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.830042+0000", 2024-04-20T01:48:06.576 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 22.734910068000001, 2024-04-20T01:48:06.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0099127780000000006, 2024-04-20T01:48:06.576 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:06.576 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:06.576 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:06.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:06.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830042+0000", 2024-04-20T01:48:06.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:06.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830042+0000", 2024-04-20T01:48:06.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:06.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830042+0000", 2024-04-20T01:48:06.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:06.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:06.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830306+0000", 2024-04-20T01:48:06.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026456199999999998 2024-04-20T01:48:06.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:06.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830326+0000", 2024-04-20T01:48:06.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9919e-05 2024-04-20T01:48:06.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830337+0000", 2024-04-20T01:48:06.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0693000000000001e-05 2024-04-20T01:48:06.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830414+0000", 2024-04-20T01:48:06.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6916999999999996e-05 2024-04-20T01:48:06.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830426+0000", 2024-04-20T01:48:06.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2231e-05 2024-04-20T01:48:06.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.839923+0000", 2024-04-20T01:48:06.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094965540000000008 2024-04-20T01:48:06.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:06.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.839955+0000", 2024-04-20T01:48:06.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2299999999999999e-05 2024-04-20T01:48:06.579 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.579 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.579 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.580 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 42/30 ECSubWrite(tid=20, reqid=client.4643.0:311, at_version=42'13, trim_to=0'0, roll_forward_to=42'10))", 2024-04-20T01:48:06.580 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.922687+0000", 2024-04-20T01:48:06.580 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 22.642264852, 2024-04-20T01:48:06.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026580829999999999, 2024-04-20T01:48:06.580 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:06.580 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:06.580 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:06.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:06.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.922687+0000", 2024-04-20T01:48:06.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:06.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.922687+0000", 2024-04-20T01:48:06.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:06.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.922687+0000", 2024-04-20T01:48:06.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:06.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:06.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.922917+0000", 2024-04-20T01:48:06.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023064200000000001 2024-04-20T01:48:06.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:06.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.922933+0000", 2024-04-20T01:48:06.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5355000000000001e-05 2024-04-20T01:48:06.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.922942+0000", 2024-04-20T01:48:06.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5659999999999996e-06 2024-04-20T01:48:06.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.922985+0000", 2024-04-20T01:48:06.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2551e-05 2024-04-20T01:48:06.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.922991+0000", 2024-04-20T01:48:06.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3709999999999996e-06 2024-04-20T01:48:06.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.925313+0000", 2024-04-20T01:48:06.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023217759999999998 2024-04-20T01:48:06.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:06.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.925346+0000", 2024-04-20T01:48:06.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2469999999999999e-05 2024-04-20T01:48:06.584 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.584 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.584 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.584 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 42/30 ECSubWrite(tid=22, reqid=client.4643.0:314, at_version=42'16, trim_to=0'0, roll_forward_to=42'12))", 2024-04-20T01:48:06.584 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.928764+0000", 2024-04-20T01:48:06.584 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 22.636187835000001, 2024-04-20T01:48:06.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036527479999999999, 2024-04-20T01:48:06.584 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:06.584 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:06.585 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:06.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:06.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.928764+0000", 2024-04-20T01:48:06.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:06.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.928764+0000", 2024-04-20T01:48:06.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:06.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.928764+0000", 2024-04-20T01:48:06.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:06.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:06.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.929079+0000", 2024-04-20T01:48:06.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031541300000000001 2024-04-20T01:48:06.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:06.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.929097+0000", 2024-04-20T01:48:06.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.823e-05 2024-04-20T01:48:06.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.929108+0000", 2024-04-20T01:48:06.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0139000000000001e-05 2024-04-20T01:48:06.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.929158+0000", 2024-04-20T01:48:06.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.994e-05 2024-04-20T01:48:06.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.929169+0000", 2024-04-20T01:48:06.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1464000000000001e-05 2024-04-20T01:48:06.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.932389+0000", 2024-04-20T01:48:06.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032196299999999998 2024-04-20T01:48:06.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:06.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.932417+0000", 2024-04-20T01:48:06.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8620999999999999e-05 2024-04-20T01:48:06.588 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.588 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.588 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.588 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 47/30 ECSubWrite(tid=39, reqid=osd.5.0:39, at_version=47'27, trim_to=0'0, roll_forward_to=46'25))", 2024-04-20T01:48:06.588 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:48.718442+0000", 2024-04-20T01:48:06.589 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 17.846509986000001, 2024-04-20T01:48:06.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027161479999999998, 2024-04-20T01:48:06.589 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:06.589 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:06.589 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:06.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:06.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.718442+0000", 2024-04-20T01:48:06.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:06.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.718442+0000", 2024-04-20T01:48:06.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:06.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.718442+0000", 2024-04-20T01:48:06.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:06.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:06.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.718464+0000", 2024-04-20T01:48:06.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2195999999999999e-05 2024-04-20T01:48:06.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:06.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.718476+0000", 2024-04-20T01:48:06.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1761e-05 2024-04-20T01:48:06.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.718485+0000", 2024-04-20T01:48:06.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9490000000000002e-06 2024-04-20T01:48:06.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.718527+0000", 2024-04-20T01:48:06.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1631000000000002e-05 2024-04-20T01:48:06.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.718533+0000", 2024-04-20T01:48:06.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4189999999999999e-06 2024-04-20T01:48:06.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.721128+0000", 2024-04-20T01:48:06.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002594903 2024-04-20T01:48:06.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:06.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.721158+0000", 2024-04-20T01:48:06.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0550999999999999e-05 2024-04-20T01:48:06.592 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.592 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.592 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.593 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 47/30 ECSubWrite(tid=40, reqid=osd.5.0:40, at_version=47'29, trim_to=0'0, roll_forward_to=46'25))", 2024-04-20T01:48:06.593 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:48.719180+0000", 2024-04-20T01:48:06.593 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 17.845772612000001, 2024-04-20T01:48:06.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002933055, 2024-04-20T01:48:06.593 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:06.593 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:06.593 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:06.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:06.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.719180+0000", 2024-04-20T01:48:06.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:06.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.719180+0000", 2024-04-20T01:48:06.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:06.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.719179+0000", 2024-04-20T01:48:06.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:06.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:06.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.719204+0000", 2024-04-20T01:48:06.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5051000000000001e-05 2024-04-20T01:48:06.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:06.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.719216+0000", 2024-04-20T01:48:06.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2361000000000001e-05 2024-04-20T01:48:06.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.719226+0000", 2024-04-20T01:48:06.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8449999999999998e-06 2024-04-20T01:48:06.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.719280+0000", 2024-04-20T01:48:06.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3532999999999997e-05 2024-04-20T01:48:06.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.719289+0000", 2024-04-20T01:48:06.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9160000000000007e-06 2024-04-20T01:48:06.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722091+0000", 2024-04-20T01:48:06.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028020089999999998 2024-04-20T01:48:06.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:06.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722113+0000", 2024-04-20T01:48:06.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2172999999999999e-05 2024-04-20T01:48:06.597 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.597 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.597 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.597 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s1 47/30 ECSubWrite(tid=241, reqid=osd.13.0:241, at_version=47'20, trim_to=0'0, roll_forward_to=44'18))", 2024-04-20T01:48:06.597 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:48.722207+0000", 2024-04-20T01:48:06.597 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 17.842745343000001, 2024-04-20T01:48:06.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002400545, 2024-04-20T01:48:06.597 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:06.597 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:06.597 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:06.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:06.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722207+0000", 2024-04-20T01:48:06.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:06.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722207+0000", 2024-04-20T01:48:06.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:06.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722207+0000", 2024-04-20T01:48:06.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:48:06.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:06.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722229+0000", 2024-04-20T01:48:06.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2589e-05 2024-04-20T01:48:06.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:06.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722243+0000", 2024-04-20T01:48:06.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.379e-05 2024-04-20T01:48:06.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722252+0000", 2024-04-20T01:48:06.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0729999999999994e-06 2024-04-20T01:48:06.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722306+0000", 2024-04-20T01:48:06.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3819000000000001e-05 2024-04-20T01:48:06.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.722315+0000", 2024-04-20T01:48:06.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1770000000000004e-06 2024-04-20T01:48:06.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.724577+0000", 2024-04-20T01:48:06.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022613170000000001 2024-04-20T01:48:06.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:06.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.724608+0000", 2024-04-20T01:48:06.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1013000000000003e-05 2024-04-20T01:48:06.601 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.601 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.601 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.601 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 48/30 ECSubWrite(tid=44, reqid=osd.5.0:44, at_version=48'33, trim_to=0'0, roll_forward_to=47'31))", 2024-04-20T01:48:06.601 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.715966+0000", 2024-04-20T01:48:06.601 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.848986412999999, 2024-04-20T01:48:06.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045136789999999996, 2024-04-20T01:48:06.601 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:06.601 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:06.602 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:06.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:06.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.715966+0000", 2024-04-20T01:48:06.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:06.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.715966+0000", 2024-04-20T01:48:06.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:06.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.715966+0000", 2024-04-20T01:48:06.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:48:06.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:06.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.715986+0000", 2024-04-20T01:48:06.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.016e-05 2024-04-20T01:48:06.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:06.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.715995+0000", 2024-04-20T01:48:06.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5950000000000005e-06 2024-04-20T01:48:06.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716004+0000", 2024-04-20T01:48:06.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8480000000000007e-06 2024-04-20T01:48:06.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716039+0000", 2024-04-20T01:48:06.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4251000000000001e-05 2024-04-20T01:48:06.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716049+0000", 2024-04-20T01:48:06.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0276e-05 2024-04-20T01:48:06.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.605 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.605 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.720450+0000", 2024-04-20T01:48:06.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044011570000000002 2024-04-20T01:48:06.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.605 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:06.605 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.720480+0000", 2024-04-20T01:48:06.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9567999999999999e-05 2024-04-20T01:48:06.605 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.605 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.605 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.605 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.as6 48/30 ECSubWrite(tid=33, reqid=osd.6.0:33, at_version=48'14, trim_to=0'0, roll_forward_to=47'12))", 2024-04-20T01:48:06.605 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.716709+0000", 2024-04-20T01:48:06.605 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.848243621000002, 2024-04-20T01:48:06.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060784910000000001, 2024-04-20T01:48:06.606 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:06.606 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:06.606 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:06.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:06.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716709+0000", 2024-04-20T01:48:06.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:06.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716709+0000", 2024-04-20T01:48:06.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:06.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716709+0000", 2024-04-20T01:48:06.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:48:06.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:06.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716729+0000", 2024-04-20T01:48:06.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0735999999999999e-05 2024-04-20T01:48:06.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:06.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716740+0000", 2024-04-20T01:48:06.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0852e-05 2024-04-20T01:48:06.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716749+0000", 2024-04-20T01:48:06.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1700000000000003e-06 2024-04-20T01:48:06.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716830+0000", 2024-04-20T01:48:06.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0618999999999995e-05 2024-04-20T01:48:06.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716841+0000", 2024-04-20T01:48:06.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1421e-05 2024-04-20T01:48:06.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.722760+0000", 2024-04-20T01:48:06.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059184210000000001 2024-04-20T01:48:06.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:06.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.722787+0000", 2024-04-20T01:48:06.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7432000000000001e-05 2024-04-20T01:48:06.609 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.609 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.609 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.610 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s1 48/30 ECSubWrite(tid=254, reqid=osd.13.0:251, at_version=0'0, trim_to=0'0, roll_forward_to=48'22))", 2024-04-20T01:48:06.610 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.718102+0000", 2024-04-20T01:48:06.610 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.846850764999999, 2024-04-20T01:48:06.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024370220000000001, 2024-04-20T01:48:06.610 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:06.610 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:06.610 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:06.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:06.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718102+0000", 2024-04-20T01:48:06.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:06.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718102+0000", 2024-04-20T01:48:06.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:06.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718101+0000", 2024-04-20T01:48:06.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:06.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:06.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718112+0000", 2024-04-20T01:48:06.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1559999999999999e-05 2024-04-20T01:48:06.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:06.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718125+0000", 2024-04-20T01:48:06.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.292e-05 2024-04-20T01:48:06.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718134+0000", 2024-04-20T01:48:06.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1039999999999996e-06 2024-04-20T01:48:06.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718275+0000", 2024-04-20T01:48:06.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014071900000000001 2024-04-20T01:48:06.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.718286+0000", 2024-04-20T01:48:06.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1223000000000001e-05 2024-04-20T01:48:06.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.720516+0000", 2024-04-20T01:48:06.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022296769999999998 2024-04-20T01:48:06.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:06.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.720539+0000", 2024-04-20T01:48:06.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2639000000000001e-05 2024-04-20T01:48:06.613 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.613 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.613 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.614 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 48/30 ECSubWrite(tid=45, reqid=osd.5.0:44, at_version=0'0, trim_to=0'0, roll_forward_to=48'33))", 2024-04-20T01:48:06.614 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.726177+0000", 2024-04-20T01:48:06.614 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.838775441999999, 2024-04-20T01:48:06.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062928719999999997, 2024-04-20T01:48:06.614 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:06.614 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:06.614 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:06.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:06.614 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726177+0000", 2024-04-20T01:48:06.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:06.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726177+0000", 2024-04-20T01:48:06.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:06.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726176+0000", 2024-04-20T01:48:06.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:06.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:06.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726187+0000", 2024-04-20T01:48:06.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1216000000000001e-05 2024-04-20T01:48:06.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:06.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726202+0000", 2024-04-20T01:48:06.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4324999999999999e-05 2024-04-20T01:48:06.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726216+0000", 2024-04-20T01:48:06.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4294000000000001e-05 2024-04-20T01:48:06.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726289+0000", 2024-04-20T01:48:06.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3349000000000005e-05 2024-04-20T01:48:06.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726300+0000", 2024-04-20T01:48:06.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0190000000000001e-05 2024-04-20T01:48:06.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.732436+0000", 2024-04-20T01:48:06.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006136028 2024-04-20T01:48:06.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:06.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.732470+0000", 2024-04-20T01:48:06.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4167e-05 2024-04-20T01:48:06.617 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.617 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.618 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.618 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 58/30 ECSubWrite(tid=52, reqid=client.4643.0:489, at_version=58'41, trim_to=0'0, roll_forward_to=55'37))", 2024-04-20T01:48:06.618 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:59.769234+0000", 2024-04-20T01:48:06.618 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.7957185229999997, 2024-04-20T01:48:06.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033258430000000002, 2024-04-20T01:48:06.618 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:06.618 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:06.618 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:06.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:06.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:59.769234+0000", 2024-04-20T01:48:06.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:06.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:59.769234+0000", 2024-04-20T01:48:06.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:06.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:59.769233+0000", 2024-04-20T01:48:06.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:06.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:06.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:59.769317+0000", 2024-04-20T01:48:06.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3807000000000006e-05 2024-04-20T01:48:06.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.620 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:06.620 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:59.769329+0000", 2024-04-20T01:48:06.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1771e-05 2024-04-20T01:48:06.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.620 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.620 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:59.769336+0000", 2024-04-20T01:48:06.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8180000000000004e-06 2024-04-20T01:48:06.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.620 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:59.769367+0000", 2024-04-20T01:48:06.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0914000000000003e-05 2024-04-20T01:48:06.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:59.769373+0000", 2024-04-20T01:48:06.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9479999999999996e-06 2024-04-20T01:48:06.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:59.772537+0000", 2024-04-20T01:48:06.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031639250000000002 2024-04-20T01:48:06.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:06.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:59.772560+0000", 2024-04-20T01:48:06.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2393999999999999e-05 2024-04-20T01:48:06.622 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.622 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.622 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.622 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 58/30 ECSubWrite(tid=53, reqid=client.4643.0:490, at_version=58'42, trim_to=0'0, roll_forward_to=55'37))", 2024-04-20T01:48:06.622 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:59.770446+0000", 2024-04-20T01:48:06.622 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.7945061850000004, 2024-04-20T01:48:06.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024106689999999998, 2024-04-20T01:48:06.622 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:06.622 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:06.623 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:06.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:06.623 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:59.770446+0000", 2024-04-20T01:48:06.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:06.623 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:59.770446+0000", 2024-04-20T01:48:06.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:06.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:59.770446+0000", 2024-04-20T01:48:06.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:06.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:06.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:59.770459+0000", 2024-04-20T01:48:06.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3678e-05 2024-04-20T01:48:06.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:06.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:59.770470+0000", 2024-04-20T01:48:06.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0905e-05 2024-04-20T01:48:06.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:59.770476+0000", 2024-04-20T01:48:06.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0340000000000002e-06 2024-04-20T01:48:06.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:59.770519+0000", 2024-04-20T01:48:06.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2344e-05 2024-04-20T01:48:06.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:59.770525+0000", 2024-04-20T01:48:06.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4799999999999998e-06 2024-04-20T01:48:06.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:59.772837+0000", 2024-04-20T01:48:06.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023119260000000002 2024-04-20T01:48:06.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:06.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:59.772857+0000", 2024-04-20T01:48:06.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9830000000000002e-05 2024-04-20T01:48:06.626 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.626 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.627 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.627 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 60/30 ECSubWrite(tid=56, reqid=osd.5.0:56, at_version=60'44, trim_to=0'0, roll_forward_to=58'42))", 2024-04-20T01:48:06.627 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:01.745989+0000", 2024-04-20T01:48:06.627 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.8189629969999999, 2024-04-20T01:48:06.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031167019999999998, 2024-04-20T01:48:06.627 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:06.627 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:06.627 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:06.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.627 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:06.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.745989+0000", 2024-04-20T01:48:06.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:06.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.745989+0000", 2024-04-20T01:48:06.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:06.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.745989+0000", 2024-04-20T01:48:06.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:48:06.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:06.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.746022+0000", 2024-04-20T01:48:06.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3210999999999998e-05 2024-04-20T01:48:06.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:06.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.746037+0000", 2024-04-20T01:48:06.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4965e-05 2024-04-20T01:48:06.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.746049+0000", 2024-04-20T01:48:06.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1593999999999999e-05 2024-04-20T01:48:06.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.746116+0000", 2024-04-20T01:48:06.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7257999999999999e-05 2024-04-20T01:48:06.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.746133+0000", 2024-04-20T01:48:06.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6419999999999999e-05 2024-04-20T01:48:06.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.749074+0000", 2024-04-20T01:48:06.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029415829999999998 2024-04-20T01:48:06.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:06.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.749106+0000", 2024-04-20T01:48:06.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1896000000000001e-05 2024-04-20T01:48:06.631 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.631 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.631 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.631 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 60/30 ECSubWrite(tid=57, reqid=osd.5.0:56, at_version=0'0, trim_to=0'0, roll_forward_to=60'44))", 2024-04-20T01:48:06.631 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:01.752594+0000", 2024-04-20T01:48:06.631 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.8123583530000005, 2024-04-20T01:48:06.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048667290000000002, 2024-04-20T01:48:06.631 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:06.631 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:06.631 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:06.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:06.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.752594+0000", 2024-04-20T01:48:06.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:06.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.752594+0000", 2024-04-20T01:48:06.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:06.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.752593+0000", 2024-04-20T01:48:06.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:06.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:06.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.752604+0000", 2024-04-20T01:48:06.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0856000000000001e-05 2024-04-20T01:48:06.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:06.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.752616+0000", 2024-04-20T01:48:06.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1929e-05 2024-04-20T01:48:06.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.752637+0000", 2024-04-20T01:48:06.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1050999999999999e-05 2024-04-20T01:48:06.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.752686+0000", 2024-04-20T01:48:06.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8772e-05 2024-04-20T01:48:06.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.752696+0000", 2024-04-20T01:48:06.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0458999999999999e-05 2024-04-20T01:48:06.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.757429+0000", 2024-04-20T01:48:06.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047330180000000003 2024-04-20T01:48:06.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.635 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:06.635 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.757461+0000", 2024-04-20T01:48:06.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1248999999999998e-05 2024-04-20T01:48:06.635 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.635 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.635 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.635 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.6s5 63/30 ECSubWrite(tid=12, reqid=client.4643.0:573, at_version=63'7, trim_to=0'0, roll_forward_to=47'6))", 2024-04-20T01:48:06.635 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:04.766561+0000", 2024-04-20T01:48:06.635 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.7983911930000001, 2024-04-20T01:48:06.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025942729999999998, 2024-04-20T01:48:06.635 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:06.635 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:06.636 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:06.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:06.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.766561+0000", 2024-04-20T01:48:06.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:06.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.766561+0000", 2024-04-20T01:48:06.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:06.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:06.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.766561+0000", 2024-04-20T01:48:06.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:48:06.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:06.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.767026+0000", 2024-04-20T01:48:06.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00046512600000000001 2024-04-20T01:48:06.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:06.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.767041+0000", 2024-04-20T01:48:06.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5163e-05 2024-04-20T01:48:06.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:06.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.767054+0000", 2024-04-20T01:48:06.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.327e-05 2024-04-20T01:48:06.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:06.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.767122+0000", 2024-04-20T01:48:06.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7541999999999998e-05 2024-04-20T01:48:06.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:06.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.767131+0000", 2024-04-20T01:48:06.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6659999999999997e-06 2024-04-20T01:48:06.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:06.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.769113+0000", 2024-04-20T01:48:06.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019818399999999999 2024-04-20T01:48:06.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:06.639 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:06.639 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:06.639 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.769155+0000", 2024-04-20T01:48:06.639 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2876000000000001e-05 2024-04-20T01:48:06.639 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.639 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.639 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.639 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:06.639 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:06.639 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:06.641 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:06.655+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:06.641 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-20T01:48:06.742 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:06.756+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:06.746 INFO:tasks.rados.rados.0.smithi165.stdout:239: done (1 left) 2024-04-20T01:48:06.747 INFO:tasks.rados.rados.0.smithi165.stdout:240: delete oid 12 current snap is 18 2024-04-20T01:48:06.747 INFO:tasks.rados.rados.0.smithi165.stdout:238: expect (ObjNum 96 snap 16 seq_num 96) 2024-04-20T01:48:06.775 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:06.775 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:06.775 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:48:06.775 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:06.785 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-20T01:48:06.843 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:48:06.857+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:48:06.912 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:06.913 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:06.913 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 30, 2024-04-20T01:48:06.913 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-20T01:48:06.913 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:06.923 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-20T01:48:06.943 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:06.958+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:06.953 INFO:tasks.rados.rados.0.smithi165.stdout:238: done (1 left) 2024-04-20T01:48:06.953 INFO:tasks.rados.rados.0.smithi165.stdout:240: done (0 left) 2024-04-20T01:48:06.954 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 47 current snap is 18 2024-04-20T01:48:06.954 INFO:tasks.rados.rados.0.smithi165.stdout:241: seq_num 111 ranges {3031040=704512,3735552=655360,4390912=507904} 2024-04-20T01:48:06.969 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-20T01:48:06.973 INFO:tasks.rados.rados.0.smithi165.stdout:241: writing smithi16579500-47 from 3031040 to 3735552 tid 1 2024-04-20T01:48:06.976 INFO:tasks.rados.rados.0.smithi165.stdout:241: writing smithi16579500-47 from 3735552 to 4390912 tid 2 2024-04-20T01:48:06.980 INFO:tasks.rados.rados.0.smithi165.stdout:241: writing smithi16579500-47 from 4390912 to 4898816 tid 3 2024-04-20T01:48:06.980 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 24 current snap is 18 2024-04-20T01:48:06.980 INFO:tasks.rados.rados.0.smithi165.stdout:242: seq_num 112 ranges {3145728=720896,3866624=802816,4669440=360448} 2024-04-20T01:48:07.001 INFO:tasks.rados.rados.0.smithi165.stdout:242: writing smithi16579500-24 from 3145728 to 3866624 tid 1 2024-04-20T01:48:07.005 INFO:tasks.rados.rados.0.smithi165.stdout:242: writing smithi16579500-24 from 3866624 to 4669440 tid 2 2024-04-20T01:48:07.008 INFO:tasks.rados.rados.0.smithi165.stdout:242: writing smithi16579500-24 from 4669440 to 5029888 tid 3 2024-04-20T01:48:07.008 INFO:tasks.rados.rados.0.smithi165.stdout:243: read oid 10 snap -1 2024-04-20T01:48:07.008 INFO:tasks.rados.rados.0.smithi165.stdout:243: expect (ObjNum 9 snap 0 seq_num 9) 2024-04-20T01:48:07.008 INFO:tasks.rados.rados.0.smithi165.stdout:244: delete oid 37 current snap is 18 2024-04-20T01:48:07.008 INFO:tasks.rados.rados.0.smithi165.stdout:241: finishing write tid 1 to smithi16579500-47 2024-04-20T01:48:07.008 INFO:tasks.rados.rados.0.smithi165.stdout:241: finishing write tid 2 to smithi16579500-47 2024-04-20T01:48:07.009 INFO:tasks.rados.rados.0.smithi165.stdout:241: finishing write tid 3 to smithi16579500-47 2024-04-20T01:48:07.009 INFO:tasks.rados.rados.0.smithi165.stdout:241: finishing write tid 4 to smithi16579500-47 2024-04-20T01:48:07.009 INFO:tasks.rados.rados.0.smithi165.stdout:241: finishing write tid 5 to smithi16579500-47 2024-04-20T01:48:07.009 INFO:tasks.rados.rados.0.smithi165.stdout:241: oid 47 updating version 0 to 46 2024-04-20T01:48:07.009 INFO:tasks.rados.rados.0.smithi165.stdout:241: oid 47 updating version 46 to 47 2024-04-20T01:48:07.009 INFO:tasks.rados.rados.0.smithi165.stdout:241: oid 47 version 47 is already newer than 45 2024-04-20T01:48:07.009 INFO:tasks.rados.rados.0.smithi165.stdout:241: oid 47 updating version 47 to 48 2024-04-20T01:48:07.009 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 47 v 48 (ObjNum 111 snap 18 seq_num 111) dirty exists 2024-04-20T01:48:07.009 INFO:tasks.rados.rados.0.smithi165.stdout:241: left oid 47 (ObjNum 111 snap 18 seq_num 111) 2024-04-20T01:48:07.009 INFO:tasks.rados.rados.0.smithi165.stdout:242: finishing write tid 1 to smithi16579500-24 2024-04-20T01:48:07.010 INFO:tasks.rados.rados.0.smithi165.stdout:242: finishing write tid 2 to smithi16579500-24 2024-04-20T01:48:07.011 INFO:tasks.rados.rados.0.smithi165.stdout:241: done (3 left) 2024-04-20T01:48:07.011 INFO:tasks.rados.rados.0.smithi165.stdout:244: done (2 left) 2024-04-20T01:48:07.011 INFO:tasks.rados.rados.0.smithi165.stdout:245: snap_remove snap 5 2024-04-20T01:48:07.045 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:07.060+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:07.097 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:07.098 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-20T01:48:07.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-20T01:48:07.098 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-20T01:48:07.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.098 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.1s4 36/30 ECSubWrite(tid=56, reqid=osd.4.0:55, at_version=0'0, trim_to=0'0, roll_forward_to=36'8))", 2024-04-20T01:48:07.098 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:37.770698+0000", 2024-04-20T01:48:07.098 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 29.340474628999999, 2024-04-20T01:48:07.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051485360000000004, 2024-04-20T01:48:07.098 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:07.098 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:07.099 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:07.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:07.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:37.770698+0000", 2024-04-20T01:48:07.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:07.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:37.770698+0000", 2024-04-20T01:48:07.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:07.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:37.770697+0000", 2024-04-20T01:48:07.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:07.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:07.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:37.770715+0000", 2024-04-20T01:48:07.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7952e-05 2024-04-20T01:48:07.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:07.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:37.770736+0000", 2024-04-20T01:48:07.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0879000000000001e-05 2024-04-20T01:48:07.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:07.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:37.770748+0000", 2024-04-20T01:48:07.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2334000000000001e-05 2024-04-20T01:48:07.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:07.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:37.771313+0000", 2024-04-20T01:48:07.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00056510299999999996 2024-04-20T01:48:07.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:07.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:37.771320+0000", 2024-04-20T01:48:07.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0899999999999999e-06 2024-04-20T01:48:07.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:07.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:37.775828+0000", 2024-04-20T01:48:07.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045072549999999999 2024-04-20T01:48:07.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:07.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:37.775846+0000", 2024-04-20T01:48:07.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8743e-05 2024-04-20T01:48:07.102 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.102 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.102 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.102 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:243 3.ds0 3:b226b74f:::smithi16579500-7:head [delete] snapc 5=[5,2] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:07.102 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.787656+0000", 2024-04-20T01:48:07.103 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.323516511000001, 2024-04-20T01:48:07.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013205184, 2024-04-20T01:48:07.103 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:07.103 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:07.103 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:07.103 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:07.103 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:07.103 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 243 2024-04-20T01:48:07.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.103 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:07.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:07.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787656+0000", 2024-04-20T01:48:07.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:07.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787656+0000", 2024-04-20T01:48:07.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:07.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787654+0000", 2024-04-20T01:48:07.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:07.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:07.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787662+0000", 2024-04-20T01:48:07.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6599999999999995e-06 2024-04-20T01:48:07.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:07.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787680+0000", 2024-04-20T01:48:07.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7672999999999998e-05 2024-04-20T01:48:07.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:07.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787688+0000", 2024-04-20T01:48:07.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.405e-06 2024-04-20T01:48:07.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:07.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787739+0000", 2024-04-20T01:48:07.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0652999999999998e-05 2024-04-20T01:48:07.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:07.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787851+0000", 2024-04-20T01:48:07.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112172 2024-04-20T01:48:07.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:07.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788643+0000", 2024-04-20T01:48:07.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00079174300000000003 2024-04-20T01:48:07.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:07.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.789564+0000", 2024-04-20T01:48:07.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00092135199999999996 2024-04-20T01:48:07.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:07.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.800781+0000", 2024-04-20T01:48:07.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011217187999999999 2024-04-20T01:48:07.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:07.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.800861+0000", 2024-04-20T01:48:07.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9796999999999995e-05 2024-04-20T01:48:07.107 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.107 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.107 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.107 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 42/30 ECSubWrite(tid=229, reqid=client.4643.0:302, at_version=42'29, trim_to=0'0, roll_forward_to=42'27))", 2024-04-20T01:48:07.107 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.829719+0000", 2024-04-20T01:48:07.107 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.281453544000001, 2024-04-20T01:48:07.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045205979999999998, 2024-04-20T01:48:07.108 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:07.108 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:07.108 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:07.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:07.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.829719+0000", 2024-04-20T01:48:07.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:07.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.829719+0000", 2024-04-20T01:48:07.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:07.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.829718+0000", 2024-04-20T01:48:07.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:07.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:07.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830290+0000", 2024-04-20T01:48:07.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00057253800000000004 2024-04-20T01:48:07.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:07.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830321+0000", 2024-04-20T01:48:07.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0368000000000001e-05 2024-04-20T01:48:07.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:07.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830349+0000", 2024-04-20T01:48:07.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8342000000000001e-05 2024-04-20T01:48:07.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:07.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830435+0000", 2024-04-20T01:48:07.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6215e-05 2024-04-20T01:48:07.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:07.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830446+0000", 2024-04-20T01:48:07.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0883999999999999e-05 2024-04-20T01:48:07.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:07.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.834215+0000", 2024-04-20T01:48:07.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003768815 2024-04-20T01:48:07.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:07.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.834239+0000", 2024-04-20T01:48:07.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4433000000000001e-05 2024-04-20T01:48:07.111 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.111 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.111 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.111 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.6s4 42/30 ECSubWrite(tid=8, reqid=client.4643.0:331, at_version=42'4, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:48:07.112 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.957689+0000", 2024-04-20T01:48:07.112 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.153482936, 2024-04-20T01:48:07.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004764936, 2024-04-20T01:48:07.112 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:07.112 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:07.112 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:07.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:07.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957689+0000", 2024-04-20T01:48:07.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:07.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957689+0000", 2024-04-20T01:48:07.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:07.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957689+0000", 2024-04-20T01:48:07.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:48:07.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:07.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957705+0000", 2024-04-20T01:48:07.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5418999999999999e-05 2024-04-20T01:48:07.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:07.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957716+0000", 2024-04-20T01:48:07.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0815999999999999e-05 2024-04-20T01:48:07.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.114 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:07.114 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957727+0000", 2024-04-20T01:48:07.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.132e-05 2024-04-20T01:48:07.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.114 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:07.114 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957787+0000", 2024-04-20T01:48:07.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9734000000000001e-05 2024-04-20T01:48:07.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.114 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:07.114 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957799+0000", 2024-04-20T01:48:07.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.27e-05 2024-04-20T01:48:07.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:07.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.962434+0000", 2024-04-20T01:48:07.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046348989999999996 2024-04-20T01:48:07.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:07.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.962454+0000", 2024-04-20T01:48:07.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0196000000000001e-05 2024-04-20T01:48:07.115 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.115 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.115 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.116 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.cs2 42/30 ECSubWrite(tid=29, reqid=client.4643.0:328, at_version=42'24, trim_to=0'0, roll_forward_to=41'19))", 2024-04-20T01:48:07.116 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.957897+0000", 2024-04-20T01:48:07.116 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.153275300000001, 2024-04-20T01:48:07.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045531, 2024-04-20T01:48:07.116 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:07.116 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:07.116 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:07.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:07.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957897+0000", 2024-04-20T01:48:07.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:07.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957897+0000", 2024-04-20T01:48:07.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:07.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957896+0000", 2024-04-20T01:48:07.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:07.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:07.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957916+0000", 2024-04-20T01:48:07.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9996e-05 2024-04-20T01:48:07.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:07.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957928+0000", 2024-04-20T01:48:07.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1773000000000001e-05 2024-04-20T01:48:07.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:07.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957952+0000", 2024-04-20T01:48:07.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3706e-05 2024-04-20T01:48:07.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:07.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957995+0000", 2024-04-20T01:48:07.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3075999999999999e-05 2024-04-20T01:48:07.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:07.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.958004+0000", 2024-04-20T01:48:07.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7280000000000001e-06 2024-04-20T01:48:07.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:07.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.962426+0000", 2024-04-20T01:48:07.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004422863 2024-04-20T01:48:07.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:07.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.962450+0000", 2024-04-20T01:48:07.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3740999999999999e-05 2024-04-20T01:48:07.119 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.120 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.120 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.120 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 47/30 ECSubWrite(tid=248, reqid=client.4643.0:390, at_version=47'40, trim_to=0'0, roll_forward_to=47'35))", 2024-04-20T01:48:07.120 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:48.747744+0000", 2024-04-20T01:48:07.120 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.363428390999999, 2024-04-20T01:48:07.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045665469999999998, 2024-04-20T01:48:07.120 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:07.120 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:07.120 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:07.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:07.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747744+0000", 2024-04-20T01:48:07.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:07.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747744+0000", 2024-04-20T01:48:07.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:07.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747743+0000", 2024-04-20T01:48:07.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:07.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:07.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747771+0000", 2024-04-20T01:48:07.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8138000000000001e-05 2024-04-20T01:48:07.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:07.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747786+0000", 2024-04-20T01:48:07.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4598e-05 2024-04-20T01:48:07.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:07.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747796+0000", 2024-04-20T01:48:07.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0582e-05 2024-04-20T01:48:07.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:07.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747870+0000", 2024-04-20T01:48:07.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3578000000000006e-05 2024-04-20T01:48:07.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:07.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747882+0000", 2024-04-20T01:48:07.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1755e-05 2024-04-20T01:48:07.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:07.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.752288+0000", 2024-04-20T01:48:07.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044065190000000002 2024-04-20T01:48:07.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:07.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.752311+0000", 2024-04-20T01:48:07.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2331e-05 2024-04-20T01:48:07.124 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.124 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.124 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.124 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 48/30 ECSubWrite(tid=253, reqid=osd.13.0:253, at_version=48'44, trim_to=0'0, roll_forward_to=47'40))", 2024-04-20T01:48:07.124 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.711468+0000", 2024-04-20T01:48:07.124 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 17.399704782000001, 2024-04-20T01:48:07.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056178349999999998, 2024-04-20T01:48:07.124 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:07.124 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:07.124 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:07.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:07.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711468+0000", 2024-04-20T01:48:07.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:07.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711468+0000", 2024-04-20T01:48:07.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:07.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711467+0000", 2024-04-20T01:48:07.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:07.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:07.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711491+0000", 2024-04-20T01:48:07.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4023e-05 2024-04-20T01:48:07.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:07.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711503+0000", 2024-04-20T01:48:07.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2065000000000001e-05 2024-04-20T01:48:07.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:07.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711513+0000", 2024-04-20T01:48:07.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8509999999999994e-06 2024-04-20T01:48:07.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:07.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711561+0000", 2024-04-20T01:48:07.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8208999999999999e-05 2024-04-20T01:48:07.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:07.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711573+0000", 2024-04-20T01:48:07.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1955e-05 2024-04-20T01:48:07.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:07.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717067+0000", 2024-04-20T01:48:07.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054938909999999999 2024-04-20T01:48:07.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:07.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717085+0000", 2024-04-20T01:48:07.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8660000000000001e-05 2024-04-20T01:48:07.128 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.128 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.128 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.128 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.1s4 48/30 ECSubWrite(tid=119, reqid=osd.4.0:119, at_version=48'14, trim_to=0'0, roll_forward_to=47'12))", 2024-04-20T01:48:07.128 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.717339+0000", 2024-04-20T01:48:07.128 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 17.393833824000001, 2024-04-20T01:48:07.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0079470240000000004, 2024-04-20T01:48:07.128 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:07.129 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:07.129 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:07.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:07.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717339+0000", 2024-04-20T01:48:07.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:07.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717339+0000", 2024-04-20T01:48:07.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:07.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717338+0000", 2024-04-20T01:48:07.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:07.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:07.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717357+0000", 2024-04-20T01:48:07.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8575000000000001e-05 2024-04-20T01:48:07.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:07.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717368+0000", 2024-04-20T01:48:07.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.145e-05 2024-04-20T01:48:07.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:07.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717377+0000", 2024-04-20T01:48:07.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4929999999999998e-06 2024-04-20T01:48:07.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:07.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717424+0000", 2024-04-20T01:48:07.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7240000000000002e-05 2024-04-20T01:48:07.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:07.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717435+0000", 2024-04-20T01:48:07.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0533000000000001e-05 2024-04-20T01:48:07.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:07.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.725244+0000", 2024-04-20T01:48:07.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0078092129999999997 2024-04-20T01:48:07.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.132 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:07.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.725286+0000", 2024-04-20T01:48:07.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1792999999999998e-05 2024-04-20T01:48:07.132 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.132 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.132 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.132 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 48/30 ECSubWrite(tid=256, reqid=client.4643.0:403, at_version=48'46, trim_to=0'0, roll_forward_to=48'44))", 2024-04-20T01:48:07.133 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.789490+0000", 2024-04-20T01:48:07.133 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 17.321682419999998, 2024-04-20T01:48:07.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005092793, 2024-04-20T01:48:07.133 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:07.133 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:07.133 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:07.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:07.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.789490+0000", 2024-04-20T01:48:07.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:07.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.789490+0000", 2024-04-20T01:48:07.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:07.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.789489+0000", 2024-04-20T01:48:07.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:07.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:07.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.789916+0000", 2024-04-20T01:48:07.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00042631400000000001 2024-04-20T01:48:07.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:07.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.789930+0000", 2024-04-20T01:48:07.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4201e-05 2024-04-20T01:48:07.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:07.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.789938+0000", 2024-04-20T01:48:07.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2560000000000002e-06 2024-04-20T01:48:07.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:07.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.790005+0000", 2024-04-20T01:48:07.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.648e-05 2024-04-20T01:48:07.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:07.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.790029+0000", 2024-04-20T01:48:07.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.444e-05 2024-04-20T01:48:07.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:07.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.794559+0000", 2024-04-20T01:48:07.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045300560000000002 2024-04-20T01:48:07.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:07.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.794583+0000", 2024-04-20T01:48:07.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3666e-05 2024-04-20T01:48:07.136 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.136 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.136 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.137 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.1s4 60/30 ECSubWrite(tid=135, reqid=osd.4.0:135, at_version=60'18, trim_to=0'0, roll_forward_to=58'16))", 2024-04-20T01:48:07.137 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:01.742576+0000", 2024-04-20T01:48:07.137 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.3685966570000003, 2024-04-20T01:48:07.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057386049999999999, 2024-04-20T01:48:07.137 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:07.137 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:07.137 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:07.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:07.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.742576+0000", 2024-04-20T01:48:07.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:07.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.742576+0000", 2024-04-20T01:48:07.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:07.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.742575+0000", 2024-04-20T01:48:07.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:07.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:07.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.742610+0000", 2024-04-20T01:48:07.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4511999999999998e-05 2024-04-20T01:48:07.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:07.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.742625+0000", 2024-04-20T01:48:07.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5362999999999998e-05 2024-04-20T01:48:07.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:07.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.742642+0000", 2024-04-20T01:48:07.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7008e-05 2024-04-20T01:48:07.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:07.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.742709+0000", 2024-04-20T01:48:07.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7167000000000003e-05 2024-04-20T01:48:07.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:07.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.742725+0000", 2024-04-20T01:48:07.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5403000000000001e-05 2024-04-20T01:48:07.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:07.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.748283+0000", 2024-04-20T01:48:07.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055577760000000004 2024-04-20T01:48:07.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:07.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.748314+0000", 2024-04-20T01:48:07.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1736999999999998e-05 2024-04-20T01:48:07.140 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.140 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.141 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.141 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.1s4 60/30 ECSubWrite(tid=139, reqid=osd.4.0:135, at_version=0'0, trim_to=0'0, roll_forward_to=60'18))", 2024-04-20T01:48:07.141 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:01.748778+0000", 2024-04-20T01:48:07.141 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.3623947210000003, 2024-04-20T01:48:07.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005700887, 2024-04-20T01:48:07.141 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:07.141 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:07.141 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:07.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:07.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.748778+0000", 2024-04-20T01:48:07.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:07.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.748778+0000", 2024-04-20T01:48:07.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:07.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.748777+0000", 2024-04-20T01:48:07.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:07.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:07.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.748791+0000", 2024-04-20T01:48:07.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3801999999999999e-05 2024-04-20T01:48:07.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:07.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.748805+0000", 2024-04-20T01:48:07.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4783000000000001e-05 2024-04-20T01:48:07.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:07.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.748816+0000", 2024-04-20T01:48:07.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0375e-05 2024-04-20T01:48:07.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:07.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.748888+0000", 2024-04-20T01:48:07.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2706999999999994e-05 2024-04-20T01:48:07.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.144 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:07.144 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.748902+0000", 2024-04-20T01:48:07.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4005999999999999e-05 2024-04-20T01:48:07.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.144 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:07.144 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.754440+0000", 2024-04-20T01:48:07.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005537536 2024-04-20T01:48:07.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.144 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:07.144 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.754479+0000", 2024-04-20T01:48:07.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8674000000000003e-05 2024-04-20T01:48:07.145 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.145 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.145 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.145 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 60/30 ECSubWrite(tid=277, reqid=osd.13.0:277, at_version=60'63, trim_to=0'0, roll_forward_to=58'61))", 2024-04-20T01:48:07.145 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:01.750833+0000", 2024-04-20T01:48:07.145 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.3603391450000002, 2024-04-20T01:48:07.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006209689, 2024-04-20T01:48:07.145 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:07.145 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:07.145 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:07.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.145 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:07.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.750833+0000", 2024-04-20T01:48:07.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:07.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.750833+0000", 2024-04-20T01:48:07.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:07.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.750833+0000", 2024-04-20T01:48:07.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:48:07.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:07.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.750856+0000", 2024-04-20T01:48:07.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2988000000000001e-05 2024-04-20T01:48:07.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:07.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.750869+0000", 2024-04-20T01:48:07.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3268000000000001e-05 2024-04-20T01:48:07.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:07.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.750879+0000", 2024-04-20T01:48:07.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6180000000000001e-06 2024-04-20T01:48:07.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:07.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.750936+0000", 2024-04-20T01:48:07.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6592000000000003e-05 2024-04-20T01:48:07.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:07.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.750949+0000", 2024-04-20T01:48:07.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3067999999999999e-05 2024-04-20T01:48:07.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:07.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.757014+0000", 2024-04-20T01:48:07.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060654619999999998 2024-04-20T01:48:07.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:07.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.757043+0000", 2024-04-20T01:48:07.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8889000000000001e-05 2024-04-20T01:48:07.149 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.149 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.149 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.149 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 60/30 ECSubWrite(tid=279, reqid=client.4643.0:518, at_version=60'65, trim_to=0'0, roll_forward_to=60'63))", 2024-04-20T01:48:07.149 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:01.834859+0000", 2024-04-20T01:48:07.149 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.2763136939999997, 2024-04-20T01:48:07.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048447610000000004, 2024-04-20T01:48:07.149 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:07.149 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:07.149 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:07.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:07.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.834859+0000", 2024-04-20T01:48:07.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:07.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.834859+0000", 2024-04-20T01:48:07.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:07.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.834858+0000", 2024-04-20T01:48:07.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:07.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:07.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.835418+0000", 2024-04-20T01:48:07.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00056089900000000001 2024-04-20T01:48:07.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:07.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.835434+0000", 2024-04-20T01:48:07.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5449999999999999e-05 2024-04-20T01:48:07.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:07.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.835444+0000", 2024-04-20T01:48:07.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0046e-05 2024-04-20T01:48:07.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:07.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.835503+0000", 2024-04-20T01:48:07.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9432999999999998e-05 2024-04-20T01:48:07.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:07.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.835514+0000", 2024-04-20T01:48:07.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.041e-05 2024-04-20T01:48:07.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:07.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.839673+0000", 2024-04-20T01:48:07.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041593410000000004 2024-04-20T01:48:07.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:07.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.839703+0000", 2024-04-20T01:48:07.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0349e-05 2024-04-20T01:48:07.153 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.153 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.153 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.153 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 60/30 ECSubWrite(tid=280, reqid=client.4643.0:519, at_version=60'66, trim_to=0'0, roll_forward_to=60'63))", 2024-04-20T01:48:07.153 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:01.837460+0000", 2024-04-20T01:48:07.153 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.2737126190000003, 2024-04-20T01:48:07.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071325599999999996, 2024-04-20T01:48:07.154 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:07.154 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:07.154 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:07.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:07.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837460+0000", 2024-04-20T01:48:07.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:07.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837460+0000", 2024-04-20T01:48:07.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:07.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837459+0000", 2024-04-20T01:48:07.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:07.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:07.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837876+0000", 2024-04-20T01:48:07.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00041722999999999999 2024-04-20T01:48:07.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:07.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837893+0000", 2024-04-20T01:48:07.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6803000000000002e-05 2024-04-20T01:48:07.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.156 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:07.156 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837903+0000", 2024-04-20T01:48:07.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6830000000000003e-06 2024-04-20T01:48:07.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.156 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:07.156 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837955+0000", 2024-04-20T01:48:07.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2160000000000002e-05 2024-04-20T01:48:07.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.156 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:07.156 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837967+0000", 2024-04-20T01:48:07.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2057e-05 2024-04-20T01:48:07.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:07.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.844571+0000", 2024-04-20T01:48:07.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066042319999999998 2024-04-20T01:48:07.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:07.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.844592+0000", 2024-04-20T01:48:07.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1268999999999999e-05 2024-04-20T01:48:07.157 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.157 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.157 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.157 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 60/30 ECSubWrite(tid=281, reqid=client.4643.0:520, at_version=60'67, trim_to=0'0, roll_forward_to=60'63))", 2024-04-20T01:48:07.158 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:01.841236+0000", 2024-04-20T01:48:07.158 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.2699369110000003, 2024-04-20T01:48:07.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004775797, 2024-04-20T01:48:07.158 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:07.158 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:07.158 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:07.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:07.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.841236+0000", 2024-04-20T01:48:07.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:07.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.841236+0000", 2024-04-20T01:48:07.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:07.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.841235+0000", 2024-04-20T01:48:07.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:07.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:07.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.841726+0000", 2024-04-20T01:48:07.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00049109499999999999 2024-04-20T01:48:07.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:07.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.841740+0000", 2024-04-20T01:48:07.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3903000000000001e-05 2024-04-20T01:48:07.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:07.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.841747+0000", 2024-04-20T01:48:07.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3420000000000002e-06 2024-04-20T01:48:07.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:07.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.841802+0000", 2024-04-20T01:48:07.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4676000000000003e-05 2024-04-20T01:48:07.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:07.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.841816+0000", 2024-04-20T01:48:07.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3818999999999999e-05 2024-04-20T01:48:07.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:07.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.845984+0000", 2024-04-20T01:48:07.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004168092 2024-04-20T01:48:07.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:07.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.846011+0000", 2024-04-20T01:48:07.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7594999999999999e-05 2024-04-20T01:48:07.161 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.161 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.161 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.162 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 63/30 ECSubWrite(tid=295, reqid=client.4643.0:574, at_version=63'79, trim_to=0'0, roll_forward_to=62'77))", 2024-04-20T01:48:07.162 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:04.795046+0000", 2024-04-20T01:48:07.162 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 2.3161264959999999, 2024-04-20T01:48:07.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076412559999999999, 2024-04-20T01:48:07.162 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:07.162 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:07.162 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:07.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:07.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795046+0000", 2024-04-20T01:48:07.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:07.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795046+0000", 2024-04-20T01:48:07.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:07.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795044+0000", 2024-04-20T01:48:07.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:07.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:07.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795601+0000", 2024-04-20T01:48:07.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00055720399999999995 2024-04-20T01:48:07.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:07.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795617+0000", 2024-04-20T01:48:07.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5341000000000001e-05 2024-04-20T01:48:07.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:07.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795627+0000", 2024-04-20T01:48:07.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0429e-05 2024-04-20T01:48:07.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:07.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795692+0000", 2024-04-20T01:48:07.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4712e-05 2024-04-20T01:48:07.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:07.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795704+0000", 2024-04-20T01:48:07.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.258e-05 2024-04-20T01:48:07.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:07.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.802650+0000", 2024-04-20T01:48:07.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069452869999999996 2024-04-20T01:48:07.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:07.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.802687+0000", 2024-04-20T01:48:07.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7483e-05 2024-04-20T01:48:07.166 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.166 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.166 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.166 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 63/30 ECSubWrite(tid=296, reqid=client.4643.0:575, at_version=63'80, trim_to=0'0, roll_forward_to=62'77))", 2024-04-20T01:48:07.166 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:04.798415+0000", 2024-04-20T01:48:07.166 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 2.3127576969999999, 2024-04-20T01:48:07.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071008770000000002, 2024-04-20T01:48:07.166 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:07.166 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:07.166 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:07.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:07.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798415+0000", 2024-04-20T01:48:07.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:07.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798415+0000", 2024-04-20T01:48:07.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:07.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798414+0000", 2024-04-20T01:48:07.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:07.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:07.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798883+0000", 2024-04-20T01:48:07.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00046914899999999998 2024-04-20T01:48:07.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:07.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798898+0000", 2024-04-20T01:48:07.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4534e-05 2024-04-20T01:48:07.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:07.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798905+0000", 2024-04-20T01:48:07.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9149999999999996e-06 2024-04-20T01:48:07.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:07.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798951+0000", 2024-04-20T01:48:07.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5776999999999999e-05 2024-04-20T01:48:07.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:07.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798961+0000", 2024-04-20T01:48:07.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9289999999999993e-06 2024-04-20T01:48:07.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:07.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.805490+0000", 2024-04-20T01:48:07.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065291029999999996 2024-04-20T01:48:07.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.170 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:07.170 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.805516+0000", 2024-04-20T01:48:07.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5347999999999999e-05 2024-04-20T01:48:07.170 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.170 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.170 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.170 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 63/30 ECSubWrite(tid=297, reqid=client.4643.0:576, at_version=63'81, trim_to=0'0, roll_forward_to=62'77))", 2024-04-20T01:48:07.170 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:04.801003+0000", 2024-04-20T01:48:07.170 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 2.3101697319999999, 2024-04-20T01:48:07.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053881149999999997, 2024-04-20T01:48:07.170 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:07.170 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:07.170 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:07.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:07.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801003+0000", 2024-04-20T01:48:07.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:07.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801003+0000", 2024-04-20T01:48:07.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:07.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801002+0000", 2024-04-20T01:48:07.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:07.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:07.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801322+0000", 2024-04-20T01:48:07.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032022799999999998 2024-04-20T01:48:07.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:07.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801336+0000", 2024-04-20T01:48:07.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3978999999999999e-05 2024-04-20T01:48:07.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:07.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801344+0000", 2024-04-20T01:48:07.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9300000000000003e-06 2024-04-20T01:48:07.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:07.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801397+0000", 2024-04-20T01:48:07.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3279000000000003e-05 2024-04-20T01:48:07.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:07.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801409+0000", 2024-04-20T01:48:07.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.166e-05 2024-04-20T01:48:07.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:07.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.806359+0000", 2024-04-20T01:48:07.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049499330000000001 2024-04-20T01:48:07.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.174 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:07.174 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.806391+0000", 2024-04-20T01:48:07.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2051000000000002e-05 2024-04-20T01:48:07.174 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.174 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.174 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.174 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 63/30 ECSubWrite(tid=298, reqid=client.4643.0:577, at_version=63'82, trim_to=0'0, roll_forward_to=62'77))", 2024-04-20T01:48:07.174 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:04.801959+0000", 2024-04-20T01:48:07.174 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 2.3092136640000001, 2024-04-20T01:48:07.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044930100000000004, 2024-04-20T01:48:07.175 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:07.175 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:07.175 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:07.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:07.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801959+0000", 2024-04-20T01:48:07.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:07.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801959+0000", 2024-04-20T01:48:07.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:07.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801958+0000", 2024-04-20T01:48:07.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:07.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:07.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801999+0000", 2024-04-20T01:48:07.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0760000000000003e-05 2024-04-20T01:48:07.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:07.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.802012+0000", 2024-04-20T01:48:07.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3597e-05 2024-04-20T01:48:07.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:07.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.802020+0000", 2024-04-20T01:48:07.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6299999999999998e-06 2024-04-20T01:48:07.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:07.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.802746+0000", 2024-04-20T01:48:07.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00072600500000000005 2024-04-20T01:48:07.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:07.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.802757+0000", 2024-04-20T01:48:07.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1469e-05 2024-04-20T01:48:07.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:07.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.806434+0000", 2024-04-20T01:48:07.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036762320000000002 2024-04-20T01:48:07.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:07.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.806452+0000", 2024-04-20T01:48:07.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8165999999999999e-05 2024-04-20T01:48:07.178 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.178 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.178 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.179 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.cs2 65/30 ECSubWrite(tid=54, reqid=client.4643.0:587, at_version=65'45, trim_to=0'0, roll_forward_to=61'43))", 2024-04-20T01:48:07.179 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:06.990464+0000", 2024-04-20T01:48:07.179 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.120708471, 2024-04-20T01:48:07.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046284020000000002, 2024-04-20T01:48:07.179 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:07.179 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:07.179 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:07.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.179 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:07.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:06.990464+0000", 2024-04-20T01:48:07.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:07.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:06.990464+0000", 2024-04-20T01:48:07.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:07.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:07.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:06.990464+0000", 2024-04-20T01:48:07.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:07.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:07.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:06.990579+0000", 2024-04-20T01:48:07.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000115784 2024-04-20T01:48:07.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:07.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:06.990593+0000", 2024-04-20T01:48:07.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3172e-05 2024-04-20T01:48:07.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:07.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:06.990604+0000", 2024-04-20T01:48:07.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1375999999999999e-05 2024-04-20T01:48:07.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:07.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:06.990664+0000", 2024-04-20T01:48:07.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0087999999999998e-05 2024-04-20T01:48:07.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.182 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.182 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:07.182 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:06.990676+0000", 2024-04-20T01:48:07.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.241e-05 2024-04-20T01:48:07.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.182 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.182 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:07.182 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:06.995064+0000", 2024-04-20T01:48:07.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043875399999999997 2024-04-20T01:48:07.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:07.182 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:07.182 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:07.183 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:06.995092+0000", 2024-04-20T01:48:07.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8331e-05 2024-04-20T01:48:07.183 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.183 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.183 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.183 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:07.183 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:07.183 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:07.184 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:48:07.185 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:48:07.185 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:07.199+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:07.185 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-20T01:48:07.186 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:48:07.207 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:48:07.226 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:48:07.248 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:48:07.268 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:48:07.285 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:48:07.299+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:48:07.289 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:48:07.309 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:48:07.315 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:07.315 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:07.315 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:48:07.315 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:07.325 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-20T01:48:07.330 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:48:07.351 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:48:07.371 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:48:07.392 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:48:07.410 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:07.424+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:07.414 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:48:07.434 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:48:07.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:48:07.450+0000 7f8111925700 0 -- 172.21.15.44:0/2480573474 >> [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] conn(0x7f810c21fb50 msgr2=0x7f810c221f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-20T01:48:07.457 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:48:07.458 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:48:07.458 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:48:07.458 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:48:07.458 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:48:07.458 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:48:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:48:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:48:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:48:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:48:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:48:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:48:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:48:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:48:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:48:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:48:07.459 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:48:07.487 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:07.501+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:07.495 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:07.495 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:07.505 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-20T01:48:07.587 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:07.602+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:07.672 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:07.673 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:07.683 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-20T01:48:07.688 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:07.703+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:07.753 INFO:tasks.rados.rados.0.smithi165.stdout:245: done (2 left) 2024-04-20T01:48:07.753 INFO:tasks.rados.rados.0.smithi165.stdout:246: snap_remove snap 17 2024-04-20T01:48:07.768 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-20T01:48:07.789 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:07.803+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:07.823 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:07.823 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:07.834 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-20T01:48:07.890 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:07.904+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:07.963 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:07.963 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:07.973 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-20T01:48:08.017 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:08.031+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:08.092 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:08.106+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:08.101 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:08.101 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:08.112 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-20T01:48:08.194 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:08.208+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:08.245 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:08.245 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:08.255 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-20T01:48:08.294 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:08.309+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:08.384 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:08.385 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:08.395 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-20T01:48:08.395 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:08.409+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:08.496 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:48:08.510+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:48:08.523 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:08.523 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:08.533 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-20T01:48:08.597 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:08.611+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:08.660 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:08.661 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:08.670 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-20T01:48:08.714 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:08.729+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:08.756 INFO:tasks.rados.rados.0.smithi165.stdout:246: done (2 left) 2024-04-20T01:48:08.756 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 20 current snap is 18 2024-04-20T01:48:08.756 INFO:tasks.rados.rados.0.smithi165.stdout:247: seq_num 114 ranges {0=753664,753664=688128,1441792=475136} 2024-04-20T01:48:08.759 INFO:tasks.rados.rados.0.smithi165.stdout:247: writing smithi16579500-20 from 0 to 753664 tid 1 2024-04-20T01:48:08.760 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is set 2024-04-20T01:48:08.763 INFO:tasks.rados.rados.0.smithi165.stdout:247: writing smithi16579500-20 from 753664 to 1441792 tid 2 2024-04-20T01:48:08.766 INFO:tasks.rados.rados.0.smithi165.stdout:247: writing smithi16579500-20 from 1441792 to 1916928 tid 3 2024-04-20T01:48:08.766 INFO:tasks.rados.rados.0.smithi165.stdout:248: snap_remove snap 16 2024-04-20T01:48:08.799 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:08.813+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:08.804 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:08.804 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:08.814 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-20T01:48:08.900 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:48:08.914+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:48:08.983 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:08.983 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:08.993 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-20T01:48:09.001 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:09.014+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:09.101 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:09.115+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:09.164 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:09.165 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:09.175 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-20T01:48:09.218 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:48:09.233+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:48:09.303 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:09.303 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:09.304 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:09.318+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:09.313 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-20T01:48:09.404 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:09.419+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:09.472 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-20T01:48:09.483 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:09.484 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:09.493 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-20T01:48:09.505 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:09.519+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:09.605 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:48:09.620+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:48:09.642 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:48:09.661 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:09.662 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:09.663 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:48:09.672 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-20T01:48:09.685 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:48:09.706 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:48:09.706 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:09.720+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:09.726 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:48:09.747 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:48:09.772 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:48:09.772 INFO:tasks.rados.rados.0.smithi165.stdout:248: done (3 left) 2024-04-20T01:48:09.772 INFO:tasks.rados.rados.0.smithi165.stdout:249: copy_from oid 12 from oid 9 current snap is 18 2024-04-20T01:48:09.772 INFO:tasks.rados.rados.0.smithi165.stdout:242: finishing write tid 3 to smithi16579500-24 2024-04-20T01:48:09.772 INFO:tasks.rados.rados.0.smithi165.stdout:243: expect (ObjNum 9 snap 0 seq_num 9) 2024-04-20T01:48:09.776 INFO:tasks.rados.rados.0.smithi165.stdout:242: finishing write tid 4 to smithi16579500-24 2024-04-20T01:48:09.777 INFO:tasks.rados.rados.0.smithi165.stdout:242: finishing write tid 5 to smithi16579500-24 2024-04-20T01:48:09.777 INFO:tasks.rados.rados.0.smithi165.stdout:242: oid 24 updating version 0 to 58 2024-04-20T01:48:09.777 INFO:tasks.rados.rados.0.smithi165.stdout:242: oid 24 updating version 58 to 61 2024-04-20T01:48:09.777 INFO:tasks.rados.rados.0.smithi165.stdout:242: oid 24 version 61 is already newer than 59 2024-04-20T01:48:09.777 INFO:tasks.rados.rados.0.smithi165.stdout:242: oid 24 version 61 is already newer than 60 2024-04-20T01:48:09.777 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 24 v 61 (ObjNum 112 snap 18 seq_num 112) dirty exists 2024-04-20T01:48:09.777 INFO:tasks.rados.rados.0.smithi165.stdout:242: left oid 24 (ObjNum 112 snap 18 seq_num 112) 2024-04-20T01:48:09.777 INFO:tasks.rados.rados.0.smithi165.stdout:247: finishing write tid 1 to smithi16579500-20 2024-04-20T01:48:09.777 INFO:tasks.rados.rados.0.smithi165.stdout:247: finishing write tid 2 to smithi16579500-20 2024-04-20T01:48:09.777 INFO:tasks.rados.rados.0.smithi165.stdout:242: done (3 left) 2024-04-20T01:48:09.777 INFO:tasks.rados.rados.0.smithi165.stdout:243: done (2 left) 2024-04-20T01:48:09.777 INFO:tasks.rados.rados.0.smithi165.stdout:250: setattr oid 6 current snap is 18 2024-04-20T01:48:09.778 INFO:tasks.rados.rados.0.smithi165.stdout:247: finishing write tid 3 to smithi16579500-20 2024-04-20T01:48:09.778 INFO:tasks.rados.rados.0.smithi165.stdout:247: finishing write tid 4 to smithi16579500-20 2024-04-20T01:48:09.778 INFO:tasks.rados.rados.0.smithi165.stdout:247: finishing write tid 5 to smithi16579500-20 2024-04-20T01:48:09.778 INFO:tasks.rados.rados.0.smithi165.stdout:247: oid 20 updating version 0 to 99 2024-04-20T01:48:09.778 INFO:tasks.rados.rados.0.smithi165.stdout:247: oid 20 updating version 99 to 100 2024-04-20T01:48:09.778 INFO:tasks.rados.rados.0.smithi165.stdout:247: oid 20 version 100 is already newer than 98 2024-04-20T01:48:09.779 INFO:tasks.rados.rados.0.smithi165.stdout:247: oid 20 version 100 is already newer than 97 2024-04-20T01:48:09.779 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 20 v 100 (ObjNum 114 snap 18 seq_num 114) dirty exists 2024-04-20T01:48:09.779 INFO:tasks.rados.rados.0.smithi165.stdout:247: left oid 20 (ObjNum 114 snap 18 seq_num 114) 2024-04-20T01:48:09.779 INFO:tasks.rados.rados.0.smithi165.stdout:247: done (2 left) 2024-04-20T01:48:09.779 INFO:tasks.rados.rados.0.smithi165.stdout:251: rmattr oid 44 current snap is 18 2024-04-20T01:48:09.779 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 50 current snap is 18 2024-04-20T01:48:09.779 INFO:tasks.rados.rados.0.smithi165.stdout:252: seq_num 115 ranges {835584=770048,1605632=704512,2310144=458752} 2024-04-20T01:48:09.787 INFO:tasks.rados.rados.0.smithi165.stdout:252: writing smithi16579500-50 from 835584 to 1605632 tid 1 2024-04-20T01:48:09.795 INFO:tasks.rados.rados.0.smithi165.stdout:252: writing smithi16579500-50 from 1605632 to 2310144 tid 2 2024-04-20T01:48:09.795 INFO:tasks.rados.rados.0.smithi165.stdout:252: writing smithi16579500-50 from 2310144 to 2768896 tid 3 2024-04-20T01:48:09.796 INFO:tasks.rados.rados.0.smithi165.stdout:249: finishing copy_from racing read to smithi16579500-12 2024-04-20T01:48:09.796 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 44 v 14 (ObjNum 57 snap 2 seq_num 57) dirty exists 2024-04-20T01:48:09.798 INFO:tasks.rados.rados.0.smithi165.stdout:251: done (3 left) 2024-04-20T01:48:09.798 INFO:tasks.rados.rados.0.smithi165.stdout:253: rollback oid 8 current snap is 18 2024-04-20T01:48:09.798 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 8 to 14 2024-04-20T01:48:09.798 INFO:tasks.rados.rados.0.smithi165.stdout:254: rollback oid 37 current snap is 18 2024-04-20T01:48:09.798 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 37 to 14 2024-04-20T01:48:09.798 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 33 current snap is 18 2024-04-20T01:48:09.798 INFO:tasks.rados.rados.0.smithi165.stdout:255: seq_num 116 ranges {0=786432,786432=458752,1245184=442368,1687552=262144} 2024-04-20T01:48:09.801 INFO:tasks.rados.rados.0.smithi165.stdout:255: writing smithi16579500-33 from 0 to 786432 tid 1 2024-04-20T01:48:09.802 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:48:09.804 INFO:tasks.rados.rados.0.smithi165.stdout:255: writing smithi16579500-33 from 786432 to 1245184 tid 2 2024-04-20T01:48:09.807 INFO:tasks.rados.rados.0.smithi165.stdout:255: writing smithi16579500-33 from 1245184 to 1687552 tid 3 2024-04-20T01:48:09.807 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:09.820+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:09.808 INFO:tasks.rados.rados.0.smithi165.stdout:255: writing smithi16579500-33 from 1687552 to 1949696 tid 4 2024-04-20T01:48:09.809 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 27 current snap is 18 2024-04-20T01:48:09.809 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 6 v 26 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-04-20T01:48:09.809 INFO:tasks.rados.rados.0.smithi165.stdout:249: finishing copy_from to smithi16579500-12 2024-04-20T01:48:09.809 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 12 v 14 (ObjNum 68 snap 5 seq_num 68) dirty exists 2024-04-20T01:48:09.809 INFO:tasks.rados.rados.0.smithi165.stdout:252: finishing write tid 1 to smithi16579500-50 2024-04-20T01:48:09.809 INFO:tasks.rados.rados.0.smithi165.stdout:253: finishing rollback tid 0 to smithi16579500-8 2024-04-20T01:48:09.809 INFO:tasks.rados.rados.0.smithi165.stdout:253: finishing rollback tid 1 to smithi16579500-8 2024-04-20T01:48:09.809 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 8 v 9 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-04-20T01:48:09.809 INFO:tasks.rados.rados.0.smithi165.stdout:252: finishing write tid 2 to smithi16579500-50 2024-04-20T01:48:09.809 INFO:tasks.rados.rados.0.smithi165.stdout:252: finishing write tid 3 to smithi16579500-50 2024-04-20T01:48:09.809 INFO:tasks.rados.rados.0.smithi165.stdout:256: seq_num 117 ranges {0=425984,425984=606208,1032192=540672,1572864=409600} 2024-04-20T01:48:09.811 INFO:tasks.rados.rados.0.smithi165.stdout:256: writing smithi16579500-27 from 0 to 425984 tid 1 2024-04-20T01:48:09.814 INFO:tasks.rados.rados.0.smithi165.stdout:256: writing smithi16579500-27 from 425984 to 1032192 tid 2 2024-04-20T01:48:09.817 INFO:tasks.rados.rados.0.smithi165.stdout:256: writing smithi16579500-27 from 1032192 to 1572864 tid 3 2024-04-20T01:48:09.820 INFO:tasks.rados.rados.0.smithi165.stdout:256: writing smithi16579500-27 from 1572864 to 1982464 tid 4 2024-04-20T01:48:09.820 INFO:tasks.rados.rados.0.smithi165.stdout:249: done (6 left) 2024-04-20T01:48:09.820 INFO:tasks.rados.rados.0.smithi165.stdout:250: done (5 left) 2024-04-20T01:48:09.821 INFO:tasks.rados.rados.0.smithi165.stdout:253: done (4 left) 2024-04-20T01:48:09.821 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 36 current snap is 18 2024-04-20T01:48:09.821 INFO:tasks.rados.rados.0.smithi165.stdout:254: finishing rollback tid 1 to smithi16579500-37 2024-04-20T01:48:09.821 INFO:tasks.rados.rados.0.smithi165.stdout:257: seq_num 118 ranges {3342336=442368,3784704=753664,4538368=655360,5193728=147456} 2024-04-20T01:48:09.826 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:48:09.841 INFO:tasks.rados.rados.0.smithi165.stdout:257: writing smithi16579500-36 from 3342336 to 3784704 tid 1 2024-04-20T01:48:09.845 INFO:tasks.rados.rados.0.smithi165.stdout:257: writing smithi16579500-36 from 3784704 to 4538368 tid 2 2024-04-20T01:48:09.847 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:48:09.851 INFO:tasks.rados.rados.0.smithi165.stdout:257: writing smithi16579500-36 from 4538368 to 5193728 tid 3 2024-04-20T01:48:09.851 INFO:tasks.rados.rados.0.smithi165.stdout:257: writing smithi16579500-36 from 5193728 to 5341184 tid 4 2024-04-20T01:48:09.851 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 8 current snap is 18 2024-04-20T01:48:09.851 INFO:tasks.rados.rados.0.smithi165.stdout:254: finishing rollback tid 2 to smithi16579500-37 2024-04-20T01:48:09.851 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 37 v 58 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-04-20T01:48:09.851 INFO:tasks.rados.rados.0.smithi165.stdout:252: finishing write tid 4 to smithi16579500-50 2024-04-20T01:48:09.851 INFO:tasks.rados.rados.0.smithi165.stdout:255: finishing write tid 1 to smithi16579500-33 2024-04-20T01:48:09.851 INFO:tasks.rados.rados.0.smithi165.stdout:252: finishing write tid 5 to smithi16579500-50 2024-04-20T01:48:09.852 INFO:tasks.rados.rados.0.smithi165.stdout:252: oid 50 updating version 0 to 59 2024-04-20T01:48:09.852 INFO:tasks.rados.rados.0.smithi165.stdout:252: oid 50 version 59 is already newer than 58 2024-04-20T01:48:09.852 INFO:tasks.rados.rados.0.smithi165.stdout:252: oid 50 updating version 59 to 61 2024-04-20T01:48:09.852 INFO:tasks.rados.rados.0.smithi165.stdout:252: oid 50 version 61 is already newer than 60 2024-04-20T01:48:09.852 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 50 v 61 (ObjNum 115 snap 18 seq_num 115) dirty exists 2024-04-20T01:48:09.852 INFO:tasks.rados.rados.0.smithi165.stdout:252: left oid 50 (ObjNum 115 snap 18 seq_num 115) 2024-04-20T01:48:09.852 INFO:tasks.rados.rados.0.smithi165.stdout:255: finishing write tid 2 to smithi16579500-33 2024-04-20T01:48:09.852 INFO:tasks.rados.rados.0.smithi165.stdout:255: finishing write tid 3 to smithi16579500-33 2024-04-20T01:48:09.852 INFO:tasks.rados.rados.0.smithi165.stdout:255: finishing write tid 4 to smithi16579500-33 2024-04-20T01:48:09.852 INFO:tasks.rados.rados.0.smithi165.stdout:255: finishing write tid 5 to smithi16579500-33 2024-04-20T01:48:09.852 INFO:tasks.rados.rados.0.smithi165.stdout:256: finishing write tid 1 to smithi16579500-27 2024-04-20T01:48:09.852 INFO:tasks.rados.rados.0.smithi165.stdout:256: finishing write tid 2 to smithi16579500-27 2024-04-20T01:48:09.852 INFO:tasks.rados.rados.0.smithi165.stdout:258: seq_num 119 ranges {0=458752,458752=507904,966656=507904,1474560=458752,1933312=81920} 2024-04-20T01:48:09.853 INFO:tasks.rados.rados.0.smithi165.stdout:258: writing smithi16579500-8 from 0 to 458752 tid 1 2024-04-20T01:48:09.855 INFO:tasks.rados.rados.0.smithi165.stdout:258: writing smithi16579500-8 from 458752 to 966656 tid 2 2024-04-20T01:48:09.858 INFO:tasks.rados.rados.0.smithi165.stdout:258: writing smithi16579500-8 from 966656 to 1474560 tid 3 2024-04-20T01:48:09.861 INFO:tasks.rados.rados.0.smithi165.stdout:258: writing smithi16579500-8 from 1474560 to 1933312 tid 4 2024-04-20T01:48:09.861 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:09.861 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:09.863 INFO:tasks.rados.rados.0.smithi165.stdout:258: writing smithi16579500-8 from 1933312 to 2015232 tid 5 2024-04-20T01:48:09.863 INFO:tasks.rados.rados.0.smithi165.stdout:252: done (5 left) 2024-04-20T01:48:09.863 INFO:tasks.rados.rados.0.smithi165.stdout:254: done (4 left) 2024-04-20T01:48:09.863 INFO:tasks.rados.rados.0.smithi165.stdout:259: rollback oid 47 current snap is 18 2024-04-20T01:48:09.863 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 47 to 14 2024-04-20T01:48:09.863 INFO:tasks.rados.rados.0.smithi165.stdout:255: finishing write tid 6 to smithi16579500-33 2024-04-20T01:48:09.863 INFO:tasks.rados.rados.0.smithi165.stdout:255: oid 33 updating version 0 to 100 2024-04-20T01:48:09.863 INFO:tasks.rados.rados.0.smithi165.stdout:255: oid 33 version 100 is already newer than 97 2024-04-20T01:48:09.863 INFO:tasks.rados.rados.0.smithi165.stdout:255: oid 33 version 100 is already newer than 96 2024-04-20T01:48:09.863 INFO:tasks.rados.rados.0.smithi165.stdout:255: oid 33 version 100 is already newer than 99 2024-04-20T01:48:09.863 INFO:tasks.rados.rados.0.smithi165.stdout:255: oid 33 version 100 is already newer than 98 2024-04-20T01:48:09.864 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 33 v 100 (ObjNum 116 snap 18 seq_num 116) dirty exists 2024-04-20T01:48:09.864 INFO:tasks.rados.rados.0.smithi165.stdout:255: left oid 33 (ObjNum 116 snap 18 seq_num 116) 2024-04-20T01:48:09.864 INFO:tasks.rados.rados.0.smithi165.stdout:256: finishing write tid 3 to smithi16579500-27 2024-04-20T01:48:09.864 INFO:tasks.rados.rados.0.smithi165.stdout:255: done (4 left) 2024-04-20T01:48:09.864 INFO:tasks.rados.rados.0.smithi165.stdout:260: copy_from oid 34 from oid 35 current snap is 18 2024-04-20T01:48:09.864 INFO:tasks.rados.rados.0.smithi165.stdout:256: finishing write tid 4 to smithi16579500-27 2024-04-20T01:48:09.864 INFO:tasks.rados.rados.0.smithi165.stdout:256: finishing write tid 5 to smithi16579500-27 2024-04-20T01:48:09.864 INFO:tasks.rados.rados.0.smithi165.stdout:256: finishing write tid 6 to smithi16579500-27 2024-04-20T01:48:09.864 INFO:tasks.rados.rados.0.smithi165.stdout:256: oid 27 updating version 0 to 28 2024-04-20T01:48:09.864 INFO:tasks.rados.rados.0.smithi165.stdout:256: oid 27 updating version 28 to 31 2024-04-20T01:48:09.864 INFO:tasks.rados.rados.0.smithi165.stdout:256: oid 27 version 31 is already newer than 29 2024-04-20T01:48:09.864 INFO:tasks.rados.rados.0.smithi165.stdout:256: oid 27 updating version 31 to 32 2024-04-20T01:48:09.864 INFO:tasks.rados.rados.0.smithi165.stdout:256: oid 27 version 32 is already newer than 30 2024-04-20T01:48:09.864 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 27 v 32 (ObjNum 117 snap 18 seq_num 117) dirty exists 2024-04-20T01:48:09.865 INFO:tasks.rados.rados.0.smithi165.stdout:256: left oid 27 (ObjNum 117 snap 18 seq_num 117) 2024-04-20T01:48:09.865 INFO:tasks.rados.rados.0.smithi165.stdout:257: finishing write tid 1 to smithi16579500-36 2024-04-20T01:48:09.865 INFO:tasks.rados.rados.0.smithi165.stdout:257: finishing write tid 2 to smithi16579500-36 2024-04-20T01:48:09.865 INFO:tasks.rados.rados.0.smithi165.stdout:257: finishing write tid 3 to smithi16579500-36 2024-04-20T01:48:09.865 INFO:tasks.rados.rados.0.smithi165.stdout:257: finishing write tid 4 to smithi16579500-36 2024-04-20T01:48:09.865 INFO:tasks.rados.rados.0.smithi165.stdout:258: finishing write tid 1 to smithi16579500-8 2024-04-20T01:48:09.865 INFO:tasks.rados.rados.0.smithi165.stdout:257: finishing write tid 5 to smithi16579500-36 2024-04-20T01:48:09.865 INFO:tasks.rados.rados.0.smithi165.stdout:256: done (4 left) 2024-04-20T01:48:09.865 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 17 current snap is 18 2024-04-20T01:48:09.865 INFO:tasks.rados.rados.0.smithi165.stdout:261: seq_num 120 ranges {3620864=475136,4096000=655360,4751360=622592,5373952=278528} 2024-04-20T01:48:09.875 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-20T01:48:09.875 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:48:09.884 INFO:tasks.rados.rados.0.smithi165.stdout:261: writing smithi16579500-17 from 3620864 to 4096000 tid 1 2024-04-20T01:48:09.888 INFO:tasks.rados.rados.0.smithi165.stdout:261: writing smithi16579500-17 from 4096000 to 4751360 tid 2 2024-04-20T01:48:09.892 INFO:tasks.rados.rados.0.smithi165.stdout:261: writing smithi16579500-17 from 4751360 to 5373952 tid 3 2024-04-20T01:48:09.895 INFO:tasks.rados.rados.0.smithi165.stdout:261: writing smithi16579500-17 from 5373952 to 5652480 tid 4 2024-04-20T01:48:09.896 INFO:tasks.rados.rados.0.smithi165.stdout:262: copy_from oid 50 from oid 14 current snap is 18 2024-04-20T01:48:09.896 INFO:tasks.rados.rados.0.smithi165.stdout:257: finishing write tid 6 to smithi16579500-36 2024-04-20T01:48:09.896 INFO:tasks.rados.rados.0.smithi165.stdout:257: oid 36 updating version 0 to 64 2024-04-20T01:48:09.896 INFO:tasks.rados.rados.0.smithi165.stdout:257: oid 36 updating version 64 to 67 2024-04-20T01:48:09.896 INFO:tasks.rados.rados.0.smithi165.stdout:257: oid 36 version 67 is already newer than 66 2024-04-20T01:48:09.896 INFO:tasks.rados.rados.0.smithi165.stdout:257: oid 36 version 67 is already newer than 65 2024-04-20T01:48:09.896 INFO:tasks.rados.rados.0.smithi165.stdout:257: oid 36 version 67 is already newer than 63 2024-04-20T01:48:09.896 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 36 v 67 (ObjNum 118 snap 18 seq_num 118) dirty exists 2024-04-20T01:48:09.896 INFO:tasks.rados.rados.0.smithi165.stdout:257: left oid 36 (ObjNum 118 snap 18 seq_num 118) 2024-04-20T01:48:09.896 INFO:tasks.rados.rados.0.smithi165.stdout:257: done (5 left) 2024-04-20T01:48:09.896 INFO:tasks.rados.rados.0.smithi165.stdout:263: copy_from oid 16 from oid 48 current snap is 18 2024-04-20T01:48:09.896 INFO:tasks.rados.rados.0.smithi165.stdout:258: finishing write tid 2 to smithi16579500-8 2024-04-20T01:48:09.897 INFO:tasks.rados.rados.0.smithi165.stdout:259: finishing rollback tid 0 to smithi16579500-47 2024-04-20T01:48:09.897 INFO:tasks.rados.rados.0.smithi165.stdout:258: finishing write tid 3 to smithi16579500-8 2024-04-20T01:48:09.897 INFO:tasks.rados.rados.0.smithi165.stdout:259: finishing rollback tid 1 to smithi16579500-47 2024-04-20T01:48:09.897 INFO:tasks.rados.rados.0.smithi165.stdout:259: finishing rollback tid 2 to smithi16579500-47 2024-04-20T01:48:09.897 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 47 v 70 (ObjNum 80 snap 8 seq_num 80) dirty exists 2024-04-20T01:48:09.897 INFO:tasks.rados.rados.0.smithi165.stdout:258: finishing write tid 4 to smithi16579500-8 2024-04-20T01:48:09.897 INFO:tasks.rados.rados.0.smithi165.stdout:258: finishing write tid 5 to smithi16579500-8 2024-04-20T01:48:09.897 INFO:tasks.rados.rados.0.smithi165.stdout:260: finishing copy_from racing read to smithi16579500-34 2024-04-20T01:48:09.897 INFO:tasks.rados.rados.0.smithi165.stdout:258: finishing write tid 6 to smithi16579500-8 2024-04-20T01:48:09.897 INFO:tasks.rados.rados.0.smithi165.stdout:260: finishing copy_from to smithi16579500-34 2024-04-20T01:48:09.897 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 34 v 38 (ObjNum 70 snap 5 seq_num 70) dirty exists 2024-04-20T01:48:09.897 INFO:tasks.rados.rados.0.smithi165.stdout:258: finishing write tid 7 to smithi16579500-8 2024-04-20T01:48:09.897 INFO:tasks.rados.rados.0.smithi165.stdout:258: oid 8 updating version 0 to 13 2024-04-20T01:48:09.897 INFO:tasks.rados.rados.0.smithi165.stdout:258: oid 8 version 13 is already newer than 12 2024-04-20T01:48:09.898 INFO:tasks.rados.rados.0.smithi165.stdout:258: oid 8 updating version 13 to 14 2024-04-20T01:48:09.898 INFO:tasks.rados.rados.0.smithi165.stdout:258: oid 8 version 14 is already newer than 10 2024-04-20T01:48:09.898 INFO:tasks.rados.rados.0.smithi165.stdout:258: oid 8 version 14 is already newer than 11 2024-04-20T01:48:09.898 INFO:tasks.rados.rados.0.smithi165.stdout:258: oid 8 updating version 14 to 15 2024-04-20T01:48:09.898 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 8 v 15 (ObjNum 119 snap 18 seq_num 119) dirty exists 2024-04-20T01:48:09.898 INFO:tasks.rados.rados.0.smithi165.stdout:258: left oid 8 (ObjNum 119 snap 18 seq_num 119) 2024-04-20T01:48:09.898 INFO:tasks.rados.rados.0.smithi165.stdout:261: finishing write tid 1 to smithi16579500-17 2024-04-20T01:48:09.898 INFO:tasks.rados.rados.0.smithi165.stdout:264: read oid 6 snap 14 2024-04-20T01:48:09.898 INFO:tasks.rados.rados.0.smithi165.stdout:264: expect (ObjNum 41 snap 0 seq_num 41) 2024-04-20T01:48:09.898 INFO:tasks.rados.rados.0.smithi165.stdout:258: done (6 left) 2024-04-20T01:48:09.898 INFO:tasks.rados.rados.0.smithi165.stdout:259: done (5 left) 2024-04-20T01:48:09.898 INFO:tasks.rados.rados.0.smithi165.stdout:260: done (4 left) 2024-04-20T01:48:09.898 INFO:tasks.rados.rados.0.smithi165.stdout:265: delete oid 24 current snap is 18 2024-04-20T01:48:09.899 INFO:tasks.rados.rados.0.smithi165.stdout:261: finishing write tid 2 to smithi16579500-17 2024-04-20T01:48:09.899 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:48:09.900 INFO:tasks.rados.rados.0.smithi165.stdout:261: finishing write tid 3 to smithi16579500-17 2024-04-20T01:48:09.900 INFO:tasks.rados.rados.0.smithi165.stdout:265: done (4 left) 2024-04-20T01:48:09.900 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 45 current snap is 18 2024-04-20T01:48:09.900 INFO:tasks.rados.rados.0.smithi165.stdout:266: seq_num 122 ranges {2834432=507904,3342336=425984,3768320=442368,4210688=540672,4751360=147456} 2024-04-20T01:48:09.907 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:09.921+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:09.919 INFO:tasks.rados.rados.0.smithi165.stdout:266: writing smithi16579500-45 from 2834432 to 3342336 tid 1 2024-04-20T01:48:09.919 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:48:09.921 INFO:tasks.rados.rados.0.smithi165.stdout:266: writing smithi16579500-45 from 3342336 to 3768320 tid 2 2024-04-20T01:48:09.924 INFO:tasks.rados.rados.0.smithi165.stdout:266: writing smithi16579500-45 from 3768320 to 4210688 tid 3 2024-04-20T01:48:09.927 INFO:tasks.rados.rados.0.smithi165.stdout:266: writing smithi16579500-45 from 4210688 to 4751360 tid 4 2024-04-20T01:48:09.928 INFO:tasks.rados.rados.0.smithi165.stdout:266: writing smithi16579500-45 from 4751360 to 4898816 tid 5 2024-04-20T01:48:09.928 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 9 current snap is 18 2024-04-20T01:48:09.928 INFO:tasks.rados.rados.0.smithi165.stdout:267: seq_num 123 ranges {1294336=524288,1818624=573440,2392064=688128,3080192=294912} 2024-04-20T01:48:09.938 INFO:tasks.rados.rados.0.smithi165.stdout:267: writing smithi16579500-9 from 1294336 to 1818624 tid 1 2024-04-20T01:48:09.941 INFO:tasks.rados.rados.0.smithi165.stdout:267: writing smithi16579500-9 from 1818624 to 2392064 tid 2 2024-04-20T01:48:09.945 INFO:tasks.rados.rados.0.smithi165.stdout:267: writing smithi16579500-9 from 2392064 to 3080192 tid 3 2024-04-20T01:48:09.946 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:48:09.947 INFO:tasks.rados.rados.0.smithi165.stdout:267: writing smithi16579500-9 from 3080192 to 3375104 tid 4 2024-04-20T01:48:09.947 INFO:tasks.rados.rados.0.smithi165.stdout:268: copy_from oid 8 from oid 1 current snap is 18 2024-04-20T01:48:09.947 INFO:tasks.rados.rados.0.smithi165.stdout:261: finishing write tid 4 to smithi16579500-17 2024-04-20T01:48:09.947 INFO:tasks.rados.rados.0.smithi165.stdout:261: finishing write tid 5 to smithi16579500-17 2024-04-20T01:48:09.948 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 34 current snap is 18 2024-04-20T01:48:09.948 INFO:tasks.rados.rados.0.smithi165.stdout:264: expect (ObjNum 41 snap 0 seq_num 41) 2024-04-20T01:48:09.967 INFO:tasks.rados.rados.0.smithi165.stdout:261: finishing write tid 6 to smithi16579500-17 2024-04-20T01:48:09.967 INFO:tasks.rados.rados.0.smithi165.stdout:261: oid 17 updating version 0 to 102 2024-04-20T01:48:09.967 INFO:tasks.rados.rados.0.smithi165.stdout:261: oid 17 updating version 102 to 106 2024-04-20T01:48:09.967 INFO:tasks.rados.rados.0.smithi165.stdout:261: oid 17 version 106 is already newer than 105 2024-04-20T01:48:09.967 INFO:tasks.rados.rados.0.smithi165.stdout:261: oid 17 version 106 is already newer than 103 2024-04-20T01:48:09.967 INFO:tasks.rados.rados.0.smithi165.stdout:261: oid 17 version 106 is already newer than 104 2024-04-20T01:48:09.967 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 17 v 106 (ObjNum 120 snap 18 seq_num 120) dirty exists 2024-04-20T01:48:09.967 INFO:tasks.rados.rados.0.smithi165.stdout:261: left oid 17 (ObjNum 120 snap 18 seq_num 120) 2024-04-20T01:48:09.967 INFO:tasks.rados.rados.0.smithi165.stdout:262: finishing copy_from racing read to smithi16579500-50 2024-04-20T01:48:09.968 INFO:tasks.rados.rados.0.smithi165.stdout:262: finishing copy_from to smithi16579500-50 2024-04-20T01:48:09.968 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 50 v 42 (ObjNum 94 snap 15 seq_num 94) dirty exists 2024-04-20T01:48:09.968 INFO:tasks.rados.rados.0.smithi165.stdout:263: finishing copy_from racing read to smithi16579500-16 2024-04-20T01:48:09.968 INFO:tasks.rados.rados.0.smithi165.stdout:263: finishing copy_from to smithi16579500-16 2024-04-20T01:48:09.968 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 16 v 43 (ObjNum 106 snap 16 seq_num 106) dirty exists 2024-04-20T01:48:09.968 INFO:tasks.rados.rados.0.smithi165.stdout:266: finishing write tid 1 to smithi16579500-45 2024-04-20T01:48:09.968 INFO:tasks.rados.rados.0.smithi165.stdout:266: finishing write tid 2 to smithi16579500-45 2024-04-20T01:48:09.968 INFO:tasks.rados.rados.0.smithi165.stdout:269: seq_num 124 ranges {1179648=540672,1720320=720896,2441216=786432,3227648=49152} 2024-04-20T01:48:09.969 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:48:09.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:48:09.984+0000 7f868d4f9700 0 -- 172.21.15.44:0/460596429 >> [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] conn(0x7f868821f470 msgr2=0x7f8688221860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-20T01:48:09.976 INFO:tasks.rados.rados.0.smithi165.stdout:269: writing smithi16579500-34 from 1179648 to 1720320 tid 1 2024-04-20T01:48:09.980 INFO:tasks.rados.rados.0.smithi165.stdout:269: writing smithi16579500-34 from 1720320 to 2441216 tid 2 2024-04-20T01:48:09.985 INFO:tasks.rados.rados.0.smithi165.stdout:269: writing smithi16579500-34 from 2441216 to 3227648 tid 3 2024-04-20T01:48:09.986 INFO:tasks.rados.rados.0.smithi165.stdout:269: writing smithi16579500-34 from 3227648 to 3276800 tid 4 2024-04-20T01:48:09.986 INFO:tasks.rados.rados.0.smithi165.stdout:261: done (7 left) 2024-04-20T01:48:09.986 INFO:tasks.rados.rados.0.smithi165.stdout:262: done (6 left) 2024-04-20T01:48:09.986 INFO:tasks.rados.rados.0.smithi165.stdout:263: done (5 left) 2024-04-20T01:48:09.987 INFO:tasks.rados.rados.0.smithi165.stdout:264: done (4 left) 2024-04-20T01:48:09.987 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 50 current snap is 18 2024-04-20T01:48:09.987 INFO:tasks.rados.rados.0.smithi165.stdout:266: finishing write tid 3 to smithi16579500-45 2024-04-20T01:48:09.987 INFO:tasks.rados.rados.0.smithi165.stdout:266: finishing write tid 4 to smithi16579500-45 2024-04-20T01:48:09.987 INFO:tasks.rados.rados.0.smithi165.stdout:266: finishing write tid 5 to smithi16579500-45 2024-04-20T01:48:09.987 INFO:tasks.rados.rados.0.smithi165.stdout:270: seq_num 125 ranges {1720320=557056,2277376=475136,2752512=507904,3260416=475136,3735552=98304} 2024-04-20T01:48:09.998 INFO:tasks.rados.rados.0.smithi165.stdout:270: writing smithi16579500-50 from 1720320 to 2277376 tid 1 2024-04-20T01:48:10.001 INFO:tasks.rados.rados.0.smithi165.stdout:270: writing smithi16579500-50 from 2277376 to 2752512 tid 2 2024-04-20T01:48:10.004 INFO:tasks.rados.rados.0.smithi165.stdout:270: writing smithi16579500-50 from 2752512 to 3260416 tid 3 2024-04-20T01:48:10.007 INFO:tasks.rados.rados.0.smithi165.stdout:270: writing smithi16579500-50 from 3260416 to 3735552 tid 4 2024-04-20T01:48:10.009 INFO:tasks.rados.rados.0.smithi165.stdout:270: writing smithi16579500-50 from 3735552 to 3833856 tid 5 2024-04-20T01:48:10.009 INFO:tasks.rados.rados.0.smithi165.stdout:271: read oid 47 snap -1 2024-04-20T01:48:10.009 INFO:tasks.rados.rados.0.smithi165.stdout:271: expect (ObjNum 80 snap 8 seq_num 80) 2024-04-20T01:48:10.009 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 42 current snap is 18 2024-04-20T01:48:10.009 INFO:tasks.rados.rados.0.smithi165.stdout:266: finishing write tid 6 to smithi16579500-45 2024-04-20T01:48:10.009 INFO:tasks.rados.rados.0.smithi165.stdout:266: finishing write tid 7 to smithi16579500-45 2024-04-20T01:48:10.009 INFO:tasks.rados.rados.0.smithi165.stdout:266: oid 45 updating version 0 to 111 2024-04-20T01:48:10.010 INFO:tasks.rados.rados.0.smithi165.stdout:266: oid 45 version 111 is already newer than 108 2024-04-20T01:48:10.010 INFO:tasks.rados.rados.0.smithi165.stdout:266: oid 45 version 111 is already newer than 106 2024-04-20T01:48:10.010 INFO:tasks.rados.rados.0.smithi165.stdout:266: oid 45 version 111 is already newer than 107 2024-04-20T01:48:10.010 INFO:tasks.rados.rados.0.smithi165.stdout:266: oid 45 version 111 is already newer than 110 2024-04-20T01:48:10.010 INFO:tasks.rados.rados.0.smithi165.stdout:266: oid 45 version 111 is already newer than 109 2024-04-20T01:48:10.010 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 45 v 111 (ObjNum 122 snap 18 seq_num 122) dirty exists 2024-04-20T01:48:10.010 INFO:tasks.rados.rados.0.smithi165.stdout:266: left oid 45 (ObjNum 122 snap 18 seq_num 122) 2024-04-20T01:48:10.010 INFO:tasks.rados.rados.0.smithi165.stdout:267: finishing write tid 1 to smithi16579500-9 2024-04-20T01:48:10.010 INFO:tasks.rados.rados.0.smithi165.stdout:267: finishing write tid 2 to smithi16579500-9 2024-04-20T01:48:10.010 INFO:tasks.rados.rados.0.smithi165.stdout:267: finishing write tid 3 to smithi16579500-9 2024-04-20T01:48:10.010 INFO:tasks.rados.rados.0.smithi165.stdout:272: seq_num 126 ranges {704512=573440,1277952=622592,1900544=622592,2523136=311296} 2024-04-20T01:48:10.011 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:10.022+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:10.015 INFO:tasks.rados.rados.0.smithi165.stdout:272: writing smithi16579500-42 from 704512 to 1277952 tid 1 2024-04-20T01:48:10.020 INFO:tasks.rados.rados.0.smithi165.stdout:272: writing smithi16579500-42 from 1277952 to 1900544 tid 2 2024-04-20T01:48:10.024 INFO:tasks.rados.rados.0.smithi165.stdout:272: writing smithi16579500-42 from 1900544 to 2523136 tid 3 2024-04-20T01:48:10.027 INFO:tasks.rados.rados.0.smithi165.stdout:272: writing smithi16579500-42 from 2523136 to 2834432 tid 4 2024-04-20T01:48:10.027 INFO:tasks.rados.rados.0.smithi165.stdout:266: done (6 left) 2024-04-20T01:48:10.027 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 31 current snap is 18 2024-04-20T01:48:10.027 INFO:tasks.rados.rados.0.smithi165.stdout:267: finishing write tid 4 to smithi16579500-9 2024-04-20T01:48:10.027 INFO:tasks.rados.rados.0.smithi165.stdout:267: finishing write tid 5 to smithi16579500-9 2024-04-20T01:48:10.027 INFO:tasks.rados.rados.0.smithi165.stdout:267: finishing write tid 6 to smithi16579500-9 2024-04-20T01:48:10.027 INFO:tasks.rados.rados.0.smithi165.stdout:267: oid 9 updating version 0 to 74 2024-04-20T01:48:10.027 INFO:tasks.rados.rados.0.smithi165.stdout:267: oid 9 updating version 74 to 76 2024-04-20T01:48:10.028 INFO:tasks.rados.rados.0.smithi165.stdout:267: oid 9 version 76 is already newer than 75 2024-04-20T01:48:10.028 INFO:tasks.rados.rados.0.smithi165.stdout:267: oid 9 updating version 76 to 78 2024-04-20T01:48:10.028 INFO:tasks.rados.rados.0.smithi165.stdout:267: oid 9 version 78 is already newer than 77 2024-04-20T01:48:10.028 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 9 v 78 (ObjNum 123 snap 18 seq_num 123) dirty exists 2024-04-20T01:48:10.028 INFO:tasks.rados.rados.0.smithi165.stdout:267: left oid 9 (ObjNum 123 snap 18 seq_num 123) 2024-04-20T01:48:10.028 INFO:tasks.rados.rados.0.smithi165.stdout:268: finishing copy_from racing read to smithi16579500-8 2024-04-20T01:48:10.028 INFO:tasks.rados.rados.0.smithi165.stdout:268: finishing copy_from to smithi16579500-8 2024-04-20T01:48:10.028 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 8 v 32 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-04-20T01:48:10.028 INFO:tasks.rados.rados.0.smithi165.stdout:269: finishing write tid 1 to smithi16579500-34 2024-04-20T01:48:10.028 INFO:tasks.rados.rados.0.smithi165.stdout:269: finishing write tid 2 to smithi16579500-34 2024-04-20T01:48:10.028 INFO:tasks.rados.rados.0.smithi165.stdout:269: finishing write tid 3 to smithi16579500-34 2024-04-20T01:48:10.028 INFO:tasks.rados.rados.0.smithi165.stdout:269: finishing write tid 4 to smithi16579500-34 2024-04-20T01:48:10.028 INFO:tasks.rados.rados.0.smithi165.stdout:269: finishing write tid 5 to smithi16579500-34 2024-04-20T01:48:10.028 INFO:tasks.rados.rados.0.smithi165.stdout:269: finishing write tid 6 to smithi16579500-34 2024-04-20T01:48:10.028 INFO:tasks.rados.rados.0.smithi165.stdout:269: oid 34 updating version 0 to 115 2024-04-20T01:48:10.029 INFO:tasks.rados.rados.0.smithi165.stdout:269: oid 34 version 115 is already newer than 112 2024-04-20T01:48:10.029 INFO:tasks.rados.rados.0.smithi165.stdout:269: oid 34 updating version 115 to 116 2024-04-20T01:48:10.029 INFO:tasks.rados.rados.0.smithi165.stdout:269: oid 34 version 116 is already newer than 113 2024-04-20T01:48:10.029 INFO:tasks.rados.rados.0.smithi165.stdout:269: oid 34 version 116 is already newer than 114 2024-04-20T01:48:10.029 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 34 v 116 (ObjNum 124 snap 18 seq_num 124) dirty exists 2024-04-20T01:48:10.029 INFO:tasks.rados.rados.0.smithi165.stdout:269: left oid 34 (ObjNum 124 snap 18 seq_num 124) 2024-04-20T01:48:10.029 INFO:tasks.rados.rados.0.smithi165.stdout:270: finishing write tid 1 to smithi16579500-50 2024-04-20T01:48:10.029 INFO:tasks.rados.rados.0.smithi165.stdout:270: finishing write tid 2 to smithi16579500-50 2024-04-20T01:48:10.029 INFO:tasks.rados.rados.0.smithi165.stdout:270: finishing write tid 3 to smithi16579500-50 2024-04-20T01:48:10.029 INFO:tasks.rados.rados.0.smithi165.stdout:270: finishing write tid 4 to smithi16579500-50 2024-04-20T01:48:10.029 INFO:tasks.rados.rados.0.smithi165.stdout:273: seq_num 127 ranges {2310144=589824,2899968=753664,3653632=475136,4128768=327680} 2024-04-20T01:48:10.032 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:10.032 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:10.042 INFO:tasks.rados.rados.0.smithi165.stdout:273: writing smithi16579500-31 from 2310144 to 2899968 tid 1 2024-04-20T01:48:10.043 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-20T01:48:10.046 INFO:tasks.rados.rados.0.smithi165.stdout:273: writing smithi16579500-31 from 2899968 to 3653632 tid 2 2024-04-20T01:48:10.049 INFO:tasks.rados.rados.0.smithi165.stdout:273: writing smithi16579500-31 from 3653632 to 4128768 tid 3 2024-04-20T01:48:10.051 INFO:tasks.rados.rados.0.smithi165.stdout:273: writing smithi16579500-31 from 4128768 to 4456448 tid 4 2024-04-20T01:48:10.051 INFO:tasks.rados.rados.0.smithi165.stdout:267: done (6 left) 2024-04-20T01:48:10.051 INFO:tasks.rados.rados.0.smithi165.stdout:268: done (5 left) 2024-04-20T01:48:10.051 INFO:tasks.rados.rados.0.smithi165.stdout:269: done (4 left) 2024-04-20T01:48:10.051 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 38 current snap is 18 2024-04-20T01:48:10.052 INFO:tasks.rados.rados.0.smithi165.stdout:270: finishing write tid 5 to smithi16579500-50 2024-04-20T01:48:10.052 INFO:tasks.rados.rados.0.smithi165.stdout:270: finishing write tid 6 to smithi16579500-50 2024-04-20T01:48:10.052 INFO:tasks.rados.rados.0.smithi165.stdout:274: seq_num 128 ranges {6406144=606208,7012352=507904,7520256=573440,8093696=475136} 2024-04-20T01:48:10.089 INFO:tasks.rados.rados.0.smithi165.stdout:274: writing smithi16579500-38 from 6406144 to 7012352 tid 1 2024-04-20T01:48:10.092 INFO:tasks.rados.rados.0.smithi165.stdout:274: writing smithi16579500-38 from 7012352 to 7520256 tid 2 2024-04-20T01:48:10.095 INFO:tasks.rados.rados.0.smithi165.stdout:274: writing smithi16579500-38 from 7520256 to 8093696 tid 3 2024-04-20T01:48:10.100 INFO:tasks.rados.rados.0.smithi165.stdout:274: writing smithi16579500-38 from 8093696 to 8568832 tid 4 2024-04-20T01:48:10.100 INFO:tasks.rados.rados.0.smithi165.stdout:275: copy_from oid 13 from oid 8 current snap is 18 2024-04-20T01:48:10.100 INFO:tasks.rados.rados.0.smithi165.stdout:272: finishing write tid 1 to smithi16579500-42 2024-04-20T01:48:10.100 INFO:tasks.rados.rados.0.smithi165.stdout:272: finishing write tid 2 to smithi16579500-42 2024-04-20T01:48:10.100 INFO:tasks.rados.rados.0.smithi165.stdout:272: finishing write tid 3 to smithi16579500-42 2024-04-20T01:48:10.100 INFO:tasks.rados.rados.0.smithi165.stdout:272: finishing write tid 4 to smithi16579500-42 2024-04-20T01:48:10.100 INFO:tasks.rados.rados.0.smithi165.stdout:272: finishing write tid 5 to smithi16579500-42 2024-04-20T01:48:10.100 INFO:tasks.rados.rados.0.smithi165.stdout:276: read oid 22 snap -1 2024-04-20T01:48:10.100 INFO:tasks.rados.rados.0.smithi165.stdout:276: expect (ObjNum 105 snap 16 seq_num 105) 2024-04-20T01:48:10.100 INFO:tasks.rados.rados.0.smithi165.stdout:277: copy_from oid 49 from oid 20 current snap is 18 2024-04-20T01:48:10.100 INFO:tasks.rados.rados.0.smithi165.stdout:272: finishing write tid 6 to smithi16579500-42 2024-04-20T01:48:10.100 INFO:tasks.rados.rados.0.smithi165.stdout:272: oid 42 updating version 0 to 38 2024-04-20T01:48:10.101 INFO:tasks.rados.rados.0.smithi165.stdout:272: oid 42 updating version 38 to 39 2024-04-20T01:48:10.101 INFO:tasks.rados.rados.0.smithi165.stdout:272: oid 42 updating version 39 to 40 2024-04-20T01:48:10.101 INFO:tasks.rados.rados.0.smithi165.stdout:272: oid 42 version 40 is already newer than 36 2024-04-20T01:48:10.101 INFO:tasks.rados.rados.0.smithi165.stdout:272: oid 42 version 40 is already newer than 37 2024-04-20T01:48:10.101 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 42 v 40 (ObjNum 126 snap 18 seq_num 126) dirty exists 2024-04-20T01:48:10.101 INFO:tasks.rados.rados.0.smithi165.stdout:272: left oid 42 (ObjNum 126 snap 18 seq_num 126) 2024-04-20T01:48:10.101 INFO:tasks.rados.rados.0.smithi165.stdout:272: done (7 left) 2024-04-20T01:48:10.101 INFO:tasks.rados.rados.0.smithi165.stdout:278: rollback oid 44 current snap is 18 2024-04-20T01:48:10.101 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 44 to 14 2024-04-20T01:48:10.101 INFO:tasks.rados.rados.0.smithi165.stdout:271: expect (ObjNum 80 snap 8 seq_num 80) 2024-04-20T01:48:10.109 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:10.123+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:10.211 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:48:10.211 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:10.224+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:10.211 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:48:10.211 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:48:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:48:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:48:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:48:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:48:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:48:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:48:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:48:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:48:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:48:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:48:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:48:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:48:10.212 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:48:10.213 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:48:10.215 INFO:tasks.rados.rados.0.smithi165.stdout:270: finishing write tid 7 to smithi16579500-50 2024-04-20T01:48:10.215 INFO:tasks.rados.rados.0.smithi165.stdout:270: oid 50 updating version 0 to 73 2024-04-20T01:48:10.215 INFO:tasks.rados.rados.0.smithi165.stdout:270: oid 50 updating version 73 to 75 2024-04-20T01:48:10.215 INFO:tasks.rados.rados.0.smithi165.stdout:270: oid 50 version 75 is already newer than 74 2024-04-20T01:48:10.215 INFO:tasks.rados.rados.0.smithi165.stdout:270: oid 50 updating version 75 to 76 2024-04-20T01:48:10.215 INFO:tasks.rados.rados.0.smithi165.stdout:270: oid 50 updating version 76 to 77 2024-04-20T01:48:10.215 INFO:tasks.rados.rados.0.smithi165.stdout:270: oid 50 version 77 is already newer than 72 2024-04-20T01:48:10.215 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 50 v 77 (ObjNum 125 snap 18 seq_num 125) dirty exists 2024-04-20T01:48:10.215 INFO:tasks.rados.rados.0.smithi165.stdout:270: left oid 50 (ObjNum 125 snap 18 seq_num 125) 2024-04-20T01:48:10.215 INFO:tasks.rados.rados.0.smithi165.stdout:273: finishing write tid 1 to smithi16579500-31 2024-04-20T01:48:10.215 INFO:tasks.rados.rados.0.smithi165.stdout:270: done (7 left) 2024-04-20T01:48:10.215 INFO:tasks.rados.rados.0.smithi165.stdout:271: done (6 left) 2024-04-20T01:48:10.216 INFO:tasks.rados.rados.0.smithi165.stdout:279: snap_create 2024-04-20T01:48:10.216 INFO:tasks.rados.rados.0.smithi165.stdout:273: finishing write tid 2 to smithi16579500-31 2024-04-20T01:48:10.216 INFO:tasks.rados.rados.0.smithi165.stdout:273: finishing write tid 3 to smithi16579500-31 2024-04-20T01:48:10.216 INFO:tasks.rados.rados.0.smithi165.stdout:273: finishing write tid 4 to smithi16579500-31 2024-04-20T01:48:10.216 INFO:tasks.rados.rados.0.smithi165.stdout:273: finishing write tid 5 to smithi16579500-31 2024-04-20T01:48:10.216 INFO:tasks.rados.rados.0.smithi165.stdout:273: finishing write tid 6 to smithi16579500-31 2024-04-20T01:48:10.216 INFO:tasks.rados.rados.0.smithi165.stdout:273: oid 31 updating version 0 to 118 2024-04-20T01:48:10.216 INFO:tasks.rados.rados.0.smithi165.stdout:273: oid 31 updating version 118 to 119 2024-04-20T01:48:10.216 INFO:tasks.rados.rados.0.smithi165.stdout:273: oid 31 version 119 is already newer than 117 2024-04-20T01:48:10.216 INFO:tasks.rados.rados.0.smithi165.stdout:273: oid 31 updating version 119 to 121 2024-04-20T01:48:10.216 INFO:tasks.rados.rados.0.smithi165.stdout:273: oid 31 version 121 is already newer than 120 2024-04-20T01:48:10.216 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 31 v 121 (ObjNum 127 snap 18 seq_num 127) dirty exists 2024-04-20T01:48:10.216 INFO:tasks.rados.rados.0.smithi165.stdout:273: left oid 31 (ObjNum 127 snap 18 seq_num 127) 2024-04-20T01:48:10.216 INFO:tasks.rados.rados.0.smithi165.stdout:274: finishing write tid 1 to smithi16579500-38 2024-04-20T01:48:10.217 INFO:tasks.rados.rados.0.smithi165.stdout:274: finishing write tid 2 to smithi16579500-38 2024-04-20T01:48:10.217 INFO:tasks.rados.rados.0.smithi165.stdout:274: finishing write tid 3 to smithi16579500-38 2024-04-20T01:48:10.217 INFO:tasks.rados.rados.0.smithi165.stdout:274: finishing write tid 4 to smithi16579500-38 2024-04-20T01:48:10.217 INFO:tasks.rados.rados.0.smithi165.stdout:274: finishing write tid 5 to smithi16579500-38 2024-04-20T01:48:10.217 INFO:tasks.rados.rados.0.smithi165.stdout:274: finishing write tid 6 to smithi16579500-38 2024-04-20T01:48:10.217 INFO:tasks.rados.rados.0.smithi165.stdout:274: oid 38 updating version 0 to 111 2024-04-20T01:48:10.217 INFO:tasks.rados.rados.0.smithi165.stdout:274: oid 38 version 111 is already newer than 110 2024-04-20T01:48:10.217 INFO:tasks.rados.rados.0.smithi165.stdout:274: oid 38 version 111 is already newer than 109 2024-04-20T01:48:10.217 INFO:tasks.rados.rados.0.smithi165.stdout:274: oid 38 updating version 111 to 112 2024-04-20T01:48:10.217 INFO:tasks.rados.rados.0.smithi165.stdout:274: oid 38 updating version 112 to 113 2024-04-20T01:48:10.217 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 38 v 113 (ObjNum 128 snap 18 seq_num 128) dirty exists 2024-04-20T01:48:10.217 INFO:tasks.rados.rados.0.smithi165.stdout:274: left oid 38 (ObjNum 128 snap 18 seq_num 128) 2024-04-20T01:48:10.217 INFO:tasks.rados.rados.0.smithi165.stdout:277: finishing copy_from racing read to smithi16579500-49 2024-04-20T01:48:10.217 INFO:tasks.rados.rados.0.smithi165.stdout:278: finishing rollback tid 0 to smithi16579500-44 2024-04-20T01:48:10.217 INFO:tasks.rados.rados.0.smithi165.stdout:278: finishing rollback tid 1 to smithi16579500-44 2024-04-20T01:48:10.218 INFO:tasks.rados.rados.0.smithi165.stdout:278: finishing rollback tid 2 to smithi16579500-44 2024-04-20T01:48:10.218 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 44 v 17 (ObjNum 57 snap 2 seq_num 57) dirty exists 2024-04-20T01:48:10.218 INFO:tasks.rados.rados.0.smithi165.stdout:275: finishing copy_from racing read to smithi16579500-13 2024-04-20T01:48:10.218 INFO:tasks.rados.rados.0.smithi165.stdout:277: finishing copy_from to smithi16579500-49 2024-04-20T01:48:10.218 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 49 v 100 (ObjNum 114 snap 18 seq_num 114) dirty exists 2024-04-20T01:48:10.218 INFO:tasks.rados.rados.0.smithi165.stdout:275: finishing copy_from to smithi16579500-13 2024-04-20T01:48:10.218 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 13 v 32 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-04-20T01:48:10.218 INFO:tasks.rados.rados.0.smithi165.stdout:276: expect (ObjNum 105 snap 16 seq_num 105) 2024-04-20T01:48:10.219 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:10.219 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:10.219 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-04-20T01:48:10.220 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:10.232 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-20T01:48:10.317 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:10.331+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:10.413 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:10.427+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:10.419 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:10.419 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:10.419 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-04-20T01:48:10.419 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-04-20T01:48:10.420 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:10.431 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-20T01:48:10.514 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:10.527+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:10.615 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:10.629+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:10.620 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-04-20T01:48:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-04-20T01:48:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-04-20T01:48:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.7s3 36/30 ECSubWrite(tid=58, reqid=client.4643.0:211, at_version=36'8, trim_to=0'0, roll_forward_to=33'5))", 2024-04-20T01:48:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.782029+0000", 2024-04-20T01:48:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.851173072000002, 2024-04-20T01:48:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0082533060000000002, 2024-04-20T01:48:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:10.620 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782029+0000", 2024-04-20T01:48:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782029+0000", 2024-04-20T01:48:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782028+0000", 2024-04-20T01:48:10.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782164+0000", 2024-04-20T01:48:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013607300000000001 2024-04-20T01:48:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782181+0000", 2024-04-20T01:48:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6651999999999999e-05 2024-04-20T01:48:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:10.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782196+0000", 2024-04-20T01:48:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5370999999999999e-05 2024-04-20T01:48:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.783434+0000", 2024-04-20T01:48:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001237581 2024-04-20T01:48:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.783444+0000", 2024-04-20T01:48:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0686999999999999e-05 2024-04-20T01:48:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:10.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.790251+0000", 2024-04-20T01:48:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068071729999999997 2024-04-20T01:48:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.790283+0000", 2024-04-20T01:48:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1250000000000001e-05 2024-04-20T01:48:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs1 36/30 ECSubWrite(tid=59, reqid=client.4643.0:214, at_version=36'14, trim_to=0'0, roll_forward_to=33'12))", 2024-04-20T01:48:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.783999+0000", 2024-04-20T01:48:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.849203996, 2024-04-20T01:48:10.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011678028, 2024-04-20T01:48:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.783999+0000", 2024-04-20T01:48:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.783999+0000", 2024-04-20T01:48:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.783997+0000", 2024-04-20T01:48:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.784024+0000", 2024-04-20T01:48:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6801999999999999e-05 2024-04-20T01:48:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.784037+0000", 2024-04-20T01:48:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2361000000000001e-05 2024-04-20T01:48:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.784046+0000", 2024-04-20T01:48:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5009999999999993e-06 2024-04-20T01:48:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.785306+0000", 2024-04-20T01:48:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001259942 2024-04-20T01:48:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.785316+0000", 2024-04-20T01:48:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0309000000000001e-05 2024-04-20T01:48:10.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.795648+0000", 2024-04-20T01:48:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010331127000000001 2024-04-20T01:48:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.795677+0000", 2024-04-20T01:48:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9016000000000001e-05 2024-04-20T01:48:10.628 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s4 36/30 ECSubWrite(tid=40, reqid=client.4643.0:215, at_version=36'28, trim_to=0'0, roll_forward_to=36'26))", 2024-04-20T01:48:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.808801+0000", 2024-04-20T01:48:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.824401397999999, 2024-04-20T01:48:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0082210219999999997, 2024-04-20T01:48:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808801+0000", 2024-04-20T01:48:10.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808801+0000", 2024-04-20T01:48:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808800+0000", 2024-04-20T01:48:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808834+0000", 2024-04-20T01:48:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3283e-05 2024-04-20T01:48:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808847+0000", 2024-04-20T01:48:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3587999999999999e-05 2024-04-20T01:48:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808857+0000", 2024-04-20T01:48:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6069999999999997e-06 2024-04-20T01:48:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808906+0000", 2024-04-20T01:48:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9283000000000002e-05 2024-04-20T01:48:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808917+0000", 2024-04-20T01:48:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0465999999999999e-05 2024-04-20T01:48:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.816976+0000", 2024-04-20T01:48:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0080597199999999994 2024-04-20T01:48:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:10.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.817022+0000", 2024-04-20T01:48:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.5802e-05 2024-04-20T01:48:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs1 37/30 ECSubWrite(tid=64, reqid=client.4643.0:225, at_version=37'16, trim_to=0'0, roll_forward_to=36'14))", 2024-04-20T01:48:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.788088+0000", 2024-04-20T01:48:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 31.845114976000001, 2024-04-20T01:48:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010015382, 2024-04-20T01:48:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.633 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788088+0000", 2024-04-20T01:48:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788088+0000", 2024-04-20T01:48:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788087+0000", 2024-04-20T01:48:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788230+0000", 2024-04-20T01:48:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014296 2024-04-20T01:48:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788255+0000", 2024-04-20T01:48:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4790000000000002e-05 2024-04-20T01:48:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788276+0000", 2024-04-20T01:48:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0806999999999999e-05 2024-04-20T01:48:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791925+0000", 2024-04-20T01:48:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003649949 2024-04-20T01:48:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791937+0000", 2024-04-20T01:48:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1544e-05 2024-04-20T01:48:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.798083+0000", 2024-04-20T01:48:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0061460079999999997 2024-04-20T01:48:10.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.798103+0000", 2024-04-20T01:48:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9907000000000001e-05 2024-04-20T01:48:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.8s1 47/30 ECSubWrite(tid=39, reqid=osd.5.0:39, at_version=47'27, trim_to=0'0, roll_forward_to=46'25))", 2024-04-20T01:48:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.718435+0000", 2024-04-20T01:48:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 21.914767981000001, 2024-04-20T01:48:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0080245720000000006, 2024-04-20T01:48:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:10.637 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.718435+0000", 2024-04-20T01:48:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.718435+0000", 2024-04-20T01:48:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:10.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.718434+0000", 2024-04-20T01:48:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.718459+0000", 2024-04-20T01:48:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5162999999999999e-05 2024-04-20T01:48:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.718477+0000", 2024-04-20T01:48:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8593999999999998e-05 2024-04-20T01:48:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.718488+0000", 2024-04-20T01:48:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0862999999999999e-05 2024-04-20T01:48:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719011+0000", 2024-04-20T01:48:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052293800000000003 2024-04-20T01:48:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719022+0000", 2024-04-20T01:48:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0862999999999999e-05 2024-04-20T01:48:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.726426+0000", 2024-04-20T01:48:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0074044039999999998 2024-04-20T01:48:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.726459+0000", 2024-04-20T01:48:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.2648000000000003e-05 2024-04-20T01:48:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.8s1 47/30 ECSubWrite(tid=40, reqid=osd.5.0:40, at_version=47'29, trim_to=0'0, roll_forward_to=46'25))", 2024-04-20T01:48:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.719174+0000", 2024-04-20T01:48:10.641 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 21.914028435999999, 2024-04-20T01:48:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0103924, 2024-04-20T01:48:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719174+0000", 2024-04-20T01:48:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719174+0000", 2024-04-20T01:48:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719173+0000", 2024-04-20T01:48:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719197+0000", 2024-04-20T01:48:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4196e-05 2024-04-20T01:48:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719214+0000", 2024-04-20T01:48:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6276000000000001e-05 2024-04-20T01:48:10.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719223+0000", 2024-04-20T01:48:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6919999999999997e-06 2024-04-20T01:48:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720898+0000", 2024-04-20T01:48:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0016744259999999999 2024-04-20T01:48:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720905+0000", 2024-04-20T01:48:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6660000000000008e-06 2024-04-20T01:48:10.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.729531+0000", 2024-04-20T01:48:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086254020000000008 2024-04-20T01:48:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.729566+0000", 2024-04-20T01:48:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5621000000000003e-05 2024-04-20T01:48:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.645 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s4 47/30 ECSubWrite(tid=80, reqid=osd.8.0:80, at_version=47'65, trim_to=0'0, roll_forward_to=44'61))", 2024-04-20T01:48:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.720292+0000", 2024-04-20T01:48:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 21.912910011000001, 2024-04-20T01:48:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0092900529999999995, 2024-04-20T01:48:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720292+0000", 2024-04-20T01:48:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:10.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720292+0000", 2024-04-20T01:48:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720292+0000", 2024-04-20T01:48:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720310+0000", 2024-04-20T01:48:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7717e-05 2024-04-20T01:48:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720318+0000", 2024-04-20T01:48:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3129999999999999e-06 2024-04-20T01:48:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720325+0000", 2024-04-20T01:48:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.832e-06 2024-04-20T01:48:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720837+0000", 2024-04-20T01:48:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051157600000000002 2024-04-20T01:48:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720846+0000", 2024-04-20T01:48:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3260000000000002e-06 2024-04-20T01:48:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.729552+0000", 2024-04-20T01:48:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0087057520000000006 2024-04-20T01:48:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.729583+0000", 2024-04-20T01:48:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0923999999999997e-05 2024-04-20T01:48:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s4 48/30 ECSubWrite(tid=85, reqid=client.4643.0:399, at_version=48'71, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.711859+0000", 2024-04-20T01:48:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 20.921343147000002, 2024-04-20T01:48:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0082489499999999997, 2024-04-20T01:48:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:10.650 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711859+0000", 2024-04-20T01:48:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711859+0000", 2024-04-20T01:48:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711858+0000", 2024-04-20T01:48:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:10.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711887+0000", 2024-04-20T01:48:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8487e-05 2024-04-20T01:48:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711908+0000", 2024-04-20T01:48:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1625e-05 2024-04-20T01:48:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711926+0000", 2024-04-20T01:48:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8113000000000001e-05 2024-04-20T01:48:10.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712063+0000", 2024-04-20T01:48:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000136907 2024-04-20T01:48:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712076+0000", 2024-04-20T01:48:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2145e-05 2024-04-20T01:48:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:10.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.720075+0000", 2024-04-20T01:48:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0079993730000000006 2024-04-20T01:48:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.720108+0000", 2024-04-20T01:48:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3386000000000002e-05 2024-04-20T01:48:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s4 48/30 ECSubWrite(tid=86, reqid=client.4643.0:400, at_version=48'72, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.713114+0000", 2024-04-20T01:48:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 20.920088884999998, 2024-04-20T01:48:10.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0088368690000000007, 2024-04-20T01:48:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713114+0000", 2024-04-20T01:48:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713114+0000", 2024-04-20T01:48:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713113+0000", 2024-04-20T01:48:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713136+0000", 2024-04-20T01:48:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2974999999999999e-05 2024-04-20T01:48:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713149+0000", 2024-04-20T01:48:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2945000000000001e-05 2024-04-20T01:48:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713159+0000", 2024-04-20T01:48:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9909999999999997e-06 2024-04-20T01:48:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713217+0000", 2024-04-20T01:48:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8085999999999999e-05 2024-04-20T01:48:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713228+0000", 2024-04-20T01:48:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1345999999999999e-05 2024-04-20T01:48:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.721915+0000", 2024-04-20T01:48:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086870850000000006 2024-04-20T01:48:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.721950+0000", 2024-04-20T01:48:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5352999999999998e-05 2024-04-20T01:48:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs1 52/30 ECSubWrite(tid=124, reqid=client.4643.0:434, at_version=52'55, trim_to=0'0, roll_forward_to=48'53))", 2024-04-20T01:48:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:53.736586+0000", 2024-04-20T01:48:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 16.896616006999999, 2024-04-20T01:48:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0094278839999999992, 2024-04-20T01:48:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736586+0000", 2024-04-20T01:48:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736586+0000", 2024-04-20T01:48:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736586+0000", 2024-04-20T01:48:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736681+0000", 2024-04-20T01:48:10.660 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4703000000000005e-05 2024-04-20T01:48:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736716+0000", 2024-04-20T01:48:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5434e-05 2024-04-20T01:48:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736737+0000", 2024-04-20T01:48:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0645999999999999e-05 2024-04-20T01:48:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736860+0000", 2024-04-20T01:48:10.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012254099999999999 2024-04-20T01:48:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736872+0000", 2024-04-20T01:48:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2824e-05 2024-04-20T01:48:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.745990+0000", 2024-04-20T01:48:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0091180250000000001 2024-04-20T01:48:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:10.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.746014+0000", 2024-04-20T01:48:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3983000000000001e-05 2024-04-20T01:48:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:573 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [create,setxattr POm]dpWYd`[NXWUkjTTrrjndlWhqT (15156) in=29b,setxattr ]bTTcZk (15156) in=7b,setxattr okg_granXfaiUobbllnc]Sk^[Vrj`[aRthnaRdoV (278) in=40b,setxattr gT`mrtdoaf^^ekYVVj`ZirVVqeUahfeOZdgUdQsV (382) in=40b,setxattr s\\mP`nbegjZ`iam]machaefN\\hcflgW[VmU^ik`_ (16840) in=40b,setxattr mPmpkek_rY\\scmjOek]degahQ^_SNk_epPi]p^fm (14456) in=40b,setxattr agOmkqtrc^nSastfh_^gPhplQZgnQkXRsXYbkW\\\\ (1789) in=40b,setxattr `Y[Zq^Pd_hpqld\\XRg`sTalN_ORNZ_aPUn_Nl\\ZQ (1330) in=40b,setxattr cnbkPlXlpjTSQl\\mdTVtmTO^ZjP[ra]OpgkWbsbZ (139) in=40b,setxattr Vjd]bV^O[pjq`XZai`go_goUTcrph[sfZXdaeo_d (453) in=40b,setxattr lPk`g]XVSTSfXOdjZeUQRqXTXgf]dUqZpmQ^hNh^ (841) in=40b,setxattr \\a\\rN^\\m`c^OlQbo^WerVQqiWZldlTX]n_o`^fcY (19224) in=40b,setxattr ehmtodtgcpX\\UmnfaWbtiaUQhSWb_lXn_qeXprOR (139) in=40b,setxattr VqR_VPdeQ]UeaTRh\\UfdplrbOaOSOsaSa_`pgZWX (10033) in=40b,setxattr U (15156) in=1b,setxattr `bXaUYb]oeUoqmrfqXgPlttVbU_laZ[rnYRZfRdd (418) in=40b,setxattr gPijt_qqRThf_ZmN`fYRO`UYUW^p_io^Sh\\_\\afh (1086) in=40b,setxattr WoqshhWZOap_beQ^YcYjaPktbdU[aac[`b\\d\\haa (734) in=40b,setxattr oOgQZWqrpnb_PhmYPbqgfRkbr_VjboWXsVpeUcPq (769) in=40b,setxattr t[S\\OOdRkdshlT\\lbkes]tbinQacc[aa^OYTep_d (1821) in=40b] snapc 1f=[1f,1b,1a,b] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-20T01:48:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:04.764695+0000", 2024-04-20T01:48:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.8685072050000002, 2024-04-20T01:48:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0091856230000000004, 2024-04-20T01:48:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:10.663 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 573 2024-04-20T01:48:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.764695+0000", 2024-04-20T01:48:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.764695+0000", 2024-04-20T01:48:10.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.764694+0000", 2024-04-20T01:48:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.765516+0000", 2024-04-20T01:48:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00082247099999999997 2024-04-20T01:48:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.765528+0000", 2024-04-20T01:48:10.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2191999999999999e-05 2024-04-20T01:48:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.765535+0000", 2024-04-20T01:48:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4119999999999998e-06 2024-04-20T01:48:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.765582+0000", 2024-04-20T01:48:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7707999999999998e-05 2024-04-20T01:48:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.666 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.765695+0000", 2024-04-20T01:48:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000112675 2024-04-20T01:48:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.766572+0000", 2024-04-20T01:48:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00087748600000000002 2024-04-20T01:48:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.773424+0000", 2024-04-20T01:48:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068519499999999999 2024-04-20T01:48:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.773548+0000", 2024-04-20T01:48:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012316900000000001 2024-04-20T01:48:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.773881+0000", 2024-04-20T01:48:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033335299999999999 2024-04-20T01:48:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:619 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [rollback 1a] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:10.668 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.811454+0000", 2024-04-20T01:48:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.82174822300000006, 2024-04-20T01:48:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0082693799999999998, 2024-04-20T01:48:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 619 2024-04-20T01:48:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.811454+0000", 2024-04-20T01:48:10.669 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.811454+0000", 2024-04-20T01:48:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.811454+0000", 2024-04-20T01:48:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:10.670 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.811457+0000", 2024-04-20T01:48:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9900000000000002e-06 2024-04-20T01:48:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.811464+0000", 2024-04-20T01:48:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8789999999999997e-06 2024-04-20T01:48:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.811470+0000", 2024-04-20T01:48:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-04-20T01:48:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.671 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.812533+0000", 2024-04-20T01:48:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0010629719999999999 2024-04-20T01:48:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.812647+0000", 2024-04-20T01:48:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000113911 2024-04-20T01:48:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.813311+0000", 2024-04-20T01:48:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00066420200000000002 2024-04-20T01:48:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:10.672 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.814350+0000", 2024-04-20T01:48:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0010393749999999999 2024-04-20T01:48:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.819649+0000", 2024-04-20T01:48:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052988430000000001 2024-04-20T01:48:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.819724+0000", 2024-04-20T01:48:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4702000000000004e-05 2024-04-20T01:48:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.673 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:641 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [append 0~507904] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.871779+0000", 2024-04-20T01:48:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.76142316499999996, 2024-04-20T01:48:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0091809730000000003, 2024-04-20T01:48:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 641 2024-04-20T01:48:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:10.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.871779+0000", 2024-04-20T01:48:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.871779+0000", 2024-04-20T01:48:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.871778+0000", 2024-04-20T01:48:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.872406+0000", 2024-04-20T01:48:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00062772399999999997 2024-04-20T01:48:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.872415+0000", 2024-04-20T01:48:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1460000000000002e-06 2024-04-20T01:48:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.872422+0000", 2024-04-20T01:48:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2430000000000002e-06 2024-04-20T01:48:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.872471+0000", 2024-04-20T01:48:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9867999999999998e-05 2024-04-20T01:48:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.872544+0000", 2024-04-20T01:48:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2841000000000003e-05 2024-04-20T01:48:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.873203+0000", 2024-04-20T01:48:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00065825699999999998 2024-04-20T01:48:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.875894+0000", 2024-04-20T01:48:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0026911959999999999 2024-04-20T01:48:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.880878+0000", 2024-04-20T01:48:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0049847559999999999 2024-04-20T01:48:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.880960+0000", 2024-04-20T01:48:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1835000000000005e-05 2024-04-20T01:48:10.678 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:642 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [append 0~507904] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.874202+0000", 2024-04-20T01:48:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.75900051899999998, 2024-04-20T01:48:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0093451449999999991, 2024-04-20T01:48:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:10.679 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 642 2024-04-20T01:48:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874202+0000", 2024-04-20T01:48:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874202+0000", 2024-04-20T01:48:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:10.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874201+0000", 2024-04-20T01:48:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874787+0000", 2024-04-20T01:48:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00058594599999999995 2024-04-20T01:48:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874796+0000", 2024-04-20T01:48:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2730000000000008e-06 2024-04-20T01:48:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874802+0000", 2024-04-20T01:48:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3619999999999997e-06 2024-04-20T01:48:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874835+0000", 2024-04-20T01:48:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3577000000000001e-05 2024-04-20T01:48:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.682 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874902+0000", 2024-04-20T01:48:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6989999999999994e-05 2024-04-20T01:48:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.875456+0000", 2024-04-20T01:48:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00055350099999999997 2024-04-20T01:48:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.876715+0000", 2024-04-20T01:48:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0012589260000000001 2024-04-20T01:48:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.883407+0000", 2024-04-20T01:48:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0066922270000000002 2024-04-20T01:48:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.883547+0000", 2024-04-20T01:48:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014018 2024-04-20T01:48:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.684 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:643 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [append 0~458752] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.876828+0000", 2024-04-20T01:48:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.75637431099999997, 2024-04-20T01:48:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096858259999999998, 2024-04-20T01:48:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 643 2024-04-20T01:48:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.685 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.876828+0000", 2024-04-20T01:48:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.876828+0000", 2024-04-20T01:48:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:10.686 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.876827+0000", 2024-04-20T01:48:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877322+0000", 2024-04-20T01:48:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049457899999999998 2024-04-20T01:48:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877333+0000", 2024-04-20T01:48:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1316e-05 2024-04-20T01:48:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:10.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877341+0000", 2024-04-20T01:48:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9910000000000002e-06 2024-04-20T01:48:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877374+0000", 2024-04-20T01:48:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3173000000000002e-05 2024-04-20T01:48:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877450+0000", 2024-04-20T01:48:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5864000000000004e-05 2024-04-20T01:48:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.878174+0000", 2024-04-20T01:48:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00072389000000000004 2024-04-20T01:48:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.880693+0000", 2024-04-20T01:48:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025188430000000002 2024-04-20T01:48:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.886426+0000", 2024-04-20T01:48:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0057329540000000002 2024-04-20T01:48:10.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.886514+0000", 2024-04-20T01:48:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8096000000000005e-05 2024-04-20T01:48:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:644 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [append 0~81920] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.877354+0000", 2024-04-20T01:48:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.75584832599999996, 2024-04-20T01:48:10.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0094386420000000006, 2024-04-20T01:48:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 644 2024-04-20T01:48:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877354+0000", 2024-04-20T01:48:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877354+0000", 2024-04-20T01:48:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877353+0000", 2024-04-20T01:48:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:10.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877455+0000", 2024-04-20T01:48:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000101229 2024-04-20T01:48:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877467+0000", 2024-04-20T01:48:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2061e-05 2024-04-20T01:48:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:10.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877474+0000", 2024-04-20T01:48:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5730000000000002e-06 2024-04-20T01:48:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.878740+0000", 2024-04-20T01:48:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001266145 2024-04-20T01:48:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.878801+0000", 2024-04-20T01:48:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0432e-05 2024-04-20T01:48:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.879293+0000", 2024-04-20T01:48:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049227199999999998 2024-04-20T01:48:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.881226+0000", 2024-04-20T01:48:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00193319 2024-04-20T01:48:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.886717+0000", 2024-04-20T01:48:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0054906019999999998 2024-04-20T01:48:10.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.886793+0000", 2024-04-20T01:48:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5957999999999993e-05 2024-04-20T01:48:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:645 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [setxattr _header (56) in=7b] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.877483+0000", 2024-04-20T01:48:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.755719421, 2024-04-20T01:48:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011773402000000001, 2024-04-20T01:48:10.696 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 645 2024-04-20T01:48:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877483+0000", 2024-04-20T01:48:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.697 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877483+0000", 2024-04-20T01:48:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877482+0000", 2024-04-20T01:48:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877496+0000", 2024-04-20T01:48:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3591000000000001e-05 2024-04-20T01:48:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877504+0000", 2024-04-20T01:48:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9880000000000004e-06 2024-04-20T01:48:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877510+0000", 2024-04-20T01:48:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1970000000000001e-06 2024-04-20T01:48:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.879785+0000", 2024-04-20T01:48:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022748339999999999 2024-04-20T01:48:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:10.699 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.879861+0000", 2024-04-20T01:48:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5965999999999997e-05 2024-04-20T01:48:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.880287+0000", 2024-04-20T01:48:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00042561299999999998 2024-04-20T01:48:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.881229+0000", 2024-04-20T01:48:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00094196299999999998 2024-04-20T01:48:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:10.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.889146+0000", 2024-04-20T01:48:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0079171320000000003 2024-04-20T01:48:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.889256+0000", 2024-04-20T01:48:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011071899999999999 2024-04-20T01:48:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:646 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.877516+0000", 2024-04-20T01:48:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.75568665000000002, 2024-04-20T01:48:10.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.017708380999999999, 2024-04-20T01:48:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 646 2024-04-20T01:48:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877516+0000", 2024-04-20T01:48:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877516+0000", 2024-04-20T01:48:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877515+0000", 2024-04-20T01:48:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877518+0000", 2024-04-20T01:48:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8940000000000002e-06 2024-04-20T01:48:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877530+0000", 2024-04-20T01:48:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2103000000000001e-05 2024-04-20T01:48:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877536+0000", 2024-04-20T01:48:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8390000000000003e-06 2024-04-20T01:48:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.880487+0000", 2024-04-20T01:48:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0029510949999999999 2024-04-20T01:48:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.880557+0000", 2024-04-20T01:48:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9443999999999994e-05 2024-04-20T01:48:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.889586+0000", 2024-04-20T01:48:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0090290790000000006 2024-04-20T01:48:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.889691+0000", 2024-04-20T01:48:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105386 2024-04-20T01:48:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:10.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.895224+0000", 2024-04-20T01:48:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.005533044 2024-04-20T01:48:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:679 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [copy-from ver 32] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.962971+0000", 2024-04-20T01:48:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.67023171699999995, 2024-04-20T01:48:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011067029000000001, 2024-04-20T01:48:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 679 2024-04-20T01:48:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.962971+0000", 2024-04-20T01:48:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.962971+0000", 2024-04-20T01:48:10.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.962969+0000", 2024-04-20T01:48:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.962976+0000", 2024-04-20T01:48:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9179999999999997e-06 2024-04-20T01:48:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.708 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.962987+0000", 2024-04-20T01:48:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0533000000000001e-05 2024-04-20T01:48:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.962993+0000", 2024-04-20T01:48:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2090000000000002e-06 2024-04-20T01:48:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963033+0000", 2024-04-20T01:48:10.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0207000000000003e-05 2024-04-20T01:48:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963106+0000", 2024-04-20T01:48:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3095000000000004e-05 2024-04-20T01:48:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.971796+0000", 2024-04-20T01:48:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086893639999999998 2024-04-20T01:48:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.972929+0000", 2024-04-20T01:48:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011335430000000001 2024-04-20T01:48:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.973994+0000", 2024-04-20T01:48:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001064923 2024-04-20T01:48:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.974038+0000", 2024-04-20T01:48:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3748000000000002e-05 2024-04-20T01:48:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:680 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.963000+0000", 2024-04-20T01:48:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.67020243400000001, 2024-04-20T01:48:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0095293779999999998, 2024-04-20T01:48:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 680 2024-04-20T01:48:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.712 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963000+0000", 2024-04-20T01:48:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963000+0000", 2024-04-20T01:48:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963000+0000", 2024-04-20T01:48:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:10.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963002+0000", 2024-04-20T01:48:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.125e-06 2024-04-20T01:48:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963007+0000", 2024-04-20T01:48:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0529999999999997e-06 2024-04-20T01:48:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963010+0000", 2024-04-20T01:48:10.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4680000000000001e-06 2024-04-20T01:48:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963239+0000", 2024-04-20T01:48:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00022873699999999999 2024-04-20T01:48:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963278+0000", 2024-04-20T01:48:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8733000000000003e-05 2024-04-20T01:48:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.972372+0000", 2024-04-20T01:48:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0090944210000000001 2024-04-20T01:48:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.972460+0000", 2024-04-20T01:48:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7349999999999995e-05 2024-04-20T01:48:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.972529+0000", 2024-04-20T01:48:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9809999999999997e-05 2024-04-20T01:48:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:10.716 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:10.716 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:10.718 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-20T01:48:10.719 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:10.731+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:10.771 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-20T01:48:10.772 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-20T01:48:10.772 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-04-20T01:48:10.773 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-04-20T01:48:10.773 INFO:tasks.rados.rados.0.smithi165.stdout:273: done (6 left) 2024-04-20T01:48:10.774 INFO:tasks.rados.rados.0.smithi165.stdout:274: done (5 left) 2024-04-20T01:48:10.774 INFO:tasks.rados.rados.0.smithi165.stdout:275: done (4 left) 2024-04-20T01:48:10.774 INFO:tasks.rados.rados.0.smithi165.stdout:276: done (3 left) 2024-04-20T01:48:10.774 INFO:tasks.rados.rados.0.smithi165.stdout:277: done (2 left) 2024-04-20T01:48:10.774 INFO:tasks.rados.rados.0.smithi165.stdout:278: done (1 left) 2024-04-20T01:48:10.774 INFO:tasks.rados.rados.0.smithi165.stdout:279: done (0 left) 2024-04-20T01:48:10.774 INFO:tasks.rados.rados.0.smithi165.stdout:280: snap_create 2024-04-20T01:48:10.780 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-20T01:48:10.849 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:10.863+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:10.919 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:10.934+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:10.923 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:10.923 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:10.923 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-04-20T01:48:10.923 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:10.937 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-20T01:48:11.021 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:11.035+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:11.121 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:11.135+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:11.122 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:11.122 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:11.123 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-04-20T01:48:11.123 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-04-20T01:48:11.123 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:11.134 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-20T01:48:11.194 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-20T01:48:11.208+0000 7f6fd7ecc700 -1 mgr.server reply reply (2) No such file or directory no change 2024-04-20T01:48:11.195 INFO:teuthology.orchestra.run.smithi044.stdout:moved 3 / 130 (2.30769%) 2024-04-20T01:48:11.195 INFO:teuthology.orchestra.run.smithi044.stdout:avg 8.125 2024-04-20T01:48:11.195 INFO:teuthology.orchestra.run.smithi044.stdout:stddev 1.86665 -> 1.57619 (expected baseline 2.75993) 2024-04-20T01:48:11.195 INFO:teuthology.orchestra.run.smithi044.stdout:min osd.9 with 5 -> 5 pgs (0.615385 -> 0.615385 * mean) 2024-04-20T01:48:11.195 INFO:teuthology.orchestra.run.smithi044.stdout:max osd.3 with 12 -> 9 pgs (1.47692 -> 1.10769 * mean) 2024-04-20T01:48:11.195 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:48:11.195 INFO:teuthology.orchestra.run.smithi044.stdout:oload 110 2024-04-20T01:48:11.195 INFO:teuthology.orchestra.run.smithi044.stdout:max_change 1 2024-04-20T01:48:11.195 INFO:teuthology.orchestra.run.smithi044.stdout:max_change_osds 4 2024-04-20T01:48:11.195 INFO:teuthology.orchestra.run.smithi044.stdout:average_utilization 0.0004 2024-04-20T01:48:11.195 INFO:teuthology.orchestra.run.smithi044.stdout:overload_utilization 0.0005 2024-04-20T01:48:11.195 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 weight 1.0000 -> 0.8432 2024-04-20T01:48:11.195 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 weight 1.0000 -> 0.8742 2024-04-20T01:48:11.198 INFO:teuthology.orchestra.run.smithi044.stderr:no change 2024-04-20T01:48:11.212 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-04-20T01:48:11.212 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-04-20T01:48:11.222 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:48:11.236+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:48:11.323 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:11.337+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:11.325 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:11.325 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-04-20T01:48:11.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-04-20T01:48:11.326 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-04-20T01:48:11.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.326 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:211 3.7s0 3:eb044288:::smithi16579500-44:head [append 0~376832] snapc 3=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:48:11.326 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.778254+0000", 2024-04-20T01:48:11.326 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.560308196000001, 2024-04-20T01:48:11.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01403427, 2024-04-20T01:48:11.326 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.326 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:11.326 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.326 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.326 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.326 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 211 2024-04-20T01:48:11.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.327 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778254+0000", 2024-04-20T01:48:11.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.327 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778254+0000", 2024-04-20T01:48:11.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778252+0000", 2024-04-20T01:48:11.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:11.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780895+0000", 2024-04-20T01:48:11.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002642658 2024-04-20T01:48:11.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780910+0000", 2024-04-20T01:48:11.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4949999999999999e-05 2024-04-20T01:48:11.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780919+0000", 2024-04-20T01:48:11.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7770000000000008e-06 2024-04-20T01:48:11.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780949+0000", 2024-04-20T01:48:11.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0639999999999998e-05 2024-04-20T01:48:11.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781077+0000", 2024-04-20T01:48:11.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000127565 2024-04-20T01:48:11.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781928+0000", 2024-04-20T01:48:11.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00085169599999999999 2024-04-20T01:48:11.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.787994+0000", 2024-04-20T01:48:11.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00606545 2024-04-20T01:48:11.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:11.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.792213+0000", 2024-04-20T01:48:11.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042187199999999996 2024-04-20T01:48:11.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.792289+0000", 2024-04-20T01:48:11.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5980000000000001e-05 2024-04-20T01:48:11.331 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.331 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.331 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.331 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:213 3.7s0 3:eb044288:::smithi16579500-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:48:11.331 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.780960+0000", 2024-04-20T01:48:11.332 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.557602115000002, 2024-04-20T01:48:11.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015313724000000001, 2024-04-20T01:48:11.332 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.332 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:11.332 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.332 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.332 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.332 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 213 2024-04-20T01:48:11.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.332 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.332 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.332 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780960+0000", 2024-04-20T01:48:11.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780960+0000", 2024-04-20T01:48:11.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780960+0000", 2024-04-20T01:48:11.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:11.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780964+0000", 2024-04-20T01:48:11.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6610000000000002e-06 2024-04-20T01:48:11.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780971+0000", 2024-04-20T01:48:11.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9270000000000005e-06 2024-04-20T01:48:11.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780978+0000", 2024-04-20T01:48:11.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1670000000000004e-06 2024-04-20T01:48:11.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.786222+0000", 2024-04-20T01:48:11.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052446979999999999 2024-04-20T01:48:11.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:11.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.786324+0000", 2024-04-20T01:48:11.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000101801 2024-04-20T01:48:11.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.793221+0000", 2024-04-20T01:48:11.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068970510000000004 2024-04-20T01:48:11.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.793323+0000", 2024-04-20T01:48:11.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102289 2024-04-20T01:48:11.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.796274+0000", 2024-04-20T01:48:11.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0029506409999999999 2024-04-20T01:48:11.336 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.336 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.336 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.336 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:214 3.bs0 3:dc0bbe39:::smithi16579500-13:head [stat,delete] snapc 3=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:48:11.336 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.781222+0000", 2024-04-20T01:48:11.336 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.557340668000002, 2024-04-20T01:48:11.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015584361, 2024-04-20T01:48:11.337 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.337 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:11.337 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.337 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.337 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.337 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 214 2024-04-20T01:48:11.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.337 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.337 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.337 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781222+0000", 2024-04-20T01:48:11.337 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.337 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781222+0000", 2024-04-20T01:48:11.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781221+0000", 2024-04-20T01:48:11.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:11.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.338 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.338 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.338 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781228+0000", 2024-04-20T01:48:11.338 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6699999999999997e-06 2024-04-20T01:48:11.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781239+0000", 2024-04-20T01:48:11.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1574e-05 2024-04-20T01:48:11.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781247+0000", 2024-04-20T01:48:11.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.593e-06 2024-04-20T01:48:11.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.339 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.339 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.339 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781288+0000", 2024-04-20T01:48:11.339 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1253e-05 2024-04-20T01:48:11.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781383+0000", 2024-04-20T01:48:11.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4788000000000001e-05 2024-04-20T01:48:11.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782250+0000", 2024-04-20T01:48:11.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00086664399999999996 2024-04-20T01:48:11.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.340 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.340 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.340 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.786611+0000", 2024-04-20T01:48:11.340 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004361714 2024-04-20T01:48:11.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:11.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.796739+0000", 2024-04-20T01:48:11.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010127519 2024-04-20T01:48:11.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.341 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.341 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.341 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.796806+0000", 2024-04-20T01:48:11.341 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7414000000000002e-05 2024-04-20T01:48:11.341 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.341 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.341 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.342 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.342 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:225 3.bs0 3:df90e529:::smithi16579500-45:head [append 0~622592] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:11.342 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.785410+0000", 2024-04-20T01:48:11.342 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.553152157, 2024-04-20T01:48:11.342 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013791425, 2024-04-20T01:48:11.342 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.342 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:11.342 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.342 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.342 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.342 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 225 2024-04-20T01:48:11.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.342 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.785410+0000", 2024-04-20T01:48:11.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.785410+0000", 2024-04-20T01:48:11.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.343 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.343 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.343 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.785407+0000", 2024-04-20T01:48:11.343 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999967 2024-04-20T01:48:11.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.786402+0000", 2024-04-20T01:48:11.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00099532000000000006 2024-04-20T01:48:11.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.786427+0000", 2024-04-20T01:48:11.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5304e-05 2024-04-20T01:48:11.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.344 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.344 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.344 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.786440+0000", 2024-04-20T01:48:11.344 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2238999999999999e-05 2024-04-20T01:48:11.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.786515+0000", 2024-04-20T01:48:11.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4923000000000001e-05 2024-04-20T01:48:11.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.786652+0000", 2024-04-20T01:48:11.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137522 2024-04-20T01:48:11.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.345 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.345 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.345 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788170+0000", 2024-04-20T01:48:11.345 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001518003 2024-04-20T01:48:11.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796858+0000", 2024-04-20T01:48:11.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086877320000000001 2024-04-20T01:48:11.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:11.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.799118+0000", 2024-04-20T01:48:11.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022599600000000001 2024-04-20T01:48:11.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.346 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.346 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.346 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.799202+0000", 2024-04-20T01:48:11.346 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3954000000000003e-05 2024-04-20T01:48:11.347 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.347 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.347 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.347 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.347 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:227 3.bs0 3:df90e529:::smithi16579500-45:head [setxattr _header (56) in=7b] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:11.347 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787843+0000", 2024-04-20T01:48:11.347 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.550719880000003, 2024-04-20T01:48:11.347 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014686189000000001, 2024-04-20T01:48:11.347 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.347 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:11.347 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.347 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.347 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.347 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 227 2024-04-20T01:48:11.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.348 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.348 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787843+0000", 2024-04-20T01:48:11.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.348 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.348 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787843+0000", 2024-04-20T01:48:11.348 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.349 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.349 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787842+0000", 2024-04-20T01:48:11.349 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:11.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.349 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.349 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787847+0000", 2024-04-20T01:48:11.349 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8130000000000001e-06 2024-04-20T01:48:11.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.349 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.349 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787853+0000", 2024-04-20T01:48:11.349 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5119999999999999e-06 2024-04-20T01:48:11.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.349 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787859+0000", 2024-04-20T01:48:11.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8119999999999996e-06 2024-04-20T01:48:11.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.790718+0000", 2024-04-20T01:48:11.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002859267 2024-04-20T01:48:11.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.350 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.350 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.350 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.790827+0000", 2024-04-20T01:48:11.350 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000109471 2024-04-20T01:48:11.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.351 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.351 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791236+0000", 2024-04-20T01:48:11.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000408637 2024-04-20T01:48:11.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.351 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.351 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796922+0000", 2024-04-20T01:48:11.351 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0056861139999999999 2024-04-20T01:48:11.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.351 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.351 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:11.351 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.802464+0000", 2024-04-20T01:48:11.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055420449999999998 2024-04-20T01:48:11.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.352 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.352 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.802529+0000", 2024-04-20T01:48:11.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4554000000000006e-05 2024-04-20T01:48:11.352 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.352 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.352 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.352 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.352 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:228 3.bs0 3:df90e529:::smithi16579500-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:11.352 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787871+0000", 2024-04-20T01:48:11.352 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.550691165000003, 2024-04-20T01:48:11.352 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.016315307000000001, 2024-04-20T01:48:11.353 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.353 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:11.353 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.353 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.353 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.353 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 228 2024-04-20T01:48:11.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.353 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.353 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.353 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.353 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787871+0000", 2024-04-20T01:48:11.353 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.354 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787871+0000", 2024-04-20T01:48:11.354 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.354 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787871+0000", 2024-04-20T01:48:11.354 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:11.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.354 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.354 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.354 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787873+0000", 2024-04-20T01:48:11.354 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.08e-06 2024-04-20T01:48:11.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.355 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.355 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787878+0000", 2024-04-20T01:48:11.355 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7299999999999996e-06 2024-04-20T01:48:11.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.355 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.355 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787881+0000", 2024-04-20T01:48:11.355 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5209999999999998e-06 2024-04-20T01:48:11.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.355 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.355 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.355 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791461+0000", 2024-04-20T01:48:11.356 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003579905 2024-04-20T01:48:11.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.356 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:11.356 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791554+0000", 2024-04-20T01:48:11.356 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2485000000000006e-05 2024-04-20T01:48:11.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.356 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.356 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.802822+0000", 2024-04-20T01:48:11.356 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011268251 2024-04-20T01:48:11.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.356 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.356 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.357 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.802922+0000", 2024-04-20T01:48:11.357 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010029700000000001 2024-04-20T01:48:11.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.357 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.357 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.357 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.804187+0000", 2024-04-20T01:48:11.357 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001264362 2024-04-20T01:48:11.357 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.357 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.357 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.357 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.357 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:385 3.1s0 3:8bb12819:::smithi16579500-21:head [copy-from ver 2] snapc d=[c,b] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-20T01:48:11.357 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.710205+0000", 2024-04-20T01:48:11.358 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 22.628357114, 2024-04-20T01:48:11.358 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.016348672000000002, 2024-04-20T01:48:11.358 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.358 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:11.358 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.358 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.358 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.358 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 385 2024-04-20T01:48:11.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.358 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.358 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.358 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.358 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710205+0000", 2024-04-20T01:48:11.359 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.359 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.359 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.359 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.359 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710205+0000", 2024-04-20T01:48:11.359 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.359 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.359 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.359 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.359 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710204+0000", 2024-04-20T01:48:11.359 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:11.359 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.359 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.359 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.359 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710212+0000", 2024-04-20T01:48:11.360 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8250000000000005e-06 2024-04-20T01:48:11.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.360 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.360 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.360 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710222+0000", 2024-04-20T01:48:11.360 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0071000000000001e-05 2024-04-20T01:48:11.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.360 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.360 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.360 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710232+0000", 2024-04-20T01:48:11.360 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0583e-05 2024-04-20T01:48:11.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.360 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.360 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.361 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710273+0000", 2024-04-20T01:48:11.361 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0407999999999997e-05 2024-04-20T01:48:11.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.361 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.361 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710360+0000", 2024-04-20T01:48:11.361 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6778000000000002e-05 2024-04-20T01:48:11.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.361 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.361 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719733+0000", 2024-04-20T01:48:11.361 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0093733800000000006 2024-04-20T01:48:11.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.361 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.362 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.362 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.723689+0000", 2024-04-20T01:48:11.362 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0039562620000000003 2024-04-20T01:48:11.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.362 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:11.362 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.726462+0000", 2024-04-20T01:48:11.362 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027724989999999999 2024-04-20T01:48:11.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.362 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.362 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.362 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.726554+0000", 2024-04-20T01:48:11.362 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2262000000000005e-05 2024-04-20T01:48:11.362 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.363 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.363 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.363 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s2 48/30 ECSubWrite(tid=85, reqid=client.4643.0:399, at_version=48'71, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:11.363 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.711991+0000", 2024-04-20T01:48:11.363 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 21.626571458000001, 2024-04-20T01:48:11.363 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013597662, 2024-04-20T01:48:11.363 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.363 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:11.363 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.363 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.363 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.363 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711991+0000", 2024-04-20T01:48:11.364 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.364 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.364 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711991+0000", 2024-04-20T01:48:11.364 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.364 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.364 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711990+0000", 2024-04-20T01:48:11.364 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:11.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.364 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.364 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.364 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712015+0000", 2024-04-20T01:48:11.364 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4516999999999999e-05 2024-04-20T01:48:11.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.365 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.365 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.365 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712032+0000", 2024-04-20T01:48:11.365 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7425999999999999e-05 2024-04-20T01:48:11.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.365 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.365 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.365 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712042+0000", 2024-04-20T01:48:11.365 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9000000000000001e-06 2024-04-20T01:48:11.365 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.365 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.365 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.365 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713222+0000", 2024-04-20T01:48:11.365 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001179776 2024-04-20T01:48:11.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.366 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.366 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713248+0000", 2024-04-20T01:48:11.366 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5990999999999999e-05 2024-04-20T01:48:11.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.366 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.366 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.725562+0000", 2024-04-20T01:48:11.366 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01231369 2024-04-20T01:48:11.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.366 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.366 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.366 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.725589+0000", 2024-04-20T01:48:11.367 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7137000000000001e-05 2024-04-20T01:48:11.367 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.367 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.367 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.367 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s2 48/30 ECSubWrite(tid=86, reqid=client.4643.0:400, at_version=48'72, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:11.367 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.713256+0000", 2024-04-20T01:48:11.367 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 21.625306366, 2024-04-20T01:48:11.367 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.017021191000000001, 2024-04-20T01:48:11.367 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.367 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:11.367 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.367 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.368 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.368 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713256+0000", 2024-04-20T01:48:11.368 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.368 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.368 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713256+0000", 2024-04-20T01:48:11.368 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.368 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.368 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.368 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713255+0000", 2024-04-20T01:48:11.368 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:11.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.369 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.369 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713277+0000", 2024-04-20T01:48:11.369 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1348e-05 2024-04-20T01:48:11.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.369 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.369 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713289+0000", 2024-04-20T01:48:11.369 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2692e-05 2024-04-20T01:48:11.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.369 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.369 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.369 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713299+0000", 2024-04-20T01:48:11.369 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0071999999999999e-05 2024-04-20T01:48:11.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.370 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.370 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714467+0000", 2024-04-20T01:48:11.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001167581 2024-04-20T01:48:11.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.370 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.370 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714478+0000", 2024-04-20T01:48:11.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1029999999999999e-05 2024-04-20T01:48:11.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.370 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.370 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.370 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730237+0000", 2024-04-20T01:48:11.370 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015758635 2024-04-20T01:48:11.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.371 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.371 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730277+0000", 2024-04-20T01:48:11.371 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0788999999999999e-05 2024-04-20T01:48:11.371 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.371 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.371 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.371 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.371 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s2 48/30 ECSubWrite(tid=87, reqid=client.4643.0:401, at_version=48'73, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:11.371 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.714498+0000", 2024-04-20T01:48:11.371 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 21.624064773000001, 2024-04-20T01:48:11.371 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015921002, 2024-04-20T01:48:11.372 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.372 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:11.372 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.372 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.372 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714498+0000", 2024-04-20T01:48:11.372 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.372 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.372 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714498+0000", 2024-04-20T01:48:11.372 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.372 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.372 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.373 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714497+0000", 2024-04-20T01:48:11.373 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:11.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.373 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.373 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714519+0000", 2024-04-20T01:48:11.373 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2574000000000001e-05 2024-04-20T01:48:11.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.373 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.373 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.374 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714531+0000", 2024-04-20T01:48:11.374 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1361e-05 2024-04-20T01:48:11.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.374 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.374 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714540+0000", 2024-04-20T01:48:11.374 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4769999999999992e-06 2024-04-20T01:48:11.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.374 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.374 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.716648+0000", 2024-04-20T01:48:11.374 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002108183 2024-04-20T01:48:11.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.374 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.375 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.375 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.716661+0000", 2024-04-20T01:48:11.375 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2490000000000001e-05 2024-04-20T01:48:11.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.375 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.375 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730395+0000", 2024-04-20T01:48:11.375 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013734333 2024-04-20T01:48:11.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.375 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.375 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.375 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730419+0000", 2024-04-20T01:48:11.375 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3489000000000002e-05 2024-04-20T01:48:11.375 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.375 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.376 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.376 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.3s2 48/30 ECSubWrite(tid=254, reqid=osd.13.0:251, at_version=0'0, trim_to=0'0, roll_forward_to=48'22))", 2024-04-20T01:48:11.376 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.719206+0000", 2024-04-20T01:48:11.376 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 21.619356274000001, 2024-04-20T01:48:11.376 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014281428000000001, 2024-04-20T01:48:11.376 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.376 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:11.376 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.376 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.376 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.376 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719206+0000", 2024-04-20T01:48:11.376 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.377 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719206+0000", 2024-04-20T01:48:11.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.377 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719206+0000", 2024-04-20T01:48:11.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:11.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.377 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.377 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.377 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719215+0000", 2024-04-20T01:48:11.377 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6229999999999992e-06 2024-04-20T01:48:11.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.378 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.378 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719237+0000", 2024-04-20T01:48:11.378 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1877000000000001e-05 2024-04-20T01:48:11.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.378 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.378 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719249+0000", 2024-04-20T01:48:11.378 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1736000000000001e-05 2024-04-20T01:48:11.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.378 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.378 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.378 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.726201+0000", 2024-04-20T01:48:11.378 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0069514930000000004 2024-04-20T01:48:11.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.379 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.379 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.379 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.726206+0000", 2024-04-20T01:48:11.379 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7830000000000004e-06 2024-04-20T01:48:11.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.379 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.379 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.379 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.733466+0000", 2024-04-20T01:48:11.379 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072593550000000003 2024-04-20T01:48:11.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.379 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.379 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.379 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.733488+0000", 2024-04-20T01:48:11.380 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2007000000000001e-05 2024-04-20T01:48:11.380 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.380 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.380 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.380 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.380 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.380 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:493 3.1s0 3:8bb12819:::smithi16579500-21:head [copy-from ver 27] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:11.380 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.773406+0000", 2024-04-20T01:48:11.380 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.565156862, 2024-04-20T01:48:11.380 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014247365999999999, 2024-04-20T01:48:11.380 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.380 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:11.380 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.380 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.380 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.381 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 493 2024-04-20T01:48:11.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.381 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.381 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.381 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773406+0000", 2024-04-20T01:48:11.381 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.381 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.381 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773406+0000", 2024-04-20T01:48:11.381 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.381 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.381 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.382 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773404+0000", 2024-04-20T01:48:11.382 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:11.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.382 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.382 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773412+0000", 2024-04-20T01:48:11.382 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7309999999999993e-06 2024-04-20T01:48:11.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.382 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.382 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.382 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773427+0000", 2024-04-20T01:48:11.382 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5167999999999999e-05 2024-04-20T01:48:11.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.383 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.383 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773435+0000", 2024-04-20T01:48:11.383 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4710000000000007e-06 2024-04-20T01:48:11.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.383 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.383 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773496+0000", 2024-04-20T01:48:11.383 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0325000000000003e-05 2024-04-20T01:48:11.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.383 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.383 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op must wait for map", 2024-04-20T01:48:11.383 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773506+0000", 2024-04-20T01:48:11.383 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9739999999999997e-06 2024-04-20T01:48:11.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.384 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.384 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.775199+0000", 2024-04-20T01:48:11.384 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001693214 2024-04-20T01:48:11.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.384 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.384 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.775299+0000", 2024-04-20T01:48:11.384 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000100079 2024-04-20T01:48:11.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.384 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.384 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.384 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.782700+0000", 2024-04-20T01:48:11.384 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0074012640000000003 2024-04-20T01:48:11.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.787277+0000", 2024-04-20T01:48:11.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045765349999999996 2024-04-20T01:48:11.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:11.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.787341+0000", 2024-04-20T01:48:11.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3998000000000006e-05 2024-04-20T01:48:11.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.385 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.385 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.385 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.787653+0000", 2024-04-20T01:48:11.385 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00031239000000000001 2024-04-20T01:48:11.386 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.386 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.386 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.386 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.386 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:494 3.1s0 3:8bb12819:::smithi16579500-21:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:11.386 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.773446+0000", 2024-04-20T01:48:11.386 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.565116768999999, 2024-04-20T01:48:11.386 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013216962000000001, 2024-04-20T01:48:11.386 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.386 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:11.386 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.386 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.386 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.386 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 494 2024-04-20T01:48:11.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.387 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.387 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.387 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773446+0000", 2024-04-20T01:48:11.387 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.387 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.387 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773446+0000", 2024-04-20T01:48:11.387 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.387 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.387 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773445+0000", 2024-04-20T01:48:11.388 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:11.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.388 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.388 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773452+0000", 2024-04-20T01:48:11.388 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3119999999999999e-06 2024-04-20T01:48:11.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.388 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.388 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773460+0000", 2024-04-20T01:48:11.388 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1189999999999997e-06 2024-04-20T01:48:11.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.388 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.388 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.389 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773464+0000", 2024-04-20T01:48:11.389 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7219999999999999e-06 2024-04-20T01:48:11.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.389 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773569+0000", 2024-04-20T01:48:11.389 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000104925 2024-04-20T01:48:11.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:11.389 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773574+0000", 2024-04-20T01:48:11.389 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9359999999999998e-06 2024-04-20T01:48:11.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.389 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.390 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.775428+0000", 2024-04-20T01:48:11.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001853265 2024-04-20T01:48:11.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.390 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:11.390 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.775467+0000", 2024-04-20T01:48:11.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9684999999999997e-05 2024-04-20T01:48:11.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.390 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.390 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.786440+0000", 2024-04-20T01:48:11.390 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010973032000000001 2024-04-20T01:48:11.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.390 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.390 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.786549+0000", 2024-04-20T01:48:11.391 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010848700000000001 2024-04-20T01:48:11.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.391 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.391 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.786663+0000", 2024-04-20T01:48:11.391 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000113941 2024-04-20T01:48:11.391 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.391 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.391 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.391 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.391 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.391 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:584 3.bs0 3:d86ca78e:::smithi16579500-26:1f [read 0~4816896,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-04-20T01:48:11.391 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:05.758316+0000", 2024-04-20T01:48:11.391 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.580246292, 2024-04-20T01:48:11.392 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015613514, 2024-04-20T01:48:11.392 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.392 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:11.392 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.392 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.392 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.392 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 584 2024-04-20T01:48:11.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.392 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.392 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.392 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.758316+0000", 2024-04-20T01:48:11.392 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.392 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.393 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.393 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.758316+0000", 2024-04-20T01:48:11.393 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.393 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.393 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.758316+0000", 2024-04-20T01:48:11.393 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:11.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.393 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.393 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.758321+0000", 2024-04-20T01:48:11.393 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9099999999999996e-06 2024-04-20T01:48:11.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.393 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.394 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.394 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.758328+0000", 2024-04-20T01:48:11.394 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4619999999999999e-06 2024-04-20T01:48:11.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.394 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.394 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.394 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.758334+0000", 2024-04-20T01:48:11.394 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7960000000000001e-06 2024-04-20T01:48:11.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.394 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.394 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.394 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.759436+0000", 2024-04-20T01:48:11.394 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011024730000000001 2024-04-20T01:48:11.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.395 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.395 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.759539+0000", 2024-04-20T01:48:11.395 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102896 2024-04-20T01:48:11.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.395 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.395 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.773930+0000", 2024-04-20T01:48:11.395 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014390464 2024-04-20T01:48:11.395 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.395 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.395 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.395 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:650 3.bs0 3:da81107c:::smithi16579500-34:head [copy-from ver 38] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:11.396 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.877476+0000", 2024-04-20T01:48:11.396 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 1.461087005, 2024-04-20T01:48:11.396 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014109823000000001, 2024-04-20T01:48:11.396 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.396 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:11.396 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.396 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.396 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.396 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 650 2024-04-20T01:48:11.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.396 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.396 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.396 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.396 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877476+0000", 2024-04-20T01:48:11.397 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.397 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.397 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.397 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877476+0000", 2024-04-20T01:48:11.397 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.397 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.397 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.397 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877470+0000", 2024-04-20T01:48:11.397 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999943 2024-04-20T01:48:11.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.397 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.397 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.398 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877485+0000", 2024-04-20T01:48:11.398 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4973e-05 2024-04-20T01:48:11.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.398 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.398 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877509+0000", 2024-04-20T01:48:11.398 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4056000000000001e-05 2024-04-20T01:48:11.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.398 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.398 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877521+0000", 2024-04-20T01:48:11.398 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2462e-05 2024-04-20T01:48:11.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.398 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.398 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.399 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877587+0000", 2024-04-20T01:48:11.399 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6008999999999998e-05 2024-04-20T01:48:11.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.399 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877709+0000", 2024-04-20T01:48:11.399 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000121537 2024-04-20T01:48:11.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.399 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.399 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.887306+0000", 2024-04-20T01:48:11.399 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0095964940000000006 2024-04-20T01:48:11.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.399 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.400 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.400 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.889750+0000", 2024-04-20T01:48:11.400 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0024441659999999998 2024-04-20T01:48:11.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.400 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:11.400 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.891529+0000", 2024-04-20T01:48:11.400 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001778916 2024-04-20T01:48:11.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.400 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.400 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.400 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.891585+0000", 2024-04-20T01:48:11.400 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6725999999999999e-05 2024-04-20T01:48:11.400 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.401 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.401 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.401 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.401 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:715 3.bs0 3:dc0bbe39:::smithi16579500-13:head [copy-from ver 32] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:11.401 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:10.113717+0000", 2024-04-20T01:48:11.401 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 1.224845307, 2024-04-20T01:48:11.401 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.020694802000000002, 2024-04-20T01:48:11.401 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.401 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:11.401 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.401 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.401 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.401 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 715 2024-04-20T01:48:11.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.402 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.402 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.402 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.402 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113717+0000", 2024-04-20T01:48:11.402 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.402 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.402 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.402 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113717+0000", 2024-04-20T01:48:11.402 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.402 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.402 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.403 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113715+0000", 2024-04-20T01:48:11.403 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:11.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.403 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.403 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.403 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113731+0000", 2024-04-20T01:48:11.403 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6231000000000001e-05 2024-04-20T01:48:11.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.403 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.403 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.403 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113751+0000", 2024-04-20T01:48:11.403 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9666000000000001e-05 2024-04-20T01:48:11.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.403 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.403 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.404 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113762+0000", 2024-04-20T01:48:11.404 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0596e-05 2024-04-20T01:48:11.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.404 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.404 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113837+0000", 2024-04-20T01:48:11.404 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5783000000000003e-05 2024-04-20T01:48:11.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.404 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.404 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113990+0000", 2024-04-20T01:48:11.404 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000152118 2024-04-20T01:48:11.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.404 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.404 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.405 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.127616+0000", 2024-04-20T01:48:11.405 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013625906 2024-04-20T01:48:11.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.405 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.405 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.405 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.129856+0000", 2024-04-20T01:48:11.405 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022408490000000001 2024-04-20T01:48:11.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.405 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.405 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:11.405 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.134361+0000", 2024-04-20T01:48:11.405 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045043349999999999 2024-04-20T01:48:11.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.405 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.406 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.406 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.134412+0000", 2024-04-20T01:48:11.406 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1310999999999997e-05 2024-04-20T01:48:11.406 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.406 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.406 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.406 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:716 3.bs0 3:dc0bbe39:::smithi16579500-13:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:11.406 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:10.113774+0000", 2024-04-20T01:48:11.406 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 1.2247888630000001, 2024-04-20T01:48:11.406 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015615891, 2024-04-20T01:48:11.406 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.407 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:11.407 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.407 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.407 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.407 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 716 2024-04-20T01:48:11.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.407 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.407 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.407 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.407 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113774+0000", 2024-04-20T01:48:11.407 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.407 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.407 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.408 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113774+0000", 2024-04-20T01:48:11.408 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.408 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.408 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113773+0000", 2024-04-20T01:48:11.408 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:11.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.408 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.408 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113776+0000", 2024-04-20T01:48:11.408 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.61e-06 2024-04-20T01:48:11.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.408 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.409 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.409 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113781+0000", 2024-04-20T01:48:11.409 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6369999999999998e-06 2024-04-20T01:48:11.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.409 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.409 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.409 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113784+0000", 2024-04-20T01:48:11.409 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4489999999999999e-06 2024-04-20T01:48:11.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.409 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.409 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.409 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.114218+0000", 2024-04-20T01:48:11.409 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00043373499999999998 2024-04-20T01:48:11.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.410 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:11.410 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.114253+0000", 2024-04-20T01:48:11.410 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5410000000000001e-05 2024-04-20T01:48:11.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.410 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.410 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.129136+0000", 2024-04-20T01:48:11.410 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014882532 2024-04-20T01:48:11.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.410 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.410 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.410 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.129263+0000", 2024-04-20T01:48:11.410 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000126919 2024-04-20T01:48:11.411 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.411 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.411 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.411 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.129390+0000", 2024-04-20T01:48:11.411 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000126957 2024-04-20T01:48:11.411 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.411 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.411 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.411 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.411 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.411 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:720 3.7s0 3:e950587e:::smithi16579500-49:head [copy-from ver 100] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:11.411 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:10.113830+0000", 2024-04-20T01:48:11.411 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 1.2247328070000001, 2024-04-20T01:48:11.411 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.016212712000000001, 2024-04-20T01:48:11.412 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.412 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:11.412 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.412 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.412 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.412 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 720 2024-04-20T01:48:11.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.412 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.412 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.412 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113830+0000", 2024-04-20T01:48:11.412 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.412 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.412 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.413 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113830+0000", 2024-04-20T01:48:11.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.413 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.413 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.413 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113829+0000", 2024-04-20T01:48:11.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:11.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.413 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.413 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.413 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113833+0000", 2024-04-20T01:48:11.413 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2239999999999997e-06 2024-04-20T01:48:11.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.413 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.413 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.413 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113843+0000", 2024-04-20T01:48:11.414 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6220000000000004e-06 2024-04-20T01:48:11.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.414 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.414 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113847+0000", 2024-04-20T01:48:11.414 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4150000000000001e-06 2024-04-20T01:48:11.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.414 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.414 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113890+0000", 2024-04-20T01:48:11.414 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2287999999999999e-05 2024-04-20T01:48:11.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.414 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.414 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.414 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113987+0000", 2024-04-20T01:48:11.415 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7149000000000001e-05 2024-04-20T01:48:11.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.415 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.415 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.124314+0000", 2024-04-20T01:48:11.415 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010327325 2024-04-20T01:48:11.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.415 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.415 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.128903+0000", 2024-04-20T01:48:11.415 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045891949999999999 2024-04-20T01:48:11.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.415 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.416 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:11.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.129948+0000", 2024-04-20T01:48:11.416 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0010448409999999999 2024-04-20T01:48:11.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.416 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.416 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.416 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.130042+0000", 2024-04-20T01:48:11.416 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4158000000000002e-05 2024-04-20T01:48:11.416 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.416 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.416 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.416 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.416 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:723 3.7s0 3:eb044288:::smithi16579500-44:head [rollback 1a] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:11.417 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:10.113898+0000", 2024-04-20T01:48:11.417 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 1.2246647580000001, 2024-04-20T01:48:11.417 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013284504000000001, 2024-04-20T01:48:11.417 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.417 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:11.417 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.417 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.417 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.417 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 723 2024-04-20T01:48:11.417 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.417 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.417 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.417 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.418 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113898+0000", 2024-04-20T01:48:11.418 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.418 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.418 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.418 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113898+0000", 2024-04-20T01:48:11.418 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.418 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.418 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.418 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113898+0000", 2024-04-20T01:48:11.418 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-04-20T01:48:11.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.419 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.419 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113900+0000", 2024-04-20T01:48:11.419 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1569999999999998e-06 2024-04-20T01:48:11.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.419 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.419 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113904+0000", 2024-04-20T01:48:11.419 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4039999999999997e-06 2024-04-20T01:48:11.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.419 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.419 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.419 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113907+0000", 2024-04-20T01:48:11.419 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.943e-06 2024-04-20T01:48:11.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.115280+0000", 2024-04-20T01:48:11.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0013727259999999999 2024-04-20T01:48:11.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.115377+0000", 2024-04-20T01:48:11.420 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7275000000000006e-05 2024-04-20T01:48:11.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.420 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.420 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.420 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.116058+0000", 2024-04-20T01:48:11.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00068105100000000001 2024-04-20T01:48:11.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.126887+0000", 2024-04-20T01:48:11.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010828723 2024-04-20T01:48:11.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:11.421 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.127106+0000", 2024-04-20T01:48:11.421 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00021916000000000001 2024-04-20T01:48:11.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.421 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.422 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.127182+0000", 2024-04-20T01:48:11.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6301e-05 2024-04-20T01:48:11.422 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.422 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.422 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.422 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.422 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:724 3.7s0 3:eb044288:::smithi16579500-44:head [append 0~0] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:11.422 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:10.113910+0000", 2024-04-20T01:48:11.422 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 1.224652686, 2024-04-20T01:48:11.422 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013446279, 2024-04-20T01:48:11.422 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.423 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:11.423 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.423 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.423 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.423 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 724 2024-04-20T01:48:11.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.423 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.423 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.423 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113910+0000", 2024-04-20T01:48:11.423 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.423 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.423 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.424 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113910+0000", 2024-04-20T01:48:11.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.424 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.424 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.424 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113909+0000", 2024-04-20T01:48:11.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:11.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.424 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.424 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.424 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113911+0000", 2024-04-20T01:48:11.424 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.784e-06 2024-04-20T01:48:11.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.424 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.424 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.424 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113914+0000", 2024-04-20T01:48:11.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1690000000000001e-06 2024-04-20T01:48:11.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113919+0000", 2024-04-20T01:48:11.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.053e-06 2024-04-20T01:48:11.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.116676+0000", 2024-04-20T01:48:11.425 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027574240000000001 2024-04-20T01:48:11.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.425 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.425 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.425 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.116803+0000", 2024-04-20T01:48:11.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000126654 2024-04-20T01:48:11.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.117277+0000", 2024-04-20T01:48:11.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00047413799999999998 2024-04-20T01:48:11.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.426 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.426 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.426 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.126888+0000", 2024-04-20T01:48:11.426 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096116610000000005 2024-04-20T01:48:11.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:11.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.127280+0000", 2024-04-20T01:48:11.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000391264 2024-04-20T01:48:11.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.427 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.427 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.427 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.127356+0000", 2024-04-20T01:48:11.427 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6470999999999993e-05 2024-04-20T01:48:11.427 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.427 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.427 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.427 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.427 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.428 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:11.429 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-20T01:48:11.430 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:11.443+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:11.530 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:11.544+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:11.613 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:11.613 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:11.613 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-04-20T01:48:11.613 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:11.625 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-20T01:48:11.627 INFO:teuthology.orchestra.run.smithi044.stdout:moved 0 / 130 (0%) 2024-04-20T01:48:11.627 INFO:teuthology.orchestra.run.smithi044.stdout:avg 8.125 2024-04-20T01:48:11.627 INFO:teuthology.orchestra.run.smithi044.stdout:stddev 1.86665 -> 1.86665 (expected baseline 2.75993) 2024-04-20T01:48:11.628 INFO:teuthology.orchestra.run.smithi044.stdout:min osd.9 with 5 -> 5 pgs (0.615385 -> 0.615385 * mean) 2024-04-20T01:48:11.628 INFO:teuthology.orchestra.run.smithi044.stdout:max osd.3 with 12 -> 12 pgs (1.47692 -> 1.47692 * mean) 2024-04-20T01:48:11.628 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:48:11.628 INFO:teuthology.orchestra.run.smithi044.stdout:oload 1000 2024-04-20T01:48:11.628 INFO:teuthology.orchestra.run.smithi044.stdout:max_change 0.05 2024-04-20T01:48:11.628 INFO:teuthology.orchestra.run.smithi044.stdout:max_change_osds 4 2024-04-20T01:48:11.628 INFO:teuthology.orchestra.run.smithi044.stdout:average_utilization 0.0004 2024-04-20T01:48:11.628 INFO:teuthology.orchestra.run.smithi044.stdout:overload_utilization 0.0041 2024-04-20T01:48:11.631 INFO:teuthology.orchestra.run.smithi044.stderr:no change 2024-04-20T01:48:11.632 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:11.645+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:11.644 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-04-20T01:48:11.645 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-04-20T01:48:11.733 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:11.747+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:11.770 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:11.771 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:11.771 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-04-20T01:48:11.771 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-04-20T01:48:11.771 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:11.785 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-20T01:48:11.785 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is unset 2024-04-20T01:48:11.786 INFO:tasks.rados.rados.0.smithi165.stdout:280: done (0 left) 2024-04-20T01:48:11.786 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 48 current snap is 20 2024-04-20T01:48:11.786 INFO:tasks.rados.rados.0.smithi165.stdout:281: seq_num 129 ranges {1785856=622592,2408448=655360,3063808=688128,3751936=212992} 2024-04-20T01:48:11.798 INFO:tasks.rados.rados.0.smithi165.stdout:281: writing smithi16579500-48 from 1785856 to 2408448 tid 1 2024-04-20T01:48:11.802 INFO:tasks.rados.rados.0.smithi165.stdout:281: writing smithi16579500-48 from 2408448 to 3063808 tid 2 2024-04-20T01:48:11.806 INFO:tasks.rados.rados.0.smithi165.stdout:281: writing smithi16579500-48 from 3063808 to 3751936 tid 3 2024-04-20T01:48:11.808 INFO:tasks.rados.rados.0.smithi165.stdout:281: writing smithi16579500-48 from 3751936 to 3964928 tid 4 2024-04-20T01:48:11.808 INFO:tasks.rados.rados.0.smithi165.stdout:282: rmattr oid 3 current snap is 20 2024-04-20T01:48:11.809 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 50 current snap is 20 2024-04-20T01:48:11.809 INFO:tasks.rados.rados.0.smithi165.stdout:283: seq_num 130 ranges {3833856=638976,4472832=671744,5144576=802816,5947392=81920} 2024-04-20T01:48:11.833 INFO:tasks.rados.rados.0.smithi165.stdout:283: writing smithi16579500-50 from 3833856 to 4472832 tid 1 2024-04-20T01:48:11.837 INFO:tasks.rados.rados.0.smithi165.stdout:283: writing smithi16579500-50 from 4472832 to 5144576 tid 2 2024-04-20T01:48:11.840 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:11.855+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:11.841 INFO:tasks.rados.rados.0.smithi165.stdout:283: writing smithi16579500-50 from 5144576 to 5947392 tid 3 2024-04-20T01:48:11.842 INFO:tasks.rados.rados.0.smithi165.stdout:283: writing smithi16579500-50 from 5947392 to 6029312 tid 4 2024-04-20T01:48:11.842 INFO:tasks.rados.rados.0.smithi165.stdout:284: snap_remove snap 19 2024-04-20T01:48:11.934 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:11.949+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:11.942 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:11.942 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-04-20T01:48:11.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-04-20T01:48:11.942 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-04-20T01:48:11.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.942 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:215 3.4s0 3:28069c2b:::smithi16579500-39:head [copy-from ver 4] snapc 3=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:48:11.942 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.797216+0000", 2024-04-20T01:48:11.942 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 34.157556151000001, 2024-04-20T01:48:11.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.020029964000000001, 2024-04-20T01:48:11.942 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.942 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:11.942 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.943 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.943 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.943 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 215 2024-04-20T01:48:11.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.943 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797216+0000", 2024-04-20T01:48:11.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797216+0000", 2024-04-20T01:48:11.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797214+0000", 2024-04-20T01:48:11.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:11.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797223+0000", 2024-04-20T01:48:11.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9709999999999993e-06 2024-04-20T01:48:11.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797240+0000", 2024-04-20T01:48:11.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6872e-05 2024-04-20T01:48:11.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797249+0000", 2024-04-20T01:48:11.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1500000000000005e-06 2024-04-20T01:48:11.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797298+0000", 2024-04-20T01:48:11.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9715000000000001e-05 2024-04-20T01:48:11.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.945 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.945 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797408+0000", 2024-04-20T01:48:11.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000109883 2024-04-20T01:48:11.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808715+0000", 2024-04-20T01:48:11.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011306145 2024-04-20T01:48:11.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.812376+0000", 2024-04-20T01:48:11.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0036618029999999999 2024-04-20T01:48:11.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:11.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.817198+0000", 2024-04-20T01:48:11.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048218749999999998 2024-04-20T01:48:11.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.817246+0000", 2024-04-20T01:48:11.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7787000000000002e-05 2024-04-20T01:48:11.947 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.947 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.947 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.948 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:216 3.4s0 3:28069c2b:::smithi16579500-39:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:48:11.948 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.797261+0000", 2024-04-20T01:48:11.948 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 34.157511423000003, 2024-04-20T01:48:11.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012431373000000001, 2024-04-20T01:48:11.948 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.948 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:11.948 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.948 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.948 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.948 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 216 2024-04-20T01:48:11.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.948 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797261+0000", 2024-04-20T01:48:11.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797261+0000", 2024-04-20T01:48:11.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797260+0000", 2024-04-20T01:48:11.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:11.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797266+0000", 2024-04-20T01:48:11.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2140000000000001e-06 2024-04-20T01:48:11.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797275+0000", 2024-04-20T01:48:11.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9309999999999997e-06 2024-04-20T01:48:11.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797282+0000", 2024-04-20T01:48:11.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5169999999999996e-06 2024-04-20T01:48:11.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797635+0000", 2024-04-20T01:48:11.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035326500000000001 2024-04-20T01:48:11.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:11.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797689+0000", 2024-04-20T01:48:11.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3687999999999998e-05 2024-04-20T01:48:11.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.809488+0000", 2024-04-20T01:48:11.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011799417 2024-04-20T01:48:11.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.809591+0000", 2024-04-20T01:48:11.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102729 2024-04-20T01:48:11.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.809692+0000", 2024-04-20T01:48:11.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000101202 2024-04-20T01:48:11.952 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.952 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.952 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.952 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:240 3.4s0 3:28069c2b:::smithi16579500-39:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:11.952 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787662+0000", 2024-04-20T01:48:11.952 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.167110010000002, 2024-04-20T01:48:11.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013455973, 2024-04-20T01:48:11.952 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.952 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:11.952 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.953 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.953 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.953 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 240 2024-04-20T01:48:11.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.953 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787662+0000", 2024-04-20T01:48:11.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787662+0000", 2024-04-20T01:48:11.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787659+0000", 2024-04-20T01:48:11.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999971 2024-04-20T01:48:11.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787676+0000", 2024-04-20T01:48:11.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6371999999999998e-05 2024-04-20T01:48:11.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787703+0000", 2024-04-20T01:48:11.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6902000000000001e-05 2024-04-20T01:48:11.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787718+0000", 2024-04-20T01:48:11.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5231999999999999e-05 2024-04-20T01:48:11.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788023+0000", 2024-04-20T01:48:11.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000304825 2024-04-20T01:48:11.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op must wait for map", 2024-04-20T01:48:11.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788031+0000", 2024-04-20T01:48:11.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7560000000000005e-06 2024-04-20T01:48:11.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791096+0000", 2024-04-20T01:48:11.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0030642849999999999 2024-04-20T01:48:11.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791195+0000", 2024-04-20T01:48:11.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9277000000000005e-05 2024-04-20T01:48:11.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.792028+0000", 2024-04-20T01:48:11.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00083305200000000003 2024-04-20T01:48:11.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.800720+0000", 2024-04-20T01:48:11.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086918859999999994 2024-04-20T01:48:11.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:11.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.801035+0000", 2024-04-20T01:48:11.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00031470199999999998 2024-04-20T01:48:11.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.801118+0000", 2024-04-20T01:48:11.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3541999999999994e-05 2024-04-20T01:48:11.958 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.958 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.958 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.958 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:241 3.4s0 3:28069c2b:::smithi16579500-39:head [rollback 2] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:11.959 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787738+0000", 2024-04-20T01:48:11.959 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.167034068, 2024-04-20T01:48:11.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01358739, 2024-04-20T01:48:11.959 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.959 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:11.959 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.959 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.959 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.959 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 241 2024-04-20T01:48:11.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.959 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787738+0000", 2024-04-20T01:48:11.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787738+0000", 2024-04-20T01:48:11.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787737+0000", 2024-04-20T01:48:11.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:11.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787742+0000", 2024-04-20T01:48:11.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4050000000000002e-06 2024-04-20T01:48:11.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787751+0000", 2024-04-20T01:48:11.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3710000000000006e-06 2024-04-20T01:48:11.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787757+0000", 2024-04-20T01:48:11.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2600000000000002e-06 2024-04-20T01:48:11.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788094+0000", 2024-04-20T01:48:11.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033620200000000002 2024-04-20T01:48:11.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:11.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788099+0000", 2024-04-20T01:48:11.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2469999999999999e-06 2024-04-20T01:48:11.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.792538+0000", 2024-04-20T01:48:11.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0044393949999999996 2024-04-20T01:48:11.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.792627+0000", 2024-04-20T01:48:11.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8646999999999993e-05 2024-04-20T01:48:11.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.793450+0000", 2024-04-20T01:48:11.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00082310900000000004 2024-04-20T01:48:11.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.800753+0000", 2024-04-20T01:48:11.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073032540000000003 2024-04-20T01:48:11.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:11.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.801274+0000", 2024-04-20T01:48:11.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052095799999999997 2024-04-20T01:48:11.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.801325+0000", 2024-04-20T01:48:11.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1251000000000001e-05 2024-04-20T01:48:11.964 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.964 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.964 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.965 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:242 3.4s0 3:28069c2b:::smithi16579500-39:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:11.965 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787763+0000", 2024-04-20T01:48:11.965 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.16700934, 2024-04-20T01:48:11.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013716078, 2024-04-20T01:48:11.965 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.965 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:11.965 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.965 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.965 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.965 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 242 2024-04-20T01:48:11.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.965 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787763+0000", 2024-04-20T01:48:11.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787763+0000", 2024-04-20T01:48:11.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787762+0000", 2024-04-20T01:48:11.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:11.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787765+0000", 2024-04-20T01:48:11.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.084e-06 2024-04-20T01:48:11.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787771+0000", 2024-04-20T01:48:11.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9020000000000004e-06 2024-04-20T01:48:11.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787781+0000", 2024-04-20T01:48:11.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1630000000000002e-06 2024-04-20T01:48:11.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788118+0000", 2024-04-20T01:48:11.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033771499999999999 2024-04-20T01:48:11.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:11.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788123+0000", 2024-04-20T01:48:11.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-04-20T01:48:11.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.793864+0000", 2024-04-20T01:48:11.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0057404309999999998 2024-04-20T01:48:11.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.793978+0000", 2024-04-20T01:48:11.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000114274 2024-04-20T01:48:11.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.794417+0000", 2024-04-20T01:48:11.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00043935000000000003 2024-04-20T01:48:11.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.800777+0000", 2024-04-20T01:48:11.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0063594230000000003 2024-04-20T01:48:11.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:11.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.801420+0000", 2024-04-20T01:48:11.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00064355900000000004 2024-04-20T01:48:11.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.801479+0000", 2024-04-20T01:48:11.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8693000000000002e-05 2024-04-20T01:48:11.970 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.970 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.970 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.970 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:292 3.4s0 3:27ac9303:::smithi16579500-19:head [read 0~1261568,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:11.970 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:43.807455+0000", 2024-04-20T01:48:11.970 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 28.147316714999999, 2024-04-20T01:48:11.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01155932, 2024-04-20T01:48:11.971 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.971 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:11.971 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.971 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.971 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.971 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 292 2024-04-20T01:48:11.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.971 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.971 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807455+0000", 2024-04-20T01:48:11.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807455+0000", 2024-04-20T01:48:11.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807455+0000", 2024-04-20T01:48:11.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:11.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807460+0000", 2024-04-20T01:48:11.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7879999999999997e-06 2024-04-20T01:48:11.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807465+0000", 2024-04-20T01:48:11.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3260000000000003e-06 2024-04-20T01:48:11.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807469+0000", 2024-04-20T01:48:11.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.703e-06 2024-04-20T01:48:11.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807605+0000", 2024-04-20T01:48:11.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013600600000000001 2024-04-20T01:48:11.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:11.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807612+0000", 2024-04-20T01:48:11.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3969999999999997e-06 2024-04-20T01:48:11.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.812370+0000", 2024-04-20T01:48:11.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047582049999999997 2024-04-20T01:48:11.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.812466+0000", 2024-04-20T01:48:11.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5373999999999999e-05 2024-04-20T01:48:11.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.819015+0000", 2024-04-20T01:48:11.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.006549048 2024-04-20T01:48:11.975 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.975 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.975 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.975 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:293 3.4s0 3:27ac9303:::smithi16579500-19:head [read 0~1261568,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:11.975 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:43.807567+0000", 2024-04-20T01:48:11.975 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 28.147204877, 2024-04-20T01:48:11.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012280444999999999, 2024-04-20T01:48:11.975 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.976 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:11.976 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.976 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.976 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.976 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 293 2024-04-20T01:48:11.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.976 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807567+0000", 2024-04-20T01:48:11.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807567+0000", 2024-04-20T01:48:11.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807567+0000", 2024-04-20T01:48:11.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:11.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807573+0000", 2024-04-20T01:48:11.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6050000000000003e-06 2024-04-20T01:48:11.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807583+0000", 2024-04-20T01:48:11.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0274e-05 2024-04-20T01:48:11.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807591+0000", 2024-04-20T01:48:11.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.892e-06 2024-04-20T01:48:11.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807645+0000", 2024-04-20T01:48:11.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3202000000000003e-05 2024-04-20T01:48:11.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:11.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807651+0000", 2024-04-20T01:48:11.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6699999999999997e-06 2024-04-20T01:48:11.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.812731+0000", 2024-04-20T01:48:11.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0050797480000000002 2024-04-20T01:48:11.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.812817+0000", 2024-04-20T01:48:11.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6415999999999994e-05 2024-04-20T01:48:11.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.819848+0000", 2024-04-20T01:48:11.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0070303809999999996 2024-04-20T01:48:11.980 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.980 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.980 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.980 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:399 3.4s0 3:2d59a0da:::smithi16579500-10:head [append 0~0] snapc d=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-04-20T01:48:11.980 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.711018+0000", 2024-04-20T01:48:11.980 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 22.243754074999998, 2024-04-20T01:48:11.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014803088000000001, 2024-04-20T01:48:11.980 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.980 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:11.980 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.980 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.980 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.980 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 399 2024-04-20T01:48:11.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.981 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711018+0000", 2024-04-20T01:48:11.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711018+0000", 2024-04-20T01:48:11.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711017+0000", 2024-04-20T01:48:11.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:11.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711023+0000", 2024-04-20T01:48:11.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7760000000000003e-06 2024-04-20T01:48:11.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711036+0000", 2024-04-20T01:48:11.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3754000000000001e-05 2024-04-20T01:48:11.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711044+0000", 2024-04-20T01:48:11.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.435e-06 2024-04-20T01:48:11.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711095+0000", 2024-04-20T01:48:11.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1038999999999997e-05 2024-04-20T01:48:11.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711221+0000", 2024-04-20T01:48:11.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000126515 2024-04-20T01:48:11.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711890+0000", 2024-04-20T01:48:11.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00066828000000000005 2024-04-20T01:48:11.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714226+0000", 2024-04-20T01:48:11.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023359040000000002 2024-04-20T01:48:11.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:11.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.725772+0000", 2024-04-20T01:48:11.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011546529 2024-04-20T01:48:11.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.725821+0000", 2024-04-20T01:48:11.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9129999999999999e-05 2024-04-20T01:48:11.984 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.984 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.985 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.985 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:400 3.4s0 3:2d59a0da:::smithi16579500-10:head [rollback b] snapc d=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-04-20T01:48:11.985 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.711049+0000", 2024-04-20T01:48:11.985 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 22.243723452000001, 2024-04-20T01:48:11.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.019555619, 2024-04-20T01:48:11.985 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.985 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:11.985 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.985 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.985 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.985 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 400 2024-04-20T01:48:11.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.985 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711049+0000", 2024-04-20T01:48:11.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711049+0000", 2024-04-20T01:48:11.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711048+0000", 2024-04-20T01:48:11.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:11.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711050+0000", 2024-04-20T01:48:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.905e-06 2024-04-20T01:48:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711054+0000", 2024-04-20T01:48:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0280000000000002e-06 2024-04-20T01:48:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711058+0000", 2024-04-20T01:48:11.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5159999999999999e-06 2024-04-20T01:48:11.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712434+0000", 2024-04-20T01:48:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001375937 2024-04-20T01:48:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712543+0000", 2024-04-20T01:48:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010908000000000001 2024-04-20T01:48:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713115+0000", 2024-04-20T01:48:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00057183500000000003 2024-04-20T01:48:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.715444+0000", 2024-04-20T01:48:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023292930000000001 2024-04-20T01:48:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730509+0000", 2024-04-20T01:48:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015064546 2024-04-20T01:48:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730604+0000", 2024-04-20T01:48:11.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5814000000000001e-05 2024-04-20T01:48:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:401 3.4s0 3:2d59a0da:::smithi16579500-10:head [append 0~0] snapc d=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-04-20T01:48:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.711061+0000", 2024-04-20T01:48:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 22.243711098999999, 2024-04-20T01:48:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.019877855999999999, 2024-04-20T01:48:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.992 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 401 2024-04-20T01:48:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711061+0000", 2024-04-20T01:48:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711061+0000", 2024-04-20T01:48:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711061+0000", 2024-04-20T01:48:11.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:11.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711064+0000", 2024-04-20T01:48:11.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8480000000000001e-06 2024-04-20T01:48:11.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:11.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711067+0000", 2024-04-20T01:48:11.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5559999999999999e-06 2024-04-20T01:48:11.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:11.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711070+0000", 2024-04-20T01:48:11.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.044e-06 2024-04-20T01:48:11.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:11.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713427+0000", 2024-04-20T01:48:11.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002356585 2024-04-20T01:48:11.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:11.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713534+0000", 2024-04-20T01:48:11.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010668200000000001 2024-04-20T01:48:11.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713981+0000", 2024-04-20T01:48:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00044727999999999999 2024-04-20T01:48:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.718977+0000", 2024-04-20T01:48:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0049960330000000004 2024-04-20T01:48:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730869+0000", 2024-04-20T01:48:11.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011892606 2024-04-20T01:48:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730939+0000", 2024-04-20T01:48:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9470999999999999e-05 2024-04-20T01:48:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:479 3.4s0 3:257d96c1:::smithi16579500-15:19 [read 0~245760,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.727997+0000", 2024-04-20T01:48:11.997 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.226774748, 2024-04-20T01:48:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.066708136000000001, 2024-04-20T01:48:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 479 2024-04-20T01:48:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.727997+0000", 2024-04-20T01:48:11.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.727997+0000", 2024-04-20T01:48:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.727995+0000", 2024-04-20T01:48:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728006+0000", 2024-04-20T01:48:11.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0479000000000001e-05 2024-04-20T01:48:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728024+0000", 2024-04-20T01:48:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7819999999999999e-05 2024-04-20T01:48:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728037+0000", 2024-04-20T01:48:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3691e-05 2024-04-20T01:48:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.730187+0000", 2024-04-20T01:48:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021495189999999999 2024-04-20T01:48:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-04-20T01:48:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.730193+0000", 2024-04-20T01:48:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4060000000000002e-06 2024-04-20T01:48:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.784252+0000", 2024-04-20T01:48:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.054058782999999999 2024-04-20T01:48:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-04-20T01:48:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.784915+0000", 2024-04-20T01:48:12.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00066283800000000001 2024-04-20T01:48:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.791659+0000", 2024-04-20T01:48:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0067437089999999996 2024-04-20T01:48:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.791831+0000", 2024-04-20T01:48:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000172473 2024-04-20T01:48:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.794706+0000", 2024-04-20T01:48:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0028743559999999998 2024-04-20T01:48:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:480 3.4s0 3:257d96c1:::smithi16579500-15:19 [read 0~245760,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.728048+0000", 2024-04-20T01:48:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.226724422, 2024-04-20T01:48:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.068695271000000002, 2024-04-20T01:48:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:12.003 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:12.004 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:12.004 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 480 2024-04-20T01:48:12.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.004 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728048+0000", 2024-04-20T01:48:12.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728048+0000", 2024-04-20T01:48:12.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.005 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.005 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728047+0000", 2024-04-20T01:48:12.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:12.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.005 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.005 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728053+0000", 2024-04-20T01:48:12.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6529999999999999e-06 2024-04-20T01:48:12.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.005 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728061+0000", 2024-04-20T01:48:12.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.867e-06 2024-04-20T01:48:12.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.006 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728068+0000", 2024-04-20T01:48:12.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.596e-06 2024-04-20T01:48:12.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.006 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.730233+0000", 2024-04-20T01:48:12.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021647680000000001 2024-04-20T01:48:12.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-04-20T01:48:12.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.730235+0000", 2024-04-20T01:48:12.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.694e-06 2024-04-20T01:48:12.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.785790+0000", 2024-04-20T01:48:12.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055555805 2024-04-20T01:48:12.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.008 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-04-20T01:48:12.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.785867+0000", 2024-04-20T01:48:12.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6564999999999996e-05 2024-04-20T01:48:12.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.008 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.794785+0000", 2024-04-20T01:48:12.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0089179169999999992 2024-04-20T01:48:12.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:12.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.794908+0000", 2024-04-20T01:48:12.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012334100000000001 2024-04-20T01:48:12.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.796743+0000", 2024-04-20T01:48:12.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0018348559999999999 2024-04-20T01:48:12.009 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.009 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.009 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.009 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:481 3.4s0 3:257d96c1:::smithi16579500-15:19 [read 0~245760,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:12.009 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.728074+0000", 2024-04-20T01:48:12.010 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.226697883, 2024-04-20T01:48:12.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.070786587999999998, 2024-04-20T01:48:12.010 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.010 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:12.010 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:12.010 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:12.010 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:12.010 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 481 2024-04-20T01:48:12.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.010 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.010 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.010 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728074+0000", 2024-04-20T01:48:12.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728074+0000", 2024-04-20T01:48:12.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728074+0000", 2024-04-20T01:48:12.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:12.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728077+0000", 2024-04-20T01:48:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4630000000000001e-06 2024-04-20T01:48:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728082+0000", 2024-04-20T01:48:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9280000000000001e-06 2024-04-20T01:48:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728085+0000", 2024-04-20T01:48:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4809999999999998e-06 2024-04-20T01:48:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.730277+0000", 2024-04-20T01:48:12.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021910639999999999 2024-04-20T01:48:12.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-04-20T01:48:12.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.730277+0000", 2024-04-20T01:48:12.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.99e-07 2024-04-20T01:48:12.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.786625+0000", 2024-04-20T01:48:12.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056347795999999999 2024-04-20T01:48:12.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.014 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-04-20T01:48:12.014 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.786743+0000", 2024-04-20T01:48:12.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011777600000000001 2024-04-20T01:48:12.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.014 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.014 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.796834+0000", 2024-04-20T01:48:12.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010091086000000001 2024-04-20T01:48:12.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.014 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:12.014 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.796945+0000", 2024-04-20T01:48:12.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000110646 2024-04-20T01:48:12.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.015 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.015 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.798861+0000", 2024-04-20T01:48:12.015 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019162999999999999 2024-04-20T01:48:12.015 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.015 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.015 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.015 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.015 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:482 3.4s0 3:27ac9303:::smithi16579500-19:head [append 0~802816] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:12.015 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.753140+0000", 2024-04-20T01:48:12.015 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.201631742, 2024-04-20T01:48:12.015 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.064361493000000006, 2024-04-20T01:48:12.015 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.015 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:12.016 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:12.016 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:12.016 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:12.016 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 482 2024-04-20T01:48:12.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.016 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.753140+0000", 2024-04-20T01:48:12.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.753140+0000", 2024-04-20T01:48:12.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.753137+0000", 2024-04-20T01:48:12.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999967 2024-04-20T01:48:12.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.754063+0000", 2024-04-20T01:48:12.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00092535499999999999 2024-04-20T01:48:12.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.754087+0000", 2024-04-20T01:48:12.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4181000000000001e-05 2024-04-20T01:48:12.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.018 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.018 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.754099+0000", 2024-04-20T01:48:12.018 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2594e-05 2024-04-20T01:48:12.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.018 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.018 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.754174+0000", 2024-04-20T01:48:12.018 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4932999999999996e-05 2024-04-20T01:48:12.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.018 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-04-20T01:48:12.019 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.754180+0000", 2024-04-20T01:48:12.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2820000000000004e-06 2024-04-20T01:48:12.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.019 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.019 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.786868+0000", 2024-04-20T01:48:12.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032688506999999999 2024-04-20T01:48:12.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.019 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-04-20T01:48:12.019 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.787722+0000", 2024-04-20T01:48:12.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00085391199999999997 2024-04-20T01:48:12.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.811320+0000", 2024-04-20T01:48:12.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.023598219 2024-04-20T01:48:12.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:12.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.811471+0000", 2024-04-20T01:48:12.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015034900000000001 2024-04-20T01:48:12.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.812850+0000", 2024-04-20T01:48:12.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0013796290000000001 2024-04-20T01:48:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.816794+0000", 2024-04-20T01:48:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0039441110000000001 2024-04-20T01:48:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.817128+0000", 2024-04-20T01:48:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000333543 2024-04-20T01:48:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.817502+0000", 2024-04-20T01:48:12.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00037399800000000001 2024-04-20T01:48:12.022 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.022 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.022 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.022 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:483 3.4s0 3:27ac9303:::smithi16579500-19:head [append 0~638976] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:12.022 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.756616+0000", 2024-04-20T01:48:12.022 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.198155797, 2024-04-20T01:48:12.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.064169961999999997, 2024-04-20T01:48:12.022 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.022 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:12.022 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:12.022 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:12.023 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:12.023 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 483 2024-04-20T01:48:12.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.023 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.023 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.023 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.756616+0000", 2024-04-20T01:48:12.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.023 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.024 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.756616+0000", 2024-04-20T01:48:12.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.024 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.024 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.756615+0000", 2024-04-20T01:48:12.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:12.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.024 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.024 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757340+0000", 2024-04-20T01:48:12.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00072406300000000005 2024-04-20T01:48:12.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757351+0000", 2024-04-20T01:48:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1399e-05 2024-04-20T01:48:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757357+0000", 2024-04-20T01:48:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2770000000000002e-06 2024-04-20T01:48:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757432+0000", 2024-04-20T01:48:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4639000000000001e-05 2024-04-20T01:48:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-04-20T01:48:12.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757434+0000", 2024-04-20T01:48:12.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3350000000000001e-06 2024-04-20T01:48:12.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.787877+0000", 2024-04-20T01:48:12.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.030443274999999999 2024-04-20T01:48:12.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-04-20T01:48:12.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.788002+0000", 2024-04-20T01:48:12.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012432900000000001 2024-04-20T01:48:12.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.814493+0000", 2024-04-20T01:48:12.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.026491297 2024-04-20T01:48:12.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:12.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.814665+0000", 2024-04-20T01:48:12.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017212799999999999 2024-04-20T01:48:12.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.815607+0000", 2024-04-20T01:48:12.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000942212 2024-04-20T01:48:12.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.817608+0000", 2024-04-20T01:48:12.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020010269999999998 2024-04-20T01:48:12.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:12.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.820465+0000", 2024-04-20T01:48:12.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0028566260000000001 2024-04-20T01:48:12.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.820786+0000", 2024-04-20T01:48:12.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00032123299999999998 2024-04-20T01:48:12.029 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.029 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.029 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.029 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:484 3.4s0 3:27ac9303:::smithi16579500-19:head [append 0~131072] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:12.029 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.757369+0000", 2024-04-20T01:48:12.029 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.197403340999999, 2024-04-20T01:48:12.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.066086373000000004, 2024-04-20T01:48:12.029 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.029 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:12.029 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:12.029 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:12.029 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:12.030 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 484 2024-04-20T01:48:12.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.030 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757369+0000", 2024-04-20T01:48:12.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757369+0000", 2024-04-20T01:48:12.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757368+0000", 2024-04-20T01:48:12.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:12.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757524+0000", 2024-04-20T01:48:12.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000155892 2024-04-20T01:48:12.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757533+0000", 2024-04-20T01:48:12.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7739999999999993e-06 2024-04-20T01:48:12.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757538+0000", 2024-04-20T01:48:12.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9099999999999996e-06 2024-04-20T01:48:12.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757586+0000", 2024-04-20T01:48:12.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7988999999999997e-05 2024-04-20T01:48:12.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-04-20T01:48:12.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757588+0000", 2024-04-20T01:48:12.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.922e-06 2024-04-20T01:48:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.788061+0000", 2024-04-20T01:48:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.030473666999999999 2024-04-20T01:48:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-04-20T01:48:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.788190+0000", 2024-04-20T01:48:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000128415 2024-04-20T01:48:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.817711+0000", 2024-04-20T01:48:12.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.029521546999999999 2024-04-20T01:48:12.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:12.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.817811+0000", 2024-04-20T01:48:12.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9242999999999998e-05 2024-04-20T01:48:12.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.818379+0000", 2024-04-20T01:48:12.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000568039 2024-04-20T01:48:12.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.823163+0000", 2024-04-20T01:48:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004784122 2024-04-20T01:48:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.823230+0000", 2024-04-20T01:48:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7317000000000006e-05 2024-04-20T01:48:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.823455+0000", 2024-04-20T01:48:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000225076 2024-04-20T01:48:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.035 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.036 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:485 3.4s0 3:27ac9303:::smithi16579500-19:head [setxattr _header (57) in=7b] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:12.036 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.757544+0000", 2024-04-20T01:48:12.036 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.197228234000001, 2024-04-20T01:48:12.036 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.068448555999999994, 2024-04-20T01:48:12.036 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.036 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:12.036 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:12.036 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:12.036 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:12.036 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 485 2024-04-20T01:48:12.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.036 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757544+0000", 2024-04-20T01:48:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757544+0000", 2024-04-20T01:48:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757544+0000", 2024-04-20T01:48:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757547+0000", 2024-04-20T01:48:12.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.089e-06 2024-04-20T01:48:12.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757556+0000", 2024-04-20T01:48:12.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7119999999999995e-06 2024-04-20T01:48:12.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757560+0000", 2024-04-20T01:48:12.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3830000000000001e-06 2024-04-20T01:48:12.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757618+0000", 2024-04-20T01:48:12.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.838e-05 2024-04-20T01:48:12.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-04-20T01:48:12.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757619+0000", 2024-04-20T01:48:12.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6899999999999996e-07 2024-04-20T01:48:12.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.788519+0000", 2024-04-20T01:48:12.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.030900542999999999 2024-04-20T01:48:12.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-04-20T01:48:12.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.788645+0000", 2024-04-20T01:48:12.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012514399999999999 2024-04-20T01:48:12.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.819052+0000", 2024-04-20T01:48:12.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.030407828000000001 2024-04-20T01:48:12.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:12.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.819150+0000", 2024-04-20T01:48:12.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7280000000000004e-05 2024-04-20T01:48:12.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.819689+0000", 2024-04-20T01:48:12.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053913300000000002 2024-04-20T01:48:12.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.824654+0000", 2024-04-20T01:48:12.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0049648260000000003 2024-04-20T01:48:12.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:12.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.825598+0000", 2024-04-20T01:48:12.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00094450800000000004 2024-04-20T01:48:12.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.825992+0000", 2024-04-20T01:48:12.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00039428400000000001 2024-04-20T01:48:12.042 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.042 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.042 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.042 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:486 3.4s0 3:27ac9303:::smithi16579500-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:12.042 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.757577+0000", 2024-04-20T01:48:12.042 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.197195549, 2024-04-20T01:48:12.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.073599585999999995, 2024-04-20T01:48:12.043 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.043 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:12.043 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:12.043 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:12.043 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:12.043 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 486 2024-04-20T01:48:12.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.043 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757577+0000", 2024-04-20T01:48:12.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757577+0000", 2024-04-20T01:48:12.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757576+0000", 2024-04-20T01:48:12.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:12.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757578+0000", 2024-04-20T01:48:12.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.075e-06 2024-04-20T01:48:12.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757583+0000", 2024-04-20T01:48:12.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.51e-06 2024-04-20T01:48:12.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757586+0000", 2024-04-20T01:48:12.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4580000000000002e-06 2024-04-20T01:48:12.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.045 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.045 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757640+0000", 2024-04-20T01:48:12.045 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3338999999999999e-05 2024-04-20T01:48:12.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-04-20T01:48:12.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757640+0000", 2024-04-20T01:48:12.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.61e-07 2024-04-20T01:48:12.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.788692+0000", 2024-04-20T01:48:12.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.031051856999999999 2024-04-20T01:48:12.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.046 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.046 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-04-20T01:48:12.046 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.788773+0000", 2024-04-20T01:48:12.046 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0569999999999996e-05 2024-04-20T01:48:12.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.819959+0000", 2024-04-20T01:48:12.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.031186445 2024-04-20T01:48:12.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:12.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.820117+0000", 2024-04-20T01:48:12.047 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015805999999999999 2024-04-20T01:48:12.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.047 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.047 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.047 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.826421+0000", 2024-04-20T01:48:12.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0063032319999999998 2024-04-20T01:48:12.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:12.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.826576+0000", 2024-04-20T01:48:12.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015581500000000001 2024-04-20T01:48:12.048 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.048 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.048 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.048 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.831176+0000", 2024-04-20T01:48:12.048 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004599812 2024-04-20T01:48:12.048 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.048 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.048 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.049 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDPGRecoveryDeleteReply(3.4s0 e58,57 3:22cffa5d:::smithi16579500-2:7,41'41)", 2024-04-20T01:48:12.049 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.810383+0000", 2024-04-20T01:48:12.049 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.144389212, 2024-04-20T01:48:12.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011240485, 2024-04-20T01:48:12.049 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.049 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.049 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.049 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.049 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810383+0000", 2024-04-20T01:48:12.049 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.049 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810383+0000", 2024-04-20T01:48:12.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810382+0000", 2024-04-20T01:48:12.050 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:12.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.050 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.050 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.050 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810389+0000", 2024-04-20T01:48:12.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8950000000000001e-06 2024-04-20T01:48:12.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810403+0000", 2024-04-20T01:48:12.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4149e-05 2024-04-20T01:48:12.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.051 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810409+0000", 2024-04-20T01:48:12.051 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1310000000000003e-06 2024-04-20T01:48:12.051 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.051 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.051 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.821321+0000", 2024-04-20T01:48:12.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010911666 2024-04-20T01:48:12.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.052 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.052 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.821623+0000", 2024-04-20T01:48:12.052 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00030231000000000002 2024-04-20T01:48:12.052 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.052 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.052 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.052 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.052 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDPGRecoveryDeleteReply(3.4s0 e58,57 3:22cffa5d:::smithi16579500-2:7,41'41)", 2024-04-20T01:48:12.052 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.810504+0000", 2024-04-20T01:48:12.052 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 12.144268496, 2024-04-20T01:48:12.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012539292, 2024-04-20T01:48:12.053 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.053 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.053 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810504+0000", 2024-04-20T01:48:12.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.053 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810504+0000", 2024-04-20T01:48:12.053 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.053 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.053 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810503+0000", 2024-04-20T01:48:12.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:12.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810508+0000", 2024-04-20T01:48:12.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3460000000000004e-06 2024-04-20T01:48:12.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810515+0000", 2024-04-20T01:48:12.054 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3320000000000003e-06 2024-04-20T01:48:12.054 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.054 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.054 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.054 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810521+0000", 2024-04-20T01:48:12.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.083e-06 2024-04-20T01:48:12.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.822436+0000", 2024-04-20T01:48:12.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011915179 2024-04-20T01:48:12.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.055 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.055 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.055 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.823043+0000", 2024-04-20T01:48:12.055 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00060685600000000002 2024-04-20T01:48:12.055 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.055 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.055 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.056 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.056 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.056 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:12.057 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-20T01:48:12.058 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:12.071+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:12.066 INFO:teuthology.orchestra.run.smithi044.stdout:moved 0 / 130 (0%) 2024-04-20T01:48:12.067 INFO:teuthology.orchestra.run.smithi044.stdout:avg 8.125 2024-04-20T01:48:12.067 INFO:teuthology.orchestra.run.smithi044.stdout:stddev 1.86665 -> 1.86665 (expected baseline 2.75993) 2024-04-20T01:48:12.067 INFO:teuthology.orchestra.run.smithi044.stdout:min osd.9 with 5 -> 5 pgs (0.615385 -> 0.615385 * mean) 2024-04-20T01:48:12.067 INFO:teuthology.orchestra.run.smithi044.stdout:max osd.3 with 12 -> 12 pgs (1.47692 -> 1.47692 * mean) 2024-04-20T01:48:12.067 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:48:12.067 INFO:teuthology.orchestra.run.smithi044.stdout:oload 1000 2024-04-20T01:48:12.067 INFO:teuthology.orchestra.run.smithi044.stdout:max_change 3 2024-04-20T01:48:12.067 INFO:teuthology.orchestra.run.smithi044.stdout:max_change_osds 4 2024-04-20T01:48:12.067 INFO:teuthology.orchestra.run.smithi044.stdout:average_utilization 0.0004 2024-04-20T01:48:12.067 INFO:teuthology.orchestra.run.smithi044.stdout:overload_utilization 0.0041 2024-04-20T01:48:12.070 INFO:teuthology.orchestra.run.smithi044.stderr:no change 2024-04-20T01:48:12.085 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-04-20T01:48:12.085 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-04-20T01:48:12.158 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:12.172+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:12.217 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:12.217 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:12.217 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-04-20T01:48:12.217 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:12.226 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-20T01:48:12.230 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-20T01:48:12.258 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:48:12.272+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:48:12.359 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:12.373+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:12.398 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:48:12.417 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:12.417 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:12.417 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-04-20T01:48:12.417 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-04-20T01:48:12.417 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:12.420 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:48:12.429 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-20T01:48:12.442 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:48:12.460 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:12.474+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:12.465 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:48:12.486 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:48:12.505 INFO:teuthology.orchestra.run.smithi044.stdout:moved 0 / 130 (0%) 2024-04-20T01:48:12.505 INFO:teuthology.orchestra.run.smithi044.stdout:avg 8.125 2024-04-20T01:48:12.505 INFO:teuthology.orchestra.run.smithi044.stdout:stddev 1.86665 -> 1.86665 (expected baseline 2.75993) 2024-04-20T01:48:12.505 INFO:teuthology.orchestra.run.smithi044.stdout:min osd.9 with 5 -> 5 pgs (0.615385 -> 0.615385 * mean) 2024-04-20T01:48:12.505 INFO:teuthology.orchestra.run.smithi044.stdout:max osd.3 with 12 -> 12 pgs (1.47692 -> 1.47692 * mean) 2024-04-20T01:48:12.505 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:48:12.505 INFO:teuthology.orchestra.run.smithi044.stdout:oload 1000 2024-04-20T01:48:12.506 INFO:teuthology.orchestra.run.smithi044.stdout:max_change 3 2024-04-20T01:48:12.506 INFO:teuthology.orchestra.run.smithi044.stdout:max_change_osds 4 2024-04-20T01:48:12.506 INFO:teuthology.orchestra.run.smithi044.stdout:average_utilization 0.0004 2024-04-20T01:48:12.506 INFO:teuthology.orchestra.run.smithi044.stdout:overload_utilization 0.0041 2024-04-20T01:48:12.507 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:48:12.508 INFO:teuthology.orchestra.run.smithi044.stderr:no change 2024-04-20T01:48:12.521 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-04-20T01:48:12.522 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2024-04-20T01:48:12.529 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:48:12.550 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:48:12.561 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:12.575+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:12.573 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:48:12.582 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:12.582 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-04-20T01:48:12.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-04-20T01:48:12.582 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-04-20T01:48:12.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.582 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.5s4 33/30 ECSubWrite(tid=2, reqid=client.4643.0:11, at_version=0'0, trim_to=0'0, roll_forward_to=32'2))", 2024-04-20T01:48:12.582 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.765877+0000", 2024-04-20T01:48:12.582 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.828759775000002, 2024-04-20T01:48:12.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0075032049999999998, 2024-04-20T01:48:12.582 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.583 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.583 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765877+0000", 2024-04-20T01:48:12.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765877+0000", 2024-04-20T01:48:12.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765877+0000", 2024-04-20T01:48:12.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:12.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765904+0000", 2024-04-20T01:48:12.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7461e-05 2024-04-20T01:48:12.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765927+0000", 2024-04-20T01:48:12.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2697999999999999e-05 2024-04-20T01:48:12.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765939+0000", 2024-04-20T01:48:12.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2142e-05 2024-04-20T01:48:12.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.768205+0000", 2024-04-20T01:48:12.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022662239999999998 2024-04-20T01:48:12.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.768221+0000", 2024-04-20T01:48:12.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6161e-05 2024-04-20T01:48:12.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.773346+0000", 2024-04-20T01:48:12.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051250469999999998 2024-04-20T01:48:12.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.773380+0000", 2024-04-20T01:48:12.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3760999999999997e-05 2024-04-20T01:48:12.586 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.587 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.587 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.587 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/30 ECSubWrite(tid=2, reqid=client.4643.0:38, at_version=0'0, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:48:12.587 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.777270+0000", 2024-04-20T01:48:12.587 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.817366872000001, 2024-04-20T01:48:12.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0057051339999999997, 2024-04-20T01:48:12.587 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.587 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.587 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777270+0000", 2024-04-20T01:48:12.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777270+0000", 2024-04-20T01:48:12.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777268+0000", 2024-04-20T01:48:12.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:12.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777287+0000", 2024-04-20T01:48:12.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9128999999999999e-05 2024-04-20T01:48:12.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777309+0000", 2024-04-20T01:48:12.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1475999999999999e-05 2024-04-20T01:48:12.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777323+0000", 2024-04-20T01:48:12.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4076e-05 2024-04-20T01:48:12.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777375+0000", 2024-04-20T01:48:12.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2685999999999997e-05 2024-04-20T01:48:12.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.777389+0000", 2024-04-20T01:48:12.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3369e-05 2024-04-20T01:48:12.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.782943+0000", 2024-04-20T01:48:12.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055546110000000001 2024-04-20T01:48:12.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.782975+0000", 2024-04-20T01:48:12.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1569000000000002e-05 2024-04-20T01:48:12.591 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.591 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.591 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.591 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/30 ECSubWrite(tid=4, reqid=client.4643.0:58, at_version=33'3, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:48:12.591 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.780211+0000", 2024-04-20T01:48:12.591 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.814426185999999, 2024-04-20T01:48:12.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0098815680000000003, 2024-04-20T01:48:12.591 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.592 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.592 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780211+0000", 2024-04-20T01:48:12.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780211+0000", 2024-04-20T01:48:12.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780209+0000", 2024-04-20T01:48:12.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:12.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780307+0000", 2024-04-20T01:48:12.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8575999999999999e-05 2024-04-20T01:48:12.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780328+0000", 2024-04-20T01:48:12.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0537999999999999e-05 2024-04-20T01:48:12.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780340+0000", 2024-04-20T01:48:12.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1997999999999999e-05 2024-04-20T01:48:12.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780423+0000", 2024-04-20T01:48:12.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2812999999999995e-05 2024-04-20T01:48:12.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780434+0000", 2024-04-20T01:48:12.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1028e-05 2024-04-20T01:48:12.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790048+0000", 2024-04-20T01:48:12.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096147339999999998 2024-04-20T01:48:12.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790092+0000", 2024-04-20T01:48:12.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3649999999999997e-05 2024-04-20T01:48:12.595 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.595 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.595 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.595 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/30 ECSubWrite(tid=5, reqid=client.4643.0:59, at_version=33'4, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:48:12.595 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.781201+0000", 2024-04-20T01:48:12.595 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.813435947999999, 2024-04-20T01:48:12.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0089164740000000006, 2024-04-20T01:48:12.595 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.596 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.596 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781201+0000", 2024-04-20T01:48:12.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781201+0000", 2024-04-20T01:48:12.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781200+0000", 2024-04-20T01:48:12.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:12.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781219+0000", 2024-04-20T01:48:12.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8748000000000001e-05 2024-04-20T01:48:12.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781236+0000", 2024-04-20T01:48:12.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7251000000000001e-05 2024-04-20T01:48:12.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781246+0000", 2024-04-20T01:48:12.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0026e-05 2024-04-20T01:48:12.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.782742+0000", 2024-04-20T01:48:12.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001496079 2024-04-20T01:48:12.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.782753+0000", 2024-04-20T01:48:12.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0767999999999999e-05 2024-04-20T01:48:12.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.599 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790103+0000", 2024-04-20T01:48:12.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073499259999999997 2024-04-20T01:48:12.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.599 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790117+0000", 2024-04-20T01:48:12.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4729000000000001e-05 2024-04-20T01:48:12.599 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.599 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.599 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.599 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.5s4 33/30 ECSubWrite(tid=5, reqid=client.4643.0:65, at_version=33'4, trim_to=0'0, roll_forward_to=32'2))", 2024-04-20T01:48:12.599 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.783666+0000", 2024-04-20T01:48:12.600 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.810970927, 2024-04-20T01:48:12.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0063804899999999999, 2024-04-20T01:48:12.600 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.600 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.600 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783666+0000", 2024-04-20T01:48:12.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.600 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.600 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783666+0000", 2024-04-20T01:48:12.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783665+0000", 2024-04-20T01:48:12.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:12.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783685+0000", 2024-04-20T01:48:12.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9735e-05 2024-04-20T01:48:12.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783701+0000", 2024-04-20T01:48:12.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6175e-05 2024-04-20T01:48:12.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.602 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.602 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783710+0000", 2024-04-20T01:48:12.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3649999999999993e-06 2024-04-20T01:48:12.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.602 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.602 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783800+0000", 2024-04-20T01:48:12.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0061000000000004e-05 2024-04-20T01:48:12.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783809+0000", 2024-04-20T01:48:12.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0750000000000004e-06 2024-04-20T01:48:12.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790003+0000", 2024-04-20T01:48:12.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0061941590000000003 2024-04-20T01:48:12.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790046+0000", 2024-04-20T01:48:12.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2831999999999999e-05 2024-04-20T01:48:12.603 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.603 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.604 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.604 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs5 33/30 ECSubWrite(tid=0, reqid=client.4643.0:106, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:48:12.604 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.825844+0000", 2024-04-20T01:48:12.604 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.768792965000003, 2024-04-20T01:48:12.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0062702069999999999, 2024-04-20T01:48:12.604 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.604 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.604 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.825844+0000", 2024-04-20T01:48:12.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.605 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.605 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.825844+0000", 2024-04-20T01:48:12.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.605 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.605 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.825843+0000", 2024-04-20T01:48:12.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:12.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.605 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.605 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826258+0000", 2024-04-20T01:48:12.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00041487099999999998 2024-04-20T01:48:12.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826269+0000", 2024-04-20T01:48:12.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0264e-05 2024-04-20T01:48:12.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826275+0000", 2024-04-20T01:48:12.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3010000000000003e-06 2024-04-20T01:48:12.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826329+0000", 2024-04-20T01:48:12.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4036999999999998e-05 2024-04-20T01:48:12.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826337+0000", 2024-04-20T01:48:12.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7470000000000005e-06 2024-04-20T01:48:12.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.832089+0000", 2024-04-20T01:48:12.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0057520339999999996 2024-04-20T01:48:12.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.832114+0000", 2024-04-20T01:48:12.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5267999999999999e-05 2024-04-20T01:48:12.607 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.607 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.608 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.608 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs5 33/30 ECSubWrite(tid=1, reqid=client.4643.0:107, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:48:12.608 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.826758+0000", 2024-04-20T01:48:12.608 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.767878764999999, 2024-04-20T01:48:12.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0053877639999999997, 2024-04-20T01:48:12.608 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.608 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.608 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826758+0000", 2024-04-20T01:48:12.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826758+0000", 2024-04-20T01:48:12.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826757+0000", 2024-04-20T01:48:12.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:12.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826771+0000", 2024-04-20T01:48:12.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3735e-05 2024-04-20T01:48:12.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826782+0000", 2024-04-20T01:48:12.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1071e-05 2024-04-20T01:48:12.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826792+0000", 2024-04-20T01:48:12.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4970000000000007e-06 2024-04-20T01:48:12.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826933+0000", 2024-04-20T01:48:12.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014126699999999999 2024-04-20T01:48:12.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.611 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.611 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826942+0000", 2024-04-20T01:48:12.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5229999999999995e-06 2024-04-20T01:48:12.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.611 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.611 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.832129+0000", 2024-04-20T01:48:12.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051871479999999999 2024-04-20T01:48:12.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.611 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.612 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.832146+0000", 2024-04-20T01:48:12.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7081000000000001e-05 2024-04-20T01:48:12.612 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.612 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.612 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.612 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.es6 33/30 ECSubWrite(tid=3, reqid=client.4643.0:104, at_version=33'4, trim_to=0'0, roll_forward_to=33'1))", 2024-04-20T01:48:12.612 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.966859+0000", 2024-04-20T01:48:12.612 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.627777268000003, 2024-04-20T01:48:12.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0063969730000000002, 2024-04-20T01:48:12.612 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.612 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.612 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966859+0000", 2024-04-20T01:48:12.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966859+0000", 2024-04-20T01:48:12.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966859+0000", 2024-04-20T01:48:12.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:12.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.614 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966876+0000", 2024-04-20T01:48:12.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7116999999999999e-05 2024-04-20T01:48:12.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.614 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966887+0000", 2024-04-20T01:48:12.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1508999999999999e-05 2024-04-20T01:48:12.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.614 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966894+0000", 2024-04-20T01:48:12.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4219999999999997e-06 2024-04-20T01:48:12.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.615 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.615 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.967342+0000", 2024-04-20T01:48:12.615 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00044805799999999998 2024-04-20T01:48:12.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.615 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.615 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.967352+0000", 2024-04-20T01:48:12.615 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0497e-05 2024-04-20T01:48:12.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.615 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.615 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.973223+0000", 2024-04-20T01:48:12.615 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0058706749999999997 2024-04-20T01:48:12.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.616 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.616 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.973256+0000", 2024-04-20T01:48:12.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3544999999999998e-05 2024-04-20T01:48:12.616 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.616 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.616 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.616 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 38/30 ECSubWrite(tid=73, reqid=osd.4.0:73, at_version=38'22, trim_to=0'0, roll_forward_to=37'18))", 2024-04-20T01:48:12.616 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:39.792927+0000", 2024-04-20T01:48:12.616 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.801709285000001, 2024-04-20T01:48:12.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.005772124, 2024-04-20T01:48:12.617 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.617 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.617 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.792927+0000", 2024-04-20T01:48:12.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.792927+0000", 2024-04-20T01:48:12.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.618 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.792926+0000", 2024-04-20T01:48:12.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:12.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.618 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.792962+0000", 2024-04-20T01:48:12.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6053999999999997e-05 2024-04-20T01:48:12.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.618 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.792982+0000", 2024-04-20T01:48:12.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0200999999999999e-05 2024-04-20T01:48:12.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.792998+0000", 2024-04-20T01:48:12.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6044000000000001e-05 2024-04-20T01:48:12.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.793081+0000", 2024-04-20T01:48:12.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3146999999999995e-05 2024-04-20T01:48:12.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.793094+0000", 2024-04-20T01:48:12.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2741999999999999e-05 2024-04-20T01:48:12.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.620 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.798662+0000", 2024-04-20T01:48:12.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055676750000000002 2024-04-20T01:48:12.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.620 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.620 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.798700+0000", 2024-04-20T01:48:12.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7832e-05 2024-04-20T01:48:12.620 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.620 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.620 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.620 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 38/30 ECSubWrite(tid=74, reqid=osd.4.0:73, at_version=0'0, trim_to=0'0, roll_forward_to=38'22))", 2024-04-20T01:48:12.620 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:39.799501+0000", 2024-04-20T01:48:12.620 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.795135316, 2024-04-20T01:48:12.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0056449669999999999, 2024-04-20T01:48:12.621 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.621 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.621 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.621 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.621 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.799501+0000", 2024-04-20T01:48:12.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.621 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.621 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.799501+0000", 2024-04-20T01:48:12.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.799500+0000", 2024-04-20T01:48:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.799514+0000", 2024-04-20T01:48:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3157e-05 2024-04-20T01:48:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.799533+0000", 2024-04-20T01:48:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9879000000000001e-05 2024-04-20T01:48:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.799544+0000", 2024-04-20T01:48:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0051000000000001e-05 2024-04-20T01:48:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.799598+0000", 2024-04-20T01:48:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4001e-05 2024-04-20T01:48:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.799608+0000", 2024-04-20T01:48:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0234999999999999e-05 2024-04-20T01:48:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.805112+0000", 2024-04-20T01:48:12.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055047100000000003 2024-04-20T01:48:12.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:39.805146+0000", 2024-04-20T01:48:12.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3939000000000002e-05 2024-04-20T01:48:12.624 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.624 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.624 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.624 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 42/30 ECSubWrite(tid=98, reqid=client.4643.0:321, at_version=42'36, trim_to=0'0, roll_forward_to=42'35))", 2024-04-20T01:48:12.625 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:43.943374+0000", 2024-04-20T01:48:12.625 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 28.651262516999999, 2024-04-20T01:48:12.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0059029729999999997, 2024-04-20T01:48:12.625 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.625 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.625 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943374+0000", 2024-04-20T01:48:12.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943374+0000", 2024-04-20T01:48:12.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.626 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.626 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943372+0000", 2024-04-20T01:48:12.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:12.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.626 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.626 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943540+0000", 2024-04-20T01:48:12.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016738300000000001 2024-04-20T01:48:12.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.626 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.626 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943564+0000", 2024-04-20T01:48:12.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4371000000000001e-05 2024-04-20T01:48:12.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943577+0000", 2024-04-20T01:48:12.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3254e-05 2024-04-20T01:48:12.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943661+0000", 2024-04-20T01:48:12.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3727000000000007e-05 2024-04-20T01:48:12.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943674+0000", 2024-04-20T01:48:12.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3097e-05 2024-04-20T01:48:12.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.949246+0000", 2024-04-20T01:48:12.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055718210000000002 2024-04-20T01:48:12.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.949277+0000", 2024-04-20T01:48:12.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1226999999999998e-05 2024-04-20T01:48:12.628 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.628 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.628 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.629 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 47/30 ECSubWrite(tid=114, reqid=client.4643.0:392, at_version=47'49, trim_to=0'0, roll_forward_to=47'47))", 2024-04-20T01:48:12.629 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.746767+0000", 2024-04-20T01:48:12.629 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 23.847869763999999, 2024-04-20T01:48:12.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0061268629999999998, 2024-04-20T01:48:12.629 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.629 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.629 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746767+0000", 2024-04-20T01:48:12.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746767+0000", 2024-04-20T01:48:12.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746766+0000", 2024-04-20T01:48:12.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:12.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746823+0000", 2024-04-20T01:48:12.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6743999999999997e-05 2024-04-20T01:48:12.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746840+0000", 2024-04-20T01:48:12.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7300999999999999e-05 2024-04-20T01:48:12.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746850+0000", 2024-04-20T01:48:12.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0356999999999999e-05 2024-04-20T01:48:12.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746919+0000", 2024-04-20T01:48:12.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8180000000000001e-05 2024-04-20T01:48:12.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746927+0000", 2024-04-20T01:48:12.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3659999999999992e-06 2024-04-20T01:48:12.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.752855+0000", 2024-04-20T01:48:12.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0059277779999999999 2024-04-20T01:48:12.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.752894+0000", 2024-04-20T01:48:12.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9057999999999998e-05 2024-04-20T01:48:12.632 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.632 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.633 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.633 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 60/30 ECSubWrite(tid=144, reqid=client.4643.0:506, at_version=60'65, trim_to=0'0, roll_forward_to=60'62))", 2024-04-20T01:48:12.633 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:01.773724+0000", 2024-04-20T01:48:12.633 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.820912543, 2024-04-20T01:48:12.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0062789609999999996, 2024-04-20T01:48:12.633 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.633 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.633 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.633 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.633 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773724+0000", 2024-04-20T01:48:12.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773724+0000", 2024-04-20T01:48:12.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773723+0000", 2024-04-20T01:48:12.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:12.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773743+0000", 2024-04-20T01:48:12.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0032000000000001e-05 2024-04-20T01:48:12.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773755+0000", 2024-04-20T01:48:12.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1156e-05 2024-04-20T01:48:12.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773763+0000", 2024-04-20T01:48:12.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2870000000000004e-06 2024-04-20T01:48:12.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773830+0000", 2024-04-20T01:48:12.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7269999999999998e-05 2024-04-20T01:48:12.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773840+0000", 2024-04-20T01:48:12.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0284e-05 2024-04-20T01:48:12.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.779970+0000", 2024-04-20T01:48:12.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.006129364 2024-04-20T01:48:12.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.780003+0000", 2024-04-20T01:48:12.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3402000000000003e-05 2024-04-20T01:48:12.636 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.637 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.637 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.637 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 63/30 ECSubWrite(tid=165, reqid=client.4643.0:566, at_version=63'82, trim_to=0'0, roll_forward_to=62'79))", 2024-04-20T01:48:12.637 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:04.760184+0000", 2024-04-20T01:48:12.637 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 7.8344527319999999, 2024-04-20T01:48:12.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0058193819999999997, 2024-04-20T01:48:12.637 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.637 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.637 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.637 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.760184+0000", 2024-04-20T01:48:12.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.760184+0000", 2024-04-20T01:48:12.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.760183+0000", 2024-04-20T01:48:12.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:12.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.760251+0000", 2024-04-20T01:48:12.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8096000000000007e-05 2024-04-20T01:48:12.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.760266+0000", 2024-04-20T01:48:12.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5370999999999999e-05 2024-04-20T01:48:12.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.760278+0000", 2024-04-20T01:48:12.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1208e-05 2024-04-20T01:48:12.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.760351+0000", 2024-04-20T01:48:12.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3227999999999998e-05 2024-04-20T01:48:12.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.760362+0000", 2024-04-20T01:48:12.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0922e-05 2024-04-20T01:48:12.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.765976+0000", 2024-04-20T01:48:12.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0056145140000000001 2024-04-20T01:48:12.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.766003+0000", 2024-04-20T01:48:12.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7081e-05 2024-04-20T01:48:12.641 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.641 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.641 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.641 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 65/30 ECSubWrite(tid=174, reqid=osd.4.0:174, at_version=65'88, trim_to=0'0, roll_forward_to=63'84))", 2024-04-20T01:48:12.641 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:06.763588+0000", 2024-04-20T01:48:12.641 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.8310486240000001, 2024-04-20T01:48:12.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0081115820000000009, 2024-04-20T01:48:12.641 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.641 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.641 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.763588+0000", 2024-04-20T01:48:12.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.763588+0000", 2024-04-20T01:48:12.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.763587+0000", 2024-04-20T01:48:12.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:12.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.763612+0000", 2024-04-20T01:48:12.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5106999999999999e-05 2024-04-20T01:48:12.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.763624+0000", 2024-04-20T01:48:12.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1487e-05 2024-04-20T01:48:12.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.763631+0000", 2024-04-20T01:48:12.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0249999999999997e-06 2024-04-20T01:48:12.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.764079+0000", 2024-04-20T01:48:12.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00044799400000000001 2024-04-20T01:48:12.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.764088+0000", 2024-04-20T01:48:12.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5240000000000006e-06 2024-04-20T01:48:12.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.771664+0000", 2024-04-20T01:48:12.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0075759160000000002 2024-04-20T01:48:12.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.771700+0000", 2024-04-20T01:48:12.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5493000000000001e-05 2024-04-20T01:48:12.645 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.645 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.645 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.645 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 68/30 ECSubWrite(tid=208, reqid=client.4643.0:669, at_version=68'109, trim_to=0'0, roll_forward_to=68'107))", 2024-04-20T01:48:12.645 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.944069+0000", 2024-04-20T01:48:12.645 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.6505673110000001, 2024-04-20T01:48:12.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0057943300000000003, 2024-04-20T01:48:12.645 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.645 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.645 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.944069+0000", 2024-04-20T01:48:12.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.944069+0000", 2024-04-20T01:48:12.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.944069+0000", 2024-04-20T01:48:12.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:12.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.944129+0000", 2024-04-20T01:48:12.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0673e-05 2024-04-20T01:48:12.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.944146+0000", 2024-04-20T01:48:12.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6356999999999999e-05 2024-04-20T01:48:12.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.944168+0000", 2024-04-20T01:48:12.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2364000000000001e-05 2024-04-20T01:48:12.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.944222+0000", 2024-04-20T01:48:12.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3908e-05 2024-04-20T01:48:12.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.944235+0000", 2024-04-20T01:48:12.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.296e-05 2024-04-20T01:48:12.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.949833+0000", 2024-04-20T01:48:12.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055982999999999996 2024-04-20T01:48:12.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.949864+0000", 2024-04-20T01:48:12.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0645999999999998e-05 2024-04-20T01:48:12.649 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.649 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.649 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.649 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 68/30 ECSubWrite(tid=209, reqid=client.4643.0:670, at_version=68'110, trim_to=0'0, roll_forward_to=68'107))", 2024-04-20T01:48:12.649 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.945938+0000", 2024-04-20T01:48:12.649 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.648699208, 2024-04-20T01:48:12.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.006638194, 2024-04-20T01:48:12.649 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.649 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.650 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.945938+0000", 2024-04-20T01:48:12.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.945938+0000", 2024-04-20T01:48:12.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.945937+0000", 2024-04-20T01:48:12.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:12.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.946003+0000", 2024-04-20T01:48:12.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6026999999999997e-05 2024-04-20T01:48:12.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.946019+0000", 2024-04-20T01:48:12.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6458000000000001e-05 2024-04-20T01:48:12.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.946029+0000", 2024-04-20T01:48:12.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.55e-06 2024-04-20T01:48:12.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.946077+0000", 2024-04-20T01:48:12.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8205999999999999e-05 2024-04-20T01:48:12.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.946089+0000", 2024-04-20T01:48:12.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2e-05 2024-04-20T01:48:12.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.952557+0000", 2024-04-20T01:48:12.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0064676919999999997 2024-04-20T01:48:12.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.952576+0000", 2024-04-20T01:48:12.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8838000000000002e-05 2024-04-20T01:48:12.653 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.653 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.653 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.653 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 68/30 ECSubWrite(tid=210, reqid=client.4643.0:671, at_version=68'111, trim_to=0'0, roll_forward_to=68'108))", 2024-04-20T01:48:12.653 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.947171+0000", 2024-04-20T01:48:12.653 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.6474657170000002, 2024-04-20T01:48:12.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0054161299999999999, 2024-04-20T01:48:12.654 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.654 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.654 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.947171+0000", 2024-04-20T01:48:12.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.947171+0000", 2024-04-20T01:48:12.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.947170+0000", 2024-04-20T01:48:12.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:12.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.947194+0000", 2024-04-20T01:48:12.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3507999999999999e-05 2024-04-20T01:48:12.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.947208+0000", 2024-04-20T01:48:12.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4652e-05 2024-04-20T01:48:12.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.947218+0000", 2024-04-20T01:48:12.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3840000000000003e-06 2024-04-20T01:48:12.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.947288+0000", 2024-04-20T01:48:12.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9694000000000001e-05 2024-04-20T01:48:12.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.947300+0000", 2024-04-20T01:48:12.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2734999999999999e-05 2024-04-20T01:48:12.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.952579+0000", 2024-04-20T01:48:12.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052787930000000004 2024-04-20T01:48:12.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.952587+0000", 2024-04-20T01:48:12.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1019999999999997e-06 2024-04-20T01:48:12.657 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.657 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.657 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.657 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 68/30 ECSubWrite(tid=220, reqid=client.4643.0:703, at_version=68'117, trim_to=0'0, roll_forward_to=68'116))", 2024-04-20T01:48:12.658 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:10.059553+0000", 2024-04-20T01:48:12.658 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.5350835009999999, 2024-04-20T01:48:12.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0062243860000000002, 2024-04-20T01:48:12.658 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.658 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.658 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059553+0000", 2024-04-20T01:48:12.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059553+0000", 2024-04-20T01:48:12.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059552+0000", 2024-04-20T01:48:12.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:12.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059733+0000", 2024-04-20T01:48:12.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018084899999999999 2024-04-20T01:48:12.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059773+0000", 2024-04-20T01:48:12.660 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0985999999999997e-05 2024-04-20T01:48:12.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.660 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.660 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059801+0000", 2024-04-20T01:48:12.660 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7727999999999999e-05 2024-04-20T01:48:12.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.660 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.660 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059897+0000", 2024-04-20T01:48:12.660 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5471999999999997e-05 2024-04-20T01:48:12.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.660 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.660 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.660 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059911+0000", 2024-04-20T01:48:12.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4644000000000001e-05 2024-04-20T01:48:12.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.661 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.661 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.065740+0000", 2024-04-20T01:48:12.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0058290709999999999 2024-04-20T01:48:12.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.661 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.661 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.661 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.065778+0000", 2024-04-20T01:48:12.661 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7234999999999998e-05 2024-04-20T01:48:12.661 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.661 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.661 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.662 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 68/30 ECSubWrite(tid=221, reqid=client.4643.0:704, at_version=68'118, trim_to=0'0, roll_forward_to=68'116))", 2024-04-20T01:48:12.662 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:10.064387+0000", 2024-04-20T01:48:12.662 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 2.530249639, 2024-04-20T01:48:12.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068350370000000004, 2024-04-20T01:48:12.662 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:12.662 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:12.662 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:12.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.662 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:12.662 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064387+0000", 2024-04-20T01:48:12.662 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.662 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:12.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064387+0000", 2024-04-20T01:48:12.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:12.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:12.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064386+0000", 2024-04-20T01:48:12.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:12.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.663 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:12.663 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064602+0000", 2024-04-20T01:48:12.663 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00021609999999999999 2024-04-20T01:48:12.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.663 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.664 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:12.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064651+0000", 2024-04-20T01:48:12.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8976e-05 2024-04-20T01:48:12.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.664 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:12.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064664+0000", 2024-04-20T01:48:12.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2632999999999999e-05 2024-04-20T01:48:12.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.664 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:12.664 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064732+0000", 2024-04-20T01:48:12.664 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8047999999999996e-05 2024-04-20T01:48:12.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.664 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:12.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064748+0000", 2024-04-20T01:48:12.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6640999999999999e-05 2024-04-20T01:48:12.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:12.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.071185+0000", 2024-04-20T01:48:12.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0064368109999999997 2024-04-20T01:48:12.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:12.665 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:12.665 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:12.665 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.071222+0000", 2024-04-20T01:48:12.665 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6942e-05 2024-04-20T01:48:12.665 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.665 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.666 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.666 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:12.666 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:12.666 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:12.668 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-20T01:48:12.668 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:48:12.669 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:48:12.669 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:48:12.669 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:48:12.669 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:12.681+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:12.686 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:48:12.697 INFO:tasks.rados.rados.0.smithi165.stdout:284: done (3 left) 2024-04-20T01:48:12.697 INFO:tasks.rados.rados.0.smithi165.stdout:285: snap_create 2024-04-20T01:48:12.697 INFO:tasks.rados.rados.0.smithi165.stdout:281: finishing write tid 1 to smithi16579500-48 2024-04-20T01:48:12.697 INFO:tasks.rados.rados.0.smithi165.stdout:281: finishing write tid 2 to smithi16579500-48 2024-04-20T01:48:12.697 INFO:tasks.rados.rados.0.smithi165.stdout:281: finishing write tid 3 to smithi16579500-48 2024-04-20T01:48:12.697 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 3 v 35 (ObjNum 225 snap 0 seq_num 3856854256) dirty exists 2024-04-20T01:48:12.697 INFO:tasks.rados.rados.0.smithi165.stdout:281: finishing write tid 4 to smithi16579500-48 2024-04-20T01:48:12.698 INFO:tasks.rados.rados.0.smithi165.stdout:281: finishing write tid 5 to smithi16579500-48 2024-04-20T01:48:12.698 INFO:tasks.rados.rados.0.smithi165.stdout:281: finishing write tid 6 to smithi16579500-48 2024-04-20T01:48:12.698 INFO:tasks.rados.rados.0.smithi165.stdout:281: oid 48 updating version 0 to 60 2024-04-20T01:48:12.698 INFO:tasks.rados.rados.0.smithi165.stdout:281: oid 48 updating version 60 to 62 2024-04-20T01:48:12.698 INFO:tasks.rados.rados.0.smithi165.stdout:281: oid 48 updating version 62 to 63 2024-04-20T01:48:12.698 INFO:tasks.rados.rados.0.smithi165.stdout:281: oid 48 updating version 63 to 64 2024-04-20T01:48:12.698 INFO:tasks.rados.rados.0.smithi165.stdout:281: oid 48 version 64 is already newer than 61 2024-04-20T01:48:12.698 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 48 v 64 (ObjNum 129 snap 20 seq_num 129) dirty exists 2024-04-20T01:48:12.698 INFO:tasks.rados.rados.0.smithi165.stdout:281: left oid 48 (ObjNum 129 snap 20 seq_num 129) 2024-04-20T01:48:12.698 INFO:tasks.rados.rados.0.smithi165.stdout:283: finishing write tid 1 to smithi16579500-50 2024-04-20T01:48:12.698 INFO:tasks.rados.rados.0.smithi165.stdout:283: finishing write tid 2 to smithi16579500-50 2024-04-20T01:48:12.698 INFO:tasks.rados.rados.0.smithi165.stdout:283: finishing write tid 3 to smithi16579500-50 2024-04-20T01:48:12.698 INFO:tasks.rados.rados.0.smithi165.stdout:283: finishing write tid 4 to smithi16579500-50 2024-04-20T01:48:12.698 INFO:tasks.rados.rados.0.smithi165.stdout:283: finishing write tid 5 to smithi16579500-50 2024-04-20T01:48:12.698 INFO:tasks.rados.rados.0.smithi165.stdout:283: finishing write tid 6 to smithi16579500-50 2024-04-20T01:48:12.699 INFO:tasks.rados.rados.0.smithi165.stdout:283: oid 50 updating version 0 to 79 2024-04-20T01:48:12.699 INFO:tasks.rados.rados.0.smithi165.stdout:283: oid 50 updating version 79 to 83 2024-04-20T01:48:12.699 INFO:tasks.rados.rados.0.smithi165.stdout:283: oid 50 version 83 is already newer than 82 2024-04-20T01:48:12.699 INFO:tasks.rados.rados.0.smithi165.stdout:283: oid 50 version 83 is already newer than 80 2024-04-20T01:48:12.699 INFO:tasks.rados.rados.0.smithi165.stdout:283: oid 50 version 83 is already newer than 81 2024-04-20T01:48:12.699 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 50 v 83 (ObjNum 130 snap 20 seq_num 130) dirty exists 2024-04-20T01:48:12.699 INFO:tasks.rados.rados.0.smithi165.stdout:283: left oid 50 (ObjNum 130 snap 20 seq_num 130) 2024-04-20T01:48:12.713 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:48:12.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:48:12.728+0000 7f45595e7700 0 -- 172.21.15.44:0/1930062611 >> [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] conn(0x7f455421f030 msgr2=0x7f4554221420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-20T01:48:12.768 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:12.782+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:12.814 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:12.814 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:12.824 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-20T01:48:12.869 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:12.883+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:12.915 INFO:teuthology.orchestra.run.smithi044.stdout:moved 0 / 130 (0%) 2024-04-20T01:48:12.915 INFO:teuthology.orchestra.run.smithi044.stdout:avg 8.125 2024-04-20T01:48:12.915 INFO:teuthology.orchestra.run.smithi044.stdout:stddev 1.86665 -> 1.86665 (expected baseline 2.75993) 2024-04-20T01:48:12.915 INFO:teuthology.orchestra.run.smithi044.stdout:min osd.9 with 5 -> 5 pgs (0.615385 -> 0.615385 * mean) 2024-04-20T01:48:12.915 INFO:teuthology.orchestra.run.smithi044.stdout:max osd.3 with 12 -> 12 pgs (1.47692 -> 1.47692 * mean) 2024-04-20T01:48:12.915 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:48:12.915 INFO:teuthology.orchestra.run.smithi044.stdout:oload 1000 2024-04-20T01:48:12.915 INFO:teuthology.orchestra.run.smithi044.stdout:max_change 3 2024-04-20T01:48:12.915 INFO:teuthology.orchestra.run.smithi044.stdout:max_change_osds 4 2024-04-20T01:48:12.915 INFO:teuthology.orchestra.run.smithi044.stdout:average_utilization 0.0004 2024-04-20T01:48:12.915 INFO:teuthology.orchestra.run.smithi044.stdout:overload_utilization 0.0042 2024-04-20T01:48:12.918 INFO:teuthology.orchestra.run.smithi044.stderr:no change 2024-04-20T01:48:12.937 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:48:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:48:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:48:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:48:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:48:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:48:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:48:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:48:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:48:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:48:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:48:12.939 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:48:12.940 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:48:12.940 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:48:12.940 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:48:12.940 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:48:12.940 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:48:12.970 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:12.984+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:12.994 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:12.994 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:13.004 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-20T01:48:13.089 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:13.102+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:13.172 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:13.187+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:13.173 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:13.173 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:13.183 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-20T01:48:13.273 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:13.288+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:13.351 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:13.351 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:13.362 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-20T01:48:13.406 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:13.420+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:13.475 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:13.488+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:13.488 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:13.489 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:13.499 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-20T01:48:13.576 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:13.591+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:13.666 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:13.666 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:13.676 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-20T01:48:13.678 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:48:13.692+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:48:13.697 INFO:tasks.rados.rados.0.smithi165.stdout:281: done (3 left) 2024-04-20T01:48:13.697 INFO:tasks.rados.rados.0.smithi165.stdout:282: done (2 left) 2024-04-20T01:48:13.697 INFO:tasks.rados.rados.0.smithi165.stdout:283: done (1 left) 2024-04-20T01:48:13.697 INFO:tasks.rados.rados.0.smithi165.stdout:285: done (0 left) 2024-04-20T01:48:13.697 INFO:tasks.rados.rados.0.smithi165.stdout:286: rmattr oid 13 current snap is 21 2024-04-20T01:48:13.697 INFO:tasks.rados.rados.0.smithi165.stdout:286: done (0 left) 2024-04-20T01:48:13.697 INFO:tasks.rados.rados.0.smithi165.stdout:287: read oid 37 snap 18 2024-04-20T01:48:13.697 INFO:tasks.rados.rados.0.smithi165.stdout:287: expect (ObjNum 36 snap 0 seq_num 36) 2024-04-20T01:48:13.698 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 34 current snap is 21 2024-04-20T01:48:13.698 INFO:tasks.rados.rados.0.smithi165.stdout:288: seq_num 131 ranges {3276800=655360,3932160=425984,4358144=638976,4997120=491520} 2024-04-20T01:48:13.718 INFO:tasks.rados.rados.0.smithi165.stdout:288: writing smithi16579500-34 from 3276800 to 3932160 tid 1 2024-04-20T01:48:13.720 INFO:tasks.rados.rados.0.smithi165.stdout:288: writing smithi16579500-34 from 3932160 to 4358144 tid 2 2024-04-20T01:48:13.724 INFO:tasks.rados.rados.0.smithi165.stdout:288: writing smithi16579500-34 from 4358144 to 4997120 tid 3 2024-04-20T01:48:13.727 INFO:tasks.rados.rados.0.smithi165.stdout:288: writing smithi16579500-34 from 4997120 to 5488640 tid 4 2024-04-20T01:48:13.727 INFO:tasks.rados.rados.0.smithi165.stdout:289: setattr oid 8 current snap is 21 2024-04-20T01:48:13.728 INFO:tasks.rados.rados.0.smithi165.stdout:287: expect (ObjNum 36 snap 0 seq_num 36) 2024-04-20T01:48:13.744 INFO:tasks.rados.rados.0.smithi165.stdout:288: finishing write tid 1 to smithi16579500-34 2024-04-20T01:48:13.744 INFO:tasks.rados.rados.0.smithi165.stdout:288: finishing write tid 2 to smithi16579500-34 2024-04-20T01:48:13.744 INFO:tasks.rados.rados.0.smithi165.stdout:288: finishing write tid 3 to smithi16579500-34 2024-04-20T01:48:13.744 INFO:tasks.rados.rados.0.smithi165.stdout:288: finishing write tid 4 to smithi16579500-34 2024-04-20T01:48:13.744 INFO:tasks.rados.rados.0.smithi165.stdout:288: finishing write tid 5 to smithi16579500-34 2024-04-20T01:48:13.745 INFO:tasks.rados.rados.0.smithi165.stdout:288: finishing write tid 6 to smithi16579500-34 2024-04-20T01:48:13.745 INFO:tasks.rados.rados.0.smithi165.stdout:288: oid 34 updating version 0 to 129 2024-04-20T01:48:13.745 INFO:tasks.rados.rados.0.smithi165.stdout:288: oid 34 version 129 is already newer than 125 2024-04-20T01:48:13.745 INFO:tasks.rados.rados.0.smithi165.stdout:288: oid 34 version 129 is already newer than 126 2024-04-20T01:48:13.745 INFO:tasks.rados.rados.0.smithi165.stdout:288: oid 34 version 129 is already newer than 128 2024-04-20T01:48:13.745 INFO:tasks.rados.rados.0.smithi165.stdout:288: oid 34 version 129 is already newer than 127 2024-04-20T01:48:13.745 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 34 v 129 (ObjNum 131 snap 21 seq_num 131) dirty exists 2024-04-20T01:48:13.745 INFO:tasks.rados.rados.0.smithi165.stdout:288: left oid 34 (ObjNum 131 snap 21 seq_num 131) 2024-04-20T01:48:13.745 INFO:tasks.rados.rados.0.smithi165.stdout:287: done (2 left) 2024-04-20T01:48:13.745 INFO:tasks.rados.rados.0.smithi165.stdout:288: done (1 left) 2024-04-20T01:48:13.745 INFO:tasks.rados.rados.0.smithi165.stdout:290: rollback oid 38 current snap is 21 2024-04-20T01:48:13.745 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 38 to 14 2024-04-20T01:48:13.745 INFO:tasks.rados.rados.0.smithi165.stdout:291: rollback oid 21 current snap is 21 2024-04-20T01:48:13.746 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 21 to 20 2024-04-20T01:48:13.746 INFO:tasks.rados.rados.0.smithi165.stdout:292: read oid 41 snap -1 2024-04-20T01:48:13.746 INFO:tasks.rados.rados.0.smithi165.stdout:292: expect (ObjNum 75 snap 7 seq_num 75) 2024-04-20T01:48:13.746 INFO:tasks.rados.rados.0.smithi165.stdout:293: rollback oid 37 current snap is 21 2024-04-20T01:48:13.746 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 37 to 18 2024-04-20T01:48:13.746 INFO:tasks.rados.rados.0.smithi165.stdout:294: read oid 34 snap -1 2024-04-20T01:48:13.746 INFO:tasks.rados.rados.0.smithi165.stdout:294: expect (ObjNum 131 snap 21 seq_num 131) 2024-04-20T01:48:13.746 INFO:tasks.rados.rados.0.smithi165.stdout:295: rollback oid 22 current snap is 21 2024-04-20T01:48:13.746 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 22 to 18 2024-04-20T01:48:13.746 INFO:tasks.rados.rados.0.smithi165.stdout:296: delete oid 43 current snap is 21 2024-04-20T01:48:13.751 INFO:tasks.rados.rados.0.smithi165.stdout:293: finishing rollback tid 0 to smithi16579500-37 2024-04-20T01:48:13.754 INFO:tasks.rados.rados.0.smithi165.stdout:295: finishing rollback tid 0 to smithi16579500-22 2024-04-20T01:48:13.754 INFO:tasks.rados.rados.0.smithi165.stdout:293: finishing rollback tid 1 to smithi16579500-37 2024-04-20T01:48:13.754 INFO:tasks.rados.rados.0.smithi165.stdout:293: finishing rollback tid 2 to smithi16579500-37 2024-04-20T01:48:13.754 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 37 v 70 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-04-20T01:48:13.755 INFO:tasks.rados.rados.0.smithi165.stdout:290: finishing rollback tid 0 to smithi16579500-38 2024-04-20T01:48:13.755 INFO:tasks.rados.rados.0.smithi165.stdout:290: finishing rollback tid 1 to smithi16579500-38 2024-04-20T01:48:13.755 INFO:tasks.rados.rados.0.smithi165.stdout:295: finishing rollback tid 1 to smithi16579500-22 2024-04-20T01:48:13.755 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 8 v 33 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-04-20T01:48:13.756 INFO:tasks.rados.rados.0.smithi165.stdout:295: finishing rollback tid 2 to smithi16579500-22 2024-04-20T01:48:13.756 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 22 v 24 (ObjNum 105 snap 16 seq_num 105) dirty exists 2024-04-20T01:48:13.756 INFO:tasks.rados.rados.0.smithi165.stdout:291: finishing rollback tid 0 to smithi16579500-21 2024-04-20T01:48:13.757 INFO:tasks.rados.rados.0.smithi165.stdout:290: finishing rollback tid 2 to smithi16579500-38 2024-04-20T01:48:13.757 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 38 v 117 (ObjNum 83 snap 8 seq_num 83) dirty exists 2024-04-20T01:48:13.760 INFO:tasks.rados.rados.0.smithi165.stdout:289: done (7 left) 2024-04-20T01:48:13.761 INFO:tasks.rados.rados.0.smithi165.stdout:290: done (6 left) 2024-04-20T01:48:13.761 INFO:tasks.rados.rados.0.smithi165.stdout:293: done (5 left) 2024-04-20T01:48:13.761 INFO:tasks.rados.rados.0.smithi165.stdout:295: done (4 left) 2024-04-20T01:48:13.761 INFO:tasks.rados.rados.0.smithi165.stdout:296: done (3 left) 2024-04-20T01:48:13.761 INFO:tasks.rados.rados.0.smithi165.stdout:297: snap_remove snap 18 2024-04-20T01:48:13.778 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:13.792+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:13.800 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-20T01:48:13.815 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:13.815 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:13.826 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-20T01:48:13.879 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:13.893+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:13.954 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:13.954 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:13.964 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-20T01:48:14.006 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:14.020+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:14.082 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:48:14.096+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:48:14.132 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:14.133 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:14.143 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-20T01:48:14.182 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:14.196+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:14.272 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:14.272 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:14.282 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-20T01:48:14.283 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:48:14.297+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:48:14.384 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:14.398+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:14.409 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:14.409 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:14.419 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-20T01:48:14.485 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:14.499+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:14.586 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:48:14.600+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:48:14.587 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:14.587 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:14.597 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-20T01:48:14.686 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:14.701+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:14.698 INFO:tasks.rados.rados.0.smithi165.stdout:297: done (3 left) 2024-04-20T01:48:14.698 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 18 current snap is 21 2024-04-20T01:48:14.703 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is unset 2024-04-20T01:48:14.705 INFO:tasks.rados.rados.0.smithi165.stdout:298: seq_num 133 ranges {1458176=688128,2146304=720896,2867200=475136,3342336=360448} 2024-04-20T01:48:14.710 INFO:tasks.rados.rados.0.smithi165.stdout:298: writing smithi16579500-18 from 1458176 to 2146304 tid 1 2024-04-20T01:48:14.720 INFO:tasks.rados.rados.0.smithi165.stdout:298: writing smithi16579500-18 from 2146304 to 2867200 tid 2 2024-04-20T01:48:14.720 INFO:tasks.rados.rados.0.smithi165.stdout:298: writing smithi16579500-18 from 2867200 to 3342336 tid 3 2024-04-20T01:48:14.721 INFO:tasks.rados.rados.0.smithi165.stdout:298: writing smithi16579500-18 from 3342336 to 3702784 tid 4 2024-04-20T01:48:14.721 INFO:tasks.rados.rados.0.smithi165.stdout:299: snap_create 2024-04-20T01:48:14.721 INFO:tasks.rados.rados.0.smithi165.stdout:291: finishing rollback tid 1 to smithi16579500-21 2024-04-20T01:48:14.721 INFO:tasks.rados.rados.0.smithi165.stdout:291: finishing rollback tid 2 to smithi16579500-21 2024-04-20T01:48:14.721 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 21 v 35 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-04-20T01:48:14.721 INFO:tasks.rados.rados.0.smithi165.stdout:292: expect (ObjNum 75 snap 7 seq_num 75) 2024-04-20T01:48:14.747 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:14.747 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:14.757 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-20T01:48:14.782 INFO:tasks.rados.rados.0.smithi165.stdout:294: expect (ObjNum 131 snap 21 seq_num 131) 2024-04-20T01:48:14.787 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:14.801+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:14.885 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:14.885 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:14.888 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:14.902+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:14.895 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-20T01:48:14.951 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-20T01:48:14.986 INFO:tasks.rados.rados.0.smithi165.stdout:298: finishing write tid 1 to smithi16579500-18 2024-04-20T01:48:14.986 INFO:tasks.rados.rados.0.smithi165.stdout:298: finishing write tid 2 to smithi16579500-18 2024-04-20T01:48:14.986 INFO:tasks.rados.rados.0.smithi165.stdout:298: finishing write tid 3 to smithi16579500-18 2024-04-20T01:48:14.986 INFO:tasks.rados.rados.0.smithi165.stdout:298: finishing write tid 4 to smithi16579500-18 2024-04-20T01:48:14.987 INFO:tasks.rados.rados.0.smithi165.stdout:298: finishing write tid 5 to smithi16579500-18 2024-04-20T01:48:14.987 INFO:tasks.rados.rados.0.smithi165.stdout:298: finishing write tid 6 to smithi16579500-18 2024-04-20T01:48:14.987 INFO:tasks.rados.rados.0.smithi165.stdout:298: oid 18 updating version 0 to 44 2024-04-20T01:48:14.987 INFO:tasks.rados.rados.0.smithi165.stdout:298: oid 18 updating version 44 to 45 2024-04-20T01:48:14.987 INFO:tasks.rados.rados.0.smithi165.stdout:298: oid 18 version 45 is already newer than 43 2024-04-20T01:48:14.987 INFO:tasks.rados.rados.0.smithi165.stdout:298: oid 18 updating version 45 to 46 2024-04-20T01:48:14.987 INFO:tasks.rados.rados.0.smithi165.stdout:298: oid 18 version 46 is already newer than 42 2024-04-20T01:48:14.987 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 18 v 46 (ObjNum 133 snap 21 seq_num 133) dirty exists 2024-04-20T01:48:14.987 INFO:tasks.rados.rados.0.smithi165.stdout:298: left oid 18 (ObjNum 133 snap 21 seq_num 133) 2024-04-20T01:48:14.989 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:15.003+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:15.062 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:15.062 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:15.072 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-20T01:48:15.090 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:15.104+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:15.124 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:48:15.146 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:48:15.167 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:48:15.189 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:48:15.191 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:15.205+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:15.201 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:15.201 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:15.209 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:48:15.211 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-20T01:48:15.229 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:48:15.250 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:48:15.272 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:48:15.291 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:48:15.292 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:48:15.305+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:48:15.312 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:48:15.333 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:48:15.339 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:15.339 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:15.339 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 30, 2024-04-20T01:48:15.339 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-20T01:48:15.339 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:15.350 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-20T01:48:15.353 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:48:15.374 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:48:15.392 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:15.405+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:15.395 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:48:15.416 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:48:15.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:48:15.430+0000 7f6632d9d700 0 -- 172.21.15.44:0/1570915962 >> [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] conn(0x7f663421f5d0 msgr2=0x7f66342219c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-20T01:48:15.493 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:15.507+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:15.520 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:15.520 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-20T01:48:15.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-20T01:48:15.520 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-20T01:48:15.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.521 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:243 3.ds0 3:b226b74f:::smithi16579500-7:head [delete] snapc 5=[5,2] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:15.521 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.787656+0000", 2024-04-20T01:48:15.521 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 36.746424458999996, 2024-04-20T01:48:15.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013205184, 2024-04-20T01:48:15.521 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:15.521 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:15.521 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:15.521 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:15.521 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:15.521 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 243 2024-04-20T01:48:15.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.521 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:15.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:15.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787656+0000", 2024-04-20T01:48:15.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:15.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787656+0000", 2024-04-20T01:48:15.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:15.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787654+0000", 2024-04-20T01:48:15.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:15.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:15.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787662+0000", 2024-04-20T01:48:15.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6599999999999995e-06 2024-04-20T01:48:15.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:15.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787680+0000", 2024-04-20T01:48:15.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7672999999999998e-05 2024-04-20T01:48:15.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:15.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787688+0000", 2024-04-20T01:48:15.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.405e-06 2024-04-20T01:48:15.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:15.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787739+0000", 2024-04-20T01:48:15.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0652999999999998e-05 2024-04-20T01:48:15.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:15.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787851+0000", 2024-04-20T01:48:15.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112172 2024-04-20T01:48:15.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:15.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788643+0000", 2024-04-20T01:48:15.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00079174300000000003 2024-04-20T01:48:15.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:15.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.789564+0000", 2024-04-20T01:48:15.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00092135199999999996 2024-04-20T01:48:15.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:15.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.800781+0000", 2024-04-20T01:48:15.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011217187999999999 2024-04-20T01:48:15.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:15.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.800861+0000", 2024-04-20T01:48:15.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9796999999999995e-05 2024-04-20T01:48:15.526 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.526 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.526 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.526 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 48/30 ECSubWrite(tid=253, reqid=osd.13.0:253, at_version=48'44, trim_to=0'0, roll_forward_to=47'40))", 2024-04-20T01:48:15.526 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.711468+0000", 2024-04-20T01:48:15.526 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 25.822612729999999, 2024-04-20T01:48:15.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056178349999999998, 2024-04-20T01:48:15.526 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:15.526 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:15.526 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:15.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:15.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711468+0000", 2024-04-20T01:48:15.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:15.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711468+0000", 2024-04-20T01:48:15.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:15.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711467+0000", 2024-04-20T01:48:15.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:15.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:15.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711491+0000", 2024-04-20T01:48:15.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4023e-05 2024-04-20T01:48:15.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:15.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711503+0000", 2024-04-20T01:48:15.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2065000000000001e-05 2024-04-20T01:48:15.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:15.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711513+0000", 2024-04-20T01:48:15.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8509999999999994e-06 2024-04-20T01:48:15.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:15.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711561+0000", 2024-04-20T01:48:15.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8208999999999999e-05 2024-04-20T01:48:15.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:15.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711573+0000", 2024-04-20T01:48:15.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1955e-05 2024-04-20T01:48:15.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:15.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717067+0000", 2024-04-20T01:48:15.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054938909999999999 2024-04-20T01:48:15.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:15.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717085+0000", 2024-04-20T01:48:15.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8660000000000001e-05 2024-04-20T01:48:15.530 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.530 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.530 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.530 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.1s4 48/30 ECSubWrite(tid=119, reqid=osd.4.0:119, at_version=48'14, trim_to=0'0, roll_forward_to=47'12))", 2024-04-20T01:48:15.530 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.717339+0000", 2024-04-20T01:48:15.531 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 25.816741772, 2024-04-20T01:48:15.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0079470240000000004, 2024-04-20T01:48:15.531 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:15.531 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:15.531 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:15.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:15.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717339+0000", 2024-04-20T01:48:15.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:15.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717339+0000", 2024-04-20T01:48:15.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:15.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717338+0000", 2024-04-20T01:48:15.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:15.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:15.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717357+0000", 2024-04-20T01:48:15.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8575000000000001e-05 2024-04-20T01:48:15.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:15.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717368+0000", 2024-04-20T01:48:15.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.145e-05 2024-04-20T01:48:15.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:15.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717377+0000", 2024-04-20T01:48:15.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4929999999999998e-06 2024-04-20T01:48:15.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:15.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717424+0000", 2024-04-20T01:48:15.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7240000000000002e-05 2024-04-20T01:48:15.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:15.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717435+0000", 2024-04-20T01:48:15.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0533000000000001e-05 2024-04-20T01:48:15.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:15.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.725244+0000", 2024-04-20T01:48:15.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0078092129999999997 2024-04-20T01:48:15.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:15.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.725286+0000", 2024-04-20T01:48:15.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1792999999999998e-05 2024-04-20T01:48:15.534 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.534 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.534 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.535 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.1s4 60/30 ECSubWrite(tid=135, reqid=osd.4.0:135, at_version=60'18, trim_to=0'0, roll_forward_to=58'16))", 2024-04-20T01:48:15.535 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:01.742576+0000", 2024-04-20T01:48:15.535 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.791504605, 2024-04-20T01:48:15.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057386049999999999, 2024-04-20T01:48:15.535 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:15.535 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:15.535 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:15.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:15.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.742576+0000", 2024-04-20T01:48:15.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:15.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.742576+0000", 2024-04-20T01:48:15.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:15.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.742575+0000", 2024-04-20T01:48:15.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:15.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:15.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.742610+0000", 2024-04-20T01:48:15.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4511999999999998e-05 2024-04-20T01:48:15.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:15.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.742625+0000", 2024-04-20T01:48:15.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5362999999999998e-05 2024-04-20T01:48:15.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:15.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.742642+0000", 2024-04-20T01:48:15.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7008e-05 2024-04-20T01:48:15.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:15.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.742709+0000", 2024-04-20T01:48:15.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7167000000000003e-05 2024-04-20T01:48:15.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:15.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.742725+0000", 2024-04-20T01:48:15.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5403000000000001e-05 2024-04-20T01:48:15.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:15.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.748283+0000", 2024-04-20T01:48:15.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055577760000000004 2024-04-20T01:48:15.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:15.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.748314+0000", 2024-04-20T01:48:15.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1736999999999998e-05 2024-04-20T01:48:15.538 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.539 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.539 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.539 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.1s4 60/30 ECSubWrite(tid=139, reqid=osd.4.0:135, at_version=0'0, trim_to=0'0, roll_forward_to=60'18))", 2024-04-20T01:48:15.539 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:01.748778+0000", 2024-04-20T01:48:15.539 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.785302669, 2024-04-20T01:48:15.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005700887, 2024-04-20T01:48:15.539 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:15.539 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:15.539 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:15.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:15.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.748778+0000", 2024-04-20T01:48:15.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:15.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.748778+0000", 2024-04-20T01:48:15.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:15.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.748777+0000", 2024-04-20T01:48:15.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:15.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:15.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.748791+0000", 2024-04-20T01:48:15.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3801999999999999e-05 2024-04-20T01:48:15.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:15.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.748805+0000", 2024-04-20T01:48:15.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4783000000000001e-05 2024-04-20T01:48:15.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:15.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.748816+0000", 2024-04-20T01:48:15.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0375e-05 2024-04-20T01:48:15.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:15.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.748888+0000", 2024-04-20T01:48:15.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2706999999999994e-05 2024-04-20T01:48:15.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:15.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.748902+0000", 2024-04-20T01:48:15.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4005999999999999e-05 2024-04-20T01:48:15.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:15.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.754440+0000", 2024-04-20T01:48:15.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005537536 2024-04-20T01:48:15.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:15.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.754479+0000", 2024-04-20T01:48:15.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8674000000000003e-05 2024-04-20T01:48:15.543 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.543 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.543 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.543 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 60/30 ECSubWrite(tid=277, reqid=osd.13.0:277, at_version=60'63, trim_to=0'0, roll_forward_to=58'61))", 2024-04-20T01:48:15.543 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:01.750833+0000", 2024-04-20T01:48:15.543 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.783247093, 2024-04-20T01:48:15.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006209689, 2024-04-20T01:48:15.543 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:15.543 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:15.543 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:15.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:15.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.750833+0000", 2024-04-20T01:48:15.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:15.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.750833+0000", 2024-04-20T01:48:15.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:15.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.750833+0000", 2024-04-20T01:48:15.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:48:15.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:15.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.750856+0000", 2024-04-20T01:48:15.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2988000000000001e-05 2024-04-20T01:48:15.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:15.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.750869+0000", 2024-04-20T01:48:15.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3268000000000001e-05 2024-04-20T01:48:15.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:15.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.750879+0000", 2024-04-20T01:48:15.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6180000000000001e-06 2024-04-20T01:48:15.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:15.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.750936+0000", 2024-04-20T01:48:15.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6592000000000003e-05 2024-04-20T01:48:15.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:15.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.750949+0000", 2024-04-20T01:48:15.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3067999999999999e-05 2024-04-20T01:48:15.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:15.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.757014+0000", 2024-04-20T01:48:15.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060654619999999998 2024-04-20T01:48:15.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:15.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.757043+0000", 2024-04-20T01:48:15.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8889000000000001e-05 2024-04-20T01:48:15.547 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.547 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.547 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.547 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 60/30 ECSubWrite(tid=280, reqid=client.4643.0:519, at_version=60'66, trim_to=0'0, roll_forward_to=60'63))", 2024-04-20T01:48:15.547 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:01.837460+0000", 2024-04-20T01:48:15.547 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.696620567, 2024-04-20T01:48:15.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071325599999999996, 2024-04-20T01:48:15.547 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:15.547 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:15.547 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:15.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:15.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837460+0000", 2024-04-20T01:48:15.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:15.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837460+0000", 2024-04-20T01:48:15.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:15.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837459+0000", 2024-04-20T01:48:15.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:15.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:15.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837876+0000", 2024-04-20T01:48:15.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00041722999999999999 2024-04-20T01:48:15.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:15.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837893+0000", 2024-04-20T01:48:15.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6803000000000002e-05 2024-04-20T01:48:15.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:15.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837903+0000", 2024-04-20T01:48:15.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6830000000000003e-06 2024-04-20T01:48:15.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:15.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837955+0000", 2024-04-20T01:48:15.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2160000000000002e-05 2024-04-20T01:48:15.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:15.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837967+0000", 2024-04-20T01:48:15.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2057e-05 2024-04-20T01:48:15.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:15.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.844571+0000", 2024-04-20T01:48:15.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066042319999999998 2024-04-20T01:48:15.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:15.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.844592+0000", 2024-04-20T01:48:15.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1268999999999999e-05 2024-04-20T01:48:15.551 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.551 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.551 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.551 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 63/30 ECSubWrite(tid=295, reqid=client.4643.0:574, at_version=63'79, trim_to=0'0, roll_forward_to=62'77))", 2024-04-20T01:48:15.551 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:04.795046+0000", 2024-04-20T01:48:15.551 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.739034444, 2024-04-20T01:48:15.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076412559999999999, 2024-04-20T01:48:15.551 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:15.552 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:15.552 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:15.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:15.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795046+0000", 2024-04-20T01:48:15.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:15.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795046+0000", 2024-04-20T01:48:15.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:15.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795044+0000", 2024-04-20T01:48:15.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:15.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:15.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795601+0000", 2024-04-20T01:48:15.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00055720399999999995 2024-04-20T01:48:15.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:15.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795617+0000", 2024-04-20T01:48:15.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5341000000000001e-05 2024-04-20T01:48:15.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:15.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795627+0000", 2024-04-20T01:48:15.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0429e-05 2024-04-20T01:48:15.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:15.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795692+0000", 2024-04-20T01:48:15.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4712e-05 2024-04-20T01:48:15.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:15.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795704+0000", 2024-04-20T01:48:15.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.258e-05 2024-04-20T01:48:15.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:15.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.802650+0000", 2024-04-20T01:48:15.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069452869999999996 2024-04-20T01:48:15.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:15.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.802687+0000", 2024-04-20T01:48:15.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7483e-05 2024-04-20T01:48:15.555 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.555 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.555 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.555 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 63/30 ECSubWrite(tid=296, reqid=client.4643.0:575, at_version=63'80, trim_to=0'0, roll_forward_to=62'77))", 2024-04-20T01:48:15.555 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:04.798415+0000", 2024-04-20T01:48:15.555 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.735665644999999, 2024-04-20T01:48:15.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071008770000000002, 2024-04-20T01:48:15.556 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:15.556 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:15.556 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:15.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:15.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798415+0000", 2024-04-20T01:48:15.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:15.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798415+0000", 2024-04-20T01:48:15.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:15.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798414+0000", 2024-04-20T01:48:15.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:15.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:15.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798883+0000", 2024-04-20T01:48:15.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00046914899999999998 2024-04-20T01:48:15.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:15.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798898+0000", 2024-04-20T01:48:15.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4534e-05 2024-04-20T01:48:15.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:15.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798905+0000", 2024-04-20T01:48:15.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9149999999999996e-06 2024-04-20T01:48:15.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:15.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798951+0000", 2024-04-20T01:48:15.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5776999999999999e-05 2024-04-20T01:48:15.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:15.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798961+0000", 2024-04-20T01:48:15.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9289999999999993e-06 2024-04-20T01:48:15.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:15.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.805490+0000", 2024-04-20T01:48:15.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065291029999999996 2024-04-20T01:48:15.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:15.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.805516+0000", 2024-04-20T01:48:15.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5347999999999999e-05 2024-04-20T01:48:15.559 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.559 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.559 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.559 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 63/30 ECSubWrite(tid=297, reqid=client.4643.0:576, at_version=63'81, trim_to=0'0, roll_forward_to=62'77))", 2024-04-20T01:48:15.559 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:04.801003+0000", 2024-04-20T01:48:15.559 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.733077679999999, 2024-04-20T01:48:15.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053881149999999997, 2024-04-20T01:48:15.560 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:15.560 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:15.560 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:15.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:15.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801003+0000", 2024-04-20T01:48:15.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:15.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801003+0000", 2024-04-20T01:48:15.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:15.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801002+0000", 2024-04-20T01:48:15.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:15.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:15.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801322+0000", 2024-04-20T01:48:15.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032022799999999998 2024-04-20T01:48:15.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:15.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801336+0000", 2024-04-20T01:48:15.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3978999999999999e-05 2024-04-20T01:48:15.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:15.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801344+0000", 2024-04-20T01:48:15.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9300000000000003e-06 2024-04-20T01:48:15.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:15.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801397+0000", 2024-04-20T01:48:15.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3279000000000003e-05 2024-04-20T01:48:15.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:15.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801409+0000", 2024-04-20T01:48:15.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.166e-05 2024-04-20T01:48:15.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:15.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.806359+0000", 2024-04-20T01:48:15.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049499330000000001 2024-04-20T01:48:15.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:15.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.806391+0000", 2024-04-20T01:48:15.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2051000000000002e-05 2024-04-20T01:48:15.563 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.563 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.563 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.563 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.cs2 68/30 ECSubWrite(tid=68, reqid=client.4643.0:614, at_version=68'59, trim_to=0'0, roll_forward_to=68'56))", 2024-04-20T01:48:15.564 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:09.812830+0000", 2024-04-20T01:48:15.564 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.7212504879999999, 2024-04-20T01:48:15.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066006179999999999, 2024-04-20T01:48:15.564 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:15.564 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:15.564 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:15.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:15.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.812830+0000", 2024-04-20T01:48:15.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:15.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.812830+0000", 2024-04-20T01:48:15.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:15.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.812829+0000", 2024-04-20T01:48:15.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:15.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:15.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.812899+0000", 2024-04-20T01:48:15.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9803999999999998e-05 2024-04-20T01:48:15.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:15.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.812912+0000", 2024-04-20T01:48:15.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3552e-05 2024-04-20T01:48:15.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:15.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.812921+0000", 2024-04-20T01:48:15.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7469999999999994e-06 2024-04-20T01:48:15.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:15.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.813009+0000", 2024-04-20T01:48:15.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8298000000000001e-05 2024-04-20T01:48:15.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:15.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.813023+0000", 2024-04-20T01:48:15.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3701e-05 2024-04-20T01:48:15.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:15.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.819402+0000", 2024-04-20T01:48:15.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063790799999999996 2024-04-20T01:48:15.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:15.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.819431+0000", 2024-04-20T01:48:15.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8513999999999998e-05 2024-04-20T01:48:15.567 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.567 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.567 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.568 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 68/30 ECSubWrite(tid=315, reqid=osd.13.0:313, at_version=0'0, trim_to=0'0, roll_forward_to=68'95))", 2024-04-20T01:48:15.568 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:09.813349+0000", 2024-04-20T01:48:15.568 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.7207308780000004, 2024-04-20T01:48:15.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060929799999999996, 2024-04-20T01:48:15.568 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:15.568 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:15.568 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:15.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:15.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.813349+0000", 2024-04-20T01:48:15.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:15.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.813349+0000", 2024-04-20T01:48:15.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:15.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.813349+0000", 2024-04-20T01:48:15.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:15.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:15.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.813359+0000", 2024-04-20T01:48:15.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8689999999999999e-06 2024-04-20T01:48:15.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:15.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.813385+0000", 2024-04-20T01:48:15.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5673e-05 2024-04-20T01:48:15.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:15.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.813393+0000", 2024-04-20T01:48:15.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8859999999999993e-06 2024-04-20T01:48:15.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:15.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.813642+0000", 2024-04-20T01:48:15.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00024854800000000002 2024-04-20T01:48:15.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:15.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.813652+0000", 2024-04-20T01:48:15.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0152e-05 2024-04-20T01:48:15.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:15.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.819419+0000", 2024-04-20T01:48:15.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057668950000000002 2024-04-20T01:48:15.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:15.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.819442+0000", 2024-04-20T01:48:15.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3441000000000001e-05 2024-04-20T01:48:15.571 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.572 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.572 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.572 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.6s4 68/30 ECSubWrite(tid=15, reqid=client.4643.0:619, at_version=68'9, trim_to=0'0, roll_forward_to=63'7))", 2024-04-20T01:48:15.572 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:09.813671+0000", 2024-04-20T01:48:15.572 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.7204091449999996, 2024-04-20T01:48:15.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058052659999999999, 2024-04-20T01:48:15.572 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:15.572 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:15.572 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:15.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:15.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.813671+0000", 2024-04-20T01:48:15.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:15.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.813671+0000", 2024-04-20T01:48:15.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:15.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.813670+0000", 2024-04-20T01:48:15.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:15.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:15.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.813694+0000", 2024-04-20T01:48:15.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3986999999999999e-05 2024-04-20T01:48:15.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:15.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.813708+0000", 2024-04-20T01:48:15.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4014e-05 2024-04-20T01:48:15.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:15.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.813719+0000", 2024-04-20T01:48:15.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1259e-05 2024-04-20T01:48:15.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:15.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.814123+0000", 2024-04-20T01:48:15.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00040386600000000001 2024-04-20T01:48:15.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:15.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.814133+0000", 2024-04-20T01:48:15.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0019e-05 2024-04-20T01:48:15.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:15.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.819461+0000", 2024-04-20T01:48:15.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053272770000000001 2024-04-20T01:48:15.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:15.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.819476+0000", 2024-04-20T01:48:15.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5894000000000001e-05 2024-04-20T01:48:15.576 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.576 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.576 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.576 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.cs2 68/30 ECSubWrite(tid=69, reqid=client.4643.0:615, at_version=68'60, trim_to=0'0, roll_forward_to=68'56))", 2024-04-20T01:48:15.576 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:09.814877+0000", 2024-04-20T01:48:15.576 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.7192032460000002, 2024-04-20T01:48:15.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076095620000000003, 2024-04-20T01:48:15.576 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:15.576 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:15.576 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:15.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:15.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.814877+0000", 2024-04-20T01:48:15.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:15.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.814877+0000", 2024-04-20T01:48:15.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:15.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.814876+0000", 2024-04-20T01:48:15.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:15.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:15.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.814983+0000", 2024-04-20T01:48:15.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010743699999999999 2024-04-20T01:48:15.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:15.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.814996+0000", 2024-04-20T01:48:15.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2884000000000001e-05 2024-04-20T01:48:15.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:15.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815005+0000", 2024-04-20T01:48:15.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5369999999999997e-06 2024-04-20T01:48:15.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:15.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815050+0000", 2024-04-20T01:48:15.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5364999999999997e-05 2024-04-20T01:48:15.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:15.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815059+0000", 2024-04-20T01:48:15.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9530000000000005e-06 2024-04-20T01:48:15.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:15.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.822460+0000", 2024-04-20T01:48:15.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0074009109999999996 2024-04-20T01:48:15.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:15.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.822487+0000", 2024-04-20T01:48:15.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6939000000000001e-05 2024-04-20T01:48:15.580 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.580 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.580 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.580 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.cs2 68/30 ECSubWrite(tid=70, reqid=client.4643.0:616, at_version=68'61, trim_to=0'0, roll_forward_to=68'56))", 2024-04-20T01:48:15.580 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:09.815775+0000", 2024-04-20T01:48:15.580 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.7183056739999998, 2024-04-20T01:48:15.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0067272979999999996, 2024-04-20T01:48:15.580 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:15.580 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:15.581 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:15.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:15.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815775+0000", 2024-04-20T01:48:15.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:15.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815775+0000", 2024-04-20T01:48:15.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:15.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815774+0000", 2024-04-20T01:48:15.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:15.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:15.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815794+0000", 2024-04-20T01:48:15.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9891999999999999e-05 2024-04-20T01:48:15.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:15.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815808+0000", 2024-04-20T01:48:15.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4584e-05 2024-04-20T01:48:15.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:15.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815819+0000", 2024-04-20T01:48:15.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0216e-05 2024-04-20T01:48:15.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:15.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.816802+0000", 2024-04-20T01:48:15.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0009835480000000001 2024-04-20T01:48:15.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:15.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.816813+0000", 2024-04-20T01:48:15.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1156e-05 2024-04-20T01:48:15.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:15.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.822491+0000", 2024-04-20T01:48:15.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056774030000000001 2024-04-20T01:48:15.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:15.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.822502+0000", 2024-04-20T01:48:15.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1297999999999999e-05 2024-04-20T01:48:15.584 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.584 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.584 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.584 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 68/30 ECSubWrite(tid=318, reqid=client.4643.0:624, at_version=68'98, trim_to=0'0, roll_forward_to=68'95))", 2024-04-20T01:48:15.584 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:09.823900+0000", 2024-04-20T01:48:15.584 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.710180845, 2024-04-20T01:48:15.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064862720000000004, 2024-04-20T01:48:15.585 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:15.585 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:15.585 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:15.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:15.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.823900+0000", 2024-04-20T01:48:15.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:15.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.823900+0000", 2024-04-20T01:48:15.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:15.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.823899+0000", 2024-04-20T01:48:15.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:15.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:15.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.824144+0000", 2024-04-20T01:48:15.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00024477 2024-04-20T01:48:15.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:15.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.824155+0000", 2024-04-20T01:48:15.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1464000000000001e-05 2024-04-20T01:48:15.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:15.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.824169+0000", 2024-04-20T01:48:15.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3712e-05 2024-04-20T01:48:15.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:15.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.824216+0000", 2024-04-20T01:48:15.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7289000000000001e-05 2024-04-20T01:48:15.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:15.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.824227+0000", 2024-04-20T01:48:15.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1127999999999999e-05 2024-04-20T01:48:15.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:15.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.830358+0000", 2024-04-20T01:48:15.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061307130000000003 2024-04-20T01:48:15.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:15.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.830386+0000", 2024-04-20T01:48:15.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7807e-05 2024-04-20T01:48:15.588 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.588 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.588 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.589 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 68/30 ECSubWrite(tid=319, reqid=client.4643.0:625, at_version=68'99, trim_to=0'0, roll_forward_to=68'95))", 2024-04-20T01:48:15.589 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:09.825402+0000", 2024-04-20T01:48:15.589 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.7086787770000003, 2024-04-20T01:48:15.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059947819999999997, 2024-04-20T01:48:15.589 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:15.589 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:15.589 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:15.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:15.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.825402+0000", 2024-04-20T01:48:15.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:15.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.825402+0000", 2024-04-20T01:48:15.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:15.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.825401+0000", 2024-04-20T01:48:15.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:15.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:15.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.825520+0000", 2024-04-20T01:48:15.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000119108 2024-04-20T01:48:15.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:15.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.825533+0000", 2024-04-20T01:48:15.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3434e-05 2024-04-20T01:48:15.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:15.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.825581+0000", 2024-04-20T01:48:15.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7243999999999997e-05 2024-04-20T01:48:15.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:15.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.825633+0000", 2024-04-20T01:48:15.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2587000000000003e-05 2024-04-20T01:48:15.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:15.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.825644+0000", 2024-04-20T01:48:15.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0876000000000001e-05 2024-04-20T01:48:15.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:15.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.831373+0000", 2024-04-20T01:48:15.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057284509999999999 2024-04-20T01:48:15.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:15.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.831396+0000", 2024-04-20T01:48:15.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3832000000000002e-05 2024-04-20T01:48:15.592 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.592 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.592 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.593 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.cs2 68/30 ECSubWrite(tid=71, reqid=client.4643.0:616, at_version=0'0, trim_to=0'0, roll_forward_to=68'61))", 2024-04-20T01:48:15.593 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:09.825641+0000", 2024-04-20T01:48:15.593 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.708439501, 2024-04-20T01:48:15.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057483329999999996, 2024-04-20T01:48:15.593 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:15.593 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:15.593 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:15.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:15.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.825641+0000", 2024-04-20T01:48:15.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:15.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.825641+0000", 2024-04-20T01:48:15.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:15.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.825640+0000", 2024-04-20T01:48:15.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:15.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:15.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.825652+0000", 2024-04-20T01:48:15.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1898e-05 2024-04-20T01:48:15.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:15.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.825663+0000", 2024-04-20T01:48:15.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0882e-05 2024-04-20T01:48:15.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:15.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.825684+0000", 2024-04-20T01:48:15.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0928999999999999e-05 2024-04-20T01:48:15.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:15.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.825731+0000", 2024-04-20T01:48:15.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7113000000000001e-05 2024-04-20T01:48:15.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:15.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.825741+0000", 2024-04-20T01:48:15.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0176999999999999e-05 2024-04-20T01:48:15.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:15.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.831358+0000", 2024-04-20T01:48:15.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056165260000000002 2024-04-20T01:48:15.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:15.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.831389+0000", 2024-04-20T01:48:15.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1532000000000002e-05 2024-04-20T01:48:15.596 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.596 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.597 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.597 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 68/30 ECSubWrite(tid=339, reqid=client.4643.0:711, at_version=68'111, trim_to=0'0, roll_forward_to=68'109))", 2024-04-20T01:48:15.597 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.112145+0000", 2024-04-20T01:48:15.597 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.4219358509999998, 2024-04-20T01:48:15.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054064270000000001, 2024-04-20T01:48:15.597 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:15.597 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:15.597 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:15.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:15.597 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.112145+0000", 2024-04-20T01:48:15.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:15.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.112145+0000", 2024-04-20T01:48:15.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:15.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.112144+0000", 2024-04-20T01:48:15.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:15.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:15.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.112601+0000", 2024-04-20T01:48:15.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045654799999999998 2024-04-20T01:48:15.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:15.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.112611+0000", 2024-04-20T01:48:15.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9410000000000002e-06 2024-04-20T01:48:15.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:15.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.112618+0000", 2024-04-20T01:48:15.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6680000000000001e-06 2024-04-20T01:48:15.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:15.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.112663+0000", 2024-04-20T01:48:15.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4975999999999998e-05 2024-04-20T01:48:15.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:15.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.112674+0000", 2024-04-20T01:48:15.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0818999999999999e-05 2024-04-20T01:48:15.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:15.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.117513+0000", 2024-04-20T01:48:15.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048388060000000002 2024-04-20T01:48:15.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:15.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.117551+0000", 2024-04-20T01:48:15.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8124999999999998e-05 2024-04-20T01:48:15.600 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.601 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.601 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.601 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.cs2 70/30 ECSubWrite(tid=103, reqid=client.4643.0:738, at_version=70'83, trim_to=0'0, roll_forward_to=70'80))", 2024-04-20T01:48:15.601 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:11.861902+0000", 2024-04-20T01:48:15.601 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 3.6721785869999999, 2024-04-20T01:48:15.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0073928659999999997, 2024-04-20T01:48:15.601 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:15.601 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:15.601 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:15.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:15.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861902+0000", 2024-04-20T01:48:15.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:15.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861902+0000", 2024-04-20T01:48:15.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:15.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:15.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861901+0000", 2024-04-20T01:48:15.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:15.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:15.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861922+0000", 2024-04-20T01:48:15.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0917e-05 2024-04-20T01:48:15.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:15.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861935+0000", 2024-04-20T01:48:15.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3237e-05 2024-04-20T01:48:15.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:15.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861945+0000", 2024-04-20T01:48:15.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0326000000000001e-05 2024-04-20T01:48:15.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:15.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.862010+0000", 2024-04-20T01:48:15.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4083000000000003e-05 2024-04-20T01:48:15.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:15.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.862019+0000", 2024-04-20T01:48:15.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0659999999999993e-06 2024-04-20T01:48:15.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:15.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.869223+0000", 2024-04-20T01:48:15.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007204612 2024-04-20T01:48:15.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:15.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:15.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:15.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.869295+0000", 2024-04-20T01:48:15.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1421999999999995e-05 2024-04-20T01:48:15.604 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.605 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.605 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.605 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:15.605 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:15.605 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:15.607 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-20T01:48:15.607 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:15.620+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:15.639 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:48:15.640 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:48:15.641 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:48:15.641 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:48:15.641 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:48:15.641 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:48:15.641 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:48:15.641 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:48:15.641 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:48:15.641 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:48:15.641 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:48:15.641 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:48:15.641 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:48:15.641 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:48:15.641 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:48:15.642 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:48:15.642 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:48:15.699 INFO:tasks.rados.rados.0.smithi165.stdout:291: done (4 left) 2024-04-20T01:48:15.700 INFO:tasks.rados.rados.0.smithi165.stdout:292: done (3 left) 2024-04-20T01:48:15.700 INFO:tasks.rados.rados.0.smithi165.stdout:294: done (2 left) 2024-04-20T01:48:15.700 INFO:tasks.rados.rados.0.smithi165.stdout:298: done (1 left) 2024-04-20T01:48:15.700 INFO:tasks.rados.rados.0.smithi165.stdout:299: done (0 left) 2024-04-20T01:48:15.700 INFO:tasks.rados.rados.0.smithi165.stdout:300: delete oid 50 current snap is 22 2024-04-20T01:48:15.707 INFO:tasks.rados.rados.0.smithi165.stdout:300: done (0 left) 2024-04-20T01:48:15.707 INFO:tasks.rados.rados.0.smithi165.stdout:301: snap_remove snap 14 2024-04-20T01:48:15.707 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:15.721+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:15.778 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:15.778 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:15.778 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:48:15.779 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:15.788 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-20T01:48:15.832 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:48:15.846+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:48:15.908 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:15.922+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:15.915 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:15.915 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:15.915 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 30, 2024-04-20T01:48:15.916 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-20T01:48:15.916 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:15.926 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-20T01:48:16.010 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:16.024+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:16.097 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:16.098 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-20T01:48:16.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-20T01:48:16.098 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-20T01:48:16.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.098 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:10 3.5s0 3:a2e9ac6b:::smithi16579500-4:head [append 0~65536,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-20T01:48:16.098 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.752800+0000", 2024-04-20T01:48:16.098 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.358076640999997, 2024-04-20T01:48:16.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011440333, 2024-04-20T01:48:16.098 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.098 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:16.098 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:16.099 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:16.099 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:16.099 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 10 2024-04-20T01:48:16.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.099 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752800+0000", 2024-04-20T01:48:16.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752800+0000", 2024-04-20T01:48:16.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752799+0000", 2024-04-20T01:48:16.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:16.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752945+0000", 2024-04-20T01:48:16.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000145761 2024-04-20T01:48:16.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752955+0000", 2024-04-20T01:48:16.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0229e-05 2024-04-20T01:48:16.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752970+0000", 2024-04-20T01:48:16.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5242000000000001e-05 2024-04-20T01:48:16.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753046+0000", 2024-04-20T01:48:16.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6218999999999996e-05 2024-04-20T01:48:16.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:16.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753172+0000", 2024-04-20T01:48:16.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012566899999999999 2024-04-20T01:48:16.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753648+0000", 2024-04-20T01:48:16.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00047565500000000001 2024-04-20T01:48:16.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.756671+0000", 2024-04-20T01:48:16.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030231849999999998 2024-04-20T01:48:16.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:16.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764191+0000", 2024-04-20T01:48:16.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007520143 2024-04-20T01:48:16.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764240+0000", 2024-04-20T01:48:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9359e-05 2024-04-20T01:48:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:11 3.5s0 3:a2e9ac6b:::smithi16579500-4:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-20T01:48:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.752987+0000", 2024-04-20T01:48:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.357890005000002, 2024-04-20T01:48:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011858695000000001, 2024-04-20T01:48:16.104 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 11 2024-04-20T01:48:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752987+0000", 2024-04-20T01:48:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752987+0000", 2024-04-20T01:48:16.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752986+0000", 2024-04-20T01:48:16.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752995+0000", 2024-04-20T01:48:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7099999999999996e-06 2024-04-20T01:48:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753000+0000", 2024-04-20T01:48:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4900000000000002e-06 2024-04-20T01:48:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753003+0000", 2024-04-20T01:48:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4620000000000001e-06 2024-04-20T01:48:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753974+0000", 2024-04-20T01:48:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000971388 2024-04-20T01:48:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754054+0000", 2024-04-20T01:48:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9159000000000006e-05 2024-04-20T01:48:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754358+0000", 2024-04-20T01:48:16.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00030434800000000002 2024-04-20T01:48:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.759666+0000", 2024-04-20T01:48:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053076169999999997 2024-04-20T01:48:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764756+0000", 2024-04-20T01:48:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050904139999999997 2024-04-20T01:48:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764845+0000", 2024-04-20T01:48:16.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9488999999999996e-05 2024-04-20T01:48:16.110 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.110 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.110 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.110 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:12 3.5s0 3:a2e9ac6b:::smithi16579500-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-20T01:48:16.110 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.753012+0000", 2024-04-20T01:48:16.110 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.357864988000003, 2024-04-20T01:48:16.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014201745, 2024-04-20T01:48:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 12 2024-04-20T01:48:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753012+0000", 2024-04-20T01:48:16.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753012+0000", 2024-04-20T01:48:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753011+0000", 2024-04-20T01:48:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753016+0000", 2024-04-20T01:48:16.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.989e-06 2024-04-20T01:48:16.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753026+0000", 2024-04-20T01:48:16.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2690000000000005e-06 2024-04-20T01:48:16.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.114 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753033+0000", 2024-04-20T01:48:16.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3010000000000001e-06 2024-04-20T01:48:16.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.114 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.114 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754553+0000", 2024-04-20T01:48:16.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001519947 2024-04-20T01:48:16.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.114 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:16.114 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754638+0000", 2024-04-20T01:48:16.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5421999999999996e-05 2024-04-20T01:48:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.765222+0000", 2024-04-20T01:48:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010583979 2024-04-20T01:48:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.765288+0000", 2024-04-20T01:48:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5296999999999995e-05 2024-04-20T01:48:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.767213+0000", 2024-04-20T01:48:16.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019259520000000001 2024-04-20T01:48:16.116 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.116 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.116 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.116 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:64 3.5s0 3:ae9f92b8:::smithi16579500-22:head [append 0~360448,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:16.116 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.781054+0000", 2024-04-20T01:48:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.32982252, 2024-04-20T01:48:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01202491, 2024-04-20T01:48:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 64 2024-04-20T01:48:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781054+0000", 2024-04-20T01:48:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781054+0000", 2024-04-20T01:48:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781053+0000", 2024-04-20T01:48:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:16.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781384+0000", 2024-04-20T01:48:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00033060899999999999 2024-04-20T01:48:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781394+0000", 2024-04-20T01:48:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0016e-05 2024-04-20T01:48:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781399+0000", 2024-04-20T01:48:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7930000000000003e-06 2024-04-20T01:48:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781446+0000", 2024-04-20T01:48:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6768999999999999e-05 2024-04-20T01:48:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781598+0000", 2024-04-20T01:48:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015158500000000001 2024-04-20T01:48:16.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.782377+0000", 2024-04-20T01:48:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000778975 2024-04-20T01:48:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784105+0000", 2024-04-20T01:48:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017285340000000001 2024-04-20T01:48:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:16.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793040+0000", 2024-04-20T01:48:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0089346440000000003 2024-04-20T01:48:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793079+0000", 2024-04-20T01:48:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9239000000000002e-05 2024-04-20T01:48:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:65 3.5s0 3:ae9f92b8:::smithi16579500-22:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:16.122 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.781409+0000", 2024-04-20T01:48:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.329467919999999, 2024-04-20T01:48:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011886404, 2024-04-20T01:48:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 65 2024-04-20T01:48:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781409+0000", 2024-04-20T01:48:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781409+0000", 2024-04-20T01:48:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781408+0000", 2024-04-20T01:48:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781411+0000", 2024-04-20T01:48:16.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6309999999999999e-06 2024-04-20T01:48:16.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781416+0000", 2024-04-20T01:48:16.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6550000000000003e-06 2024-04-20T01:48:16.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781423+0000", 2024-04-20T01:48:16.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4540000000000001e-06 2024-04-20T01:48:16.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.782838+0000", 2024-04-20T01:48:16.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001414741 2024-04-20T01:48:16.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:16.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.782921+0000", 2024-04-20T01:48:16.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3018999999999999e-05 2024-04-20T01:48:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.783284+0000", 2024-04-20T01:48:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00036349 2024-04-20T01:48:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784149+0000", 2024-04-20T01:48:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00086487700000000001 2024-04-20T01:48:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793195+0000", 2024-04-20T01:48:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0090454590000000005 2024-04-20T01:48:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793295+0000", 2024-04-20T01:48:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010049 2024-04-20T01:48:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.128 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:66 3.5s0 3:ae9f92b8:::smithi16579500-22:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:16.129 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.781436+0000", 2024-04-20T01:48:16.129 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.329440489, 2024-04-20T01:48:16.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014237204999999999, 2024-04-20T01:48:16.129 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.129 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:16.129 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:16.129 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:16.129 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:16.129 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 66 2024-04-20T01:48:16.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.129 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781436+0000", 2024-04-20T01:48:16.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781436+0000", 2024-04-20T01:48:16.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781436+0000", 2024-04-20T01:48:16.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781441+0000", 2024-04-20T01:48:16.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3199999999999999e-06 2024-04-20T01:48:16.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781446+0000", 2024-04-20T01:48:16.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.544e-06 2024-04-20T01:48:16.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781451+0000", 2024-04-20T01:48:16.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5570000000000003e-06 2024-04-20T01:48:16.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.132 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.783430+0000", 2024-04-20T01:48:16.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019785699999999998 2024-04-20T01:48:16.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.132 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:16.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.783482+0000", 2024-04-20T01:48:16.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1780000000000002e-05 2024-04-20T01:48:16.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793826+0000", 2024-04-20T01:48:16.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010344132000000001 2024-04-20T01:48:16.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:16.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793907+0000", 2024-04-20T01:48:16.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1173000000000004e-05 2024-04-20T01:48:16.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.795673+0000", 2024-04-20T01:48:16.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001766442 2024-04-20T01:48:16.134 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.134 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.134 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.134 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s6 33/30 ECSubWrite(tid=177, reqid=client.4643.0:83, at_version=33'6, trim_to=0'0, roll_forward_to=33'4))", 2024-04-20T01:48:16.134 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.800448+0000", 2024-04-20T01:48:16.134 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.310428549000001, 2024-04-20T01:48:16.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005436504, 2024-04-20T01:48:16.134 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.134 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.134 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.800448+0000", 2024-04-20T01:48:16.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.800448+0000", 2024-04-20T01:48:16.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.800448+0000", 2024-04-20T01:48:16.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.800463+0000", 2024-04-20T01:48:16.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5854000000000001e-05 2024-04-20T01:48:16.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.800476+0000", 2024-04-20T01:48:16.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2157999999999999e-05 2024-04-20T01:48:16.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.800485+0000", 2024-04-20T01:48:16.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9020000000000005e-06 2024-04-20T01:48:16.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.800538+0000", 2024-04-20T01:48:16.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3647000000000003e-05 2024-04-20T01:48:16.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.800545+0000", 2024-04-20T01:48:16.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8580000000000002e-06 2024-04-20T01:48:16.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.805859+0000", 2024-04-20T01:48:16.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053143360000000002 2024-04-20T01:48:16.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.805885+0000", 2024-04-20T01:48:16.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5301999999999999e-05 2024-04-20T01:48:16.138 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.138 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.139 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.139 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.bs5 47/30 ECSubWrite(tid=114, reqid=client.4643.0:392, at_version=47'49, trim_to=0'0, roll_forward_to=47'47))", 2024-04-20T01:48:16.139 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:48.746780+0000", 2024-04-20T01:48:16.139 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.364096929999999, 2024-04-20T01:48:16.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054560069999999997, 2024-04-20T01:48:16.139 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.139 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.139 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746780+0000", 2024-04-20T01:48:16.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746780+0000", 2024-04-20T01:48:16.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746779+0000", 2024-04-20T01:48:16.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:16.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746900+0000", 2024-04-20T01:48:16.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121138 2024-04-20T01:48:16.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746917+0000", 2024-04-20T01:48:16.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6852999999999999e-05 2024-04-20T01:48:16.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746927+0000", 2024-04-20T01:48:16.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0237999999999999e-05 2024-04-20T01:48:16.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746982+0000", 2024-04-20T01:48:16.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5290000000000001e-05 2024-04-20T01:48:16.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746993+0000", 2024-04-20T01:48:16.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0781000000000001e-05 2024-04-20T01:48:16.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.752210+0000", 2024-04-20T01:48:16.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052170059999999997 2024-04-20T01:48:16.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.752236+0000", 2024-04-20T01:48:16.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5672000000000001e-05 2024-04-20T01:48:16.143 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.143 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.143 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.143 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 47/30 ECSubWrite(tid=247, reqid=client.4643.0:389, at_version=47'39, trim_to=0'0, roll_forward_to=47'35))", 2024-04-20T01:48:16.144 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:48.746964+0000", 2024-04-20T01:48:16.144 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.363912295999999, 2024-04-20T01:48:16.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056576650000000001, 2024-04-20T01:48:16.144 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.144 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.144 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.144 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.144 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746964+0000", 2024-04-20T01:48:16.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.145 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.145 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746964+0000", 2024-04-20T01:48:16.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.145 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.145 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746964+0000", 2024-04-20T01:48:16.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:16.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747077+0000", 2024-04-20T01:48:16.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113581 2024-04-20T01:48:16.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747089+0000", 2024-04-20T01:48:16.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1795e-05 2024-04-20T01:48:16.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747098+0000", 2024-04-20T01:48:16.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1630000000000002e-06 2024-04-20T01:48:16.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747143+0000", 2024-04-20T01:48:16.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5286e-05 2024-04-20T01:48:16.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747154+0000", 2024-04-20T01:48:16.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0488e-05 2024-04-20T01:48:16.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.752599+0000", 2024-04-20T01:48:16.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054446670000000003 2024-04-20T01:48:16.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.752622+0000", 2024-04-20T01:48:16.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3521999999999999e-05 2024-04-20T01:48:16.148 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.148 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.148 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.149 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 47/30 ECSubWrite(tid=248, reqid=client.4643.0:390, at_version=47'40, trim_to=0'0, roll_forward_to=47'35))", 2024-04-20T01:48:16.149 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:48.747742+0000", 2024-04-20T01:48:16.149 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.363134812999999, 2024-04-20T01:48:16.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049122280000000003, 2024-04-20T01:48:16.149 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.149 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.149 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747742+0000", 2024-04-20T01:48:16.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747742+0000", 2024-04-20T01:48:16.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747741+0000", 2024-04-20T01:48:16.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:16.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747771+0000", 2024-04-20T01:48:16.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0212000000000001e-05 2024-04-20T01:48:16.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747785+0000", 2024-04-20T01:48:16.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3305000000000001e-05 2024-04-20T01:48:16.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747794+0000", 2024-04-20T01:48:16.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8780000000000004e-06 2024-04-20T01:48:16.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747843+0000", 2024-04-20T01:48:16.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9455e-05 2024-04-20T01:48:16.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747854+0000", 2024-04-20T01:48:16.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0723999999999999e-05 2024-04-20T01:48:16.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.752643+0000", 2024-04-20T01:48:16.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047892969999999997 2024-04-20T01:48:16.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.752654+0000", 2024-04-20T01:48:16.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1133e-05 2024-04-20T01:48:16.153 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.153 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.153 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.153 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s6 48/30 ECSubWrite(tid=251, reqid=osd.13.0:251, at_version=48'22, trim_to=0'0, roll_forward_to=47'20))", 2024-04-20T01:48:16.153 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.710374+0000", 2024-04-20T01:48:16.154 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.400502586000002, 2024-04-20T01:48:16.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069805379999999997, 2024-04-20T01:48:16.154 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.154 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.154 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710374+0000", 2024-04-20T01:48:16.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710374+0000", 2024-04-20T01:48:16.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710374+0000", 2024-04-20T01:48:16.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710395+0000", 2024-04-20T01:48:16.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1396e-05 2024-04-20T01:48:16.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.156 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.156 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710417+0000", 2024-04-20T01:48:16.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1857000000000001e-05 2024-04-20T01:48:16.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.156 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.156 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710423+0000", 2024-04-20T01:48:16.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1469999999999998e-06 2024-04-20T01:48:16.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.156 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710498+0000", 2024-04-20T01:48:16.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5072000000000002e-05 2024-04-20T01:48:16.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710509+0000", 2024-04-20T01:48:16.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0685e-05 2024-04-20T01:48:16.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717332+0000", 2024-04-20T01:48:16.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006823048 2024-04-20T01:48:16.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717355+0000", 2024-04-20T01:48:16.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2731e-05 2024-04-20T01:48:16.158 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.158 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.158 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.158 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 48/30 ECSubWrite(tid=253, reqid=osd.13.0:253, at_version=48'44, trim_to=0'0, roll_forward_to=47'40))", 2024-04-20T01:48:16.158 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.711134+0000", 2024-04-20T01:48:16.158 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.399742609, 2024-04-20T01:48:16.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006623304, 2024-04-20T01:48:16.159 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.159 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.159 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711134+0000", 2024-04-20T01:48:16.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711134+0000", 2024-04-20T01:48:16.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711134+0000", 2024-04-20T01:48:16.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711155+0000", 2024-04-20T01:48:16.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1673000000000001e-05 2024-04-20T01:48:16.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711166+0000", 2024-04-20T01:48:16.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0558e-05 2024-04-20T01:48:16.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711174+0000", 2024-04-20T01:48:16.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3359999999999995e-06 2024-04-20T01:48:16.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711257+0000", 2024-04-20T01:48:16.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3312999999999993e-05 2024-04-20T01:48:16.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711273+0000", 2024-04-20T01:48:16.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5701000000000001e-05 2024-04-20T01:48:16.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717738+0000", 2024-04-20T01:48:16.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064653339999999997 2024-04-20T01:48:16.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717757+0000", 2024-04-20T01:48:16.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8972999999999999e-05 2024-04-20T01:48:16.163 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.163 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.163 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.163 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 60/30 ECSubWrite(tid=280, reqid=client.4643.0:519, at_version=60'66, trim_to=0'0, roll_forward_to=60'63))", 2024-04-20T01:48:16.163 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:01.837547+0000", 2024-04-20T01:48:16.163 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.273330132, 2024-04-20T01:48:16.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053258159999999997, 2024-04-20T01:48:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837547+0000", 2024-04-20T01:48:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837547+0000", 2024-04-20T01:48:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837542+0000", 2024-04-20T01:48:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999952 2024-04-20T01:48:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837820+0000", 2024-04-20T01:48:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00027825499999999998 2024-04-20T01:48:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837834+0000", 2024-04-20T01:48:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3789e-05 2024-04-20T01:48:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837843+0000", 2024-04-20T01:48:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0720000000000006e-06 2024-04-20T01:48:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837887+0000", 2024-04-20T01:48:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4434000000000002e-05 2024-04-20T01:48:16.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837899+0000", 2024-04-20T01:48:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1554e-05 2024-04-20T01:48:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.842847+0000", 2024-04-20T01:48:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049483449999999998 2024-04-20T01:48:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.842872+0000", 2024-04-20T01:48:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5024e-05 2024-04-20T01:48:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 63/30 ECSubWrite(tid=295, reqid=client.4643.0:574, at_version=63'79, trim_to=0'0, roll_forward_to=62'77))", 2024-04-20T01:48:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:04.795161+0000", 2024-04-20T01:48:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 11.31571589, 2024-04-20T01:48:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0075404740000000001, 2024-04-20T01:48:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.168 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795161+0000", 2024-04-20T01:48:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795161+0000", 2024-04-20T01:48:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795159+0000", 2024-04-20T01:48:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795678+0000", 2024-04-20T01:48:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00051905099999999997 2024-04-20T01:48:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795694+0000", 2024-04-20T01:48:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6076e-05 2024-04-20T01:48:16.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795704+0000", 2024-04-20T01:48:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0078000000000001e-05 2024-04-20T01:48:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795753+0000", 2024-04-20T01:48:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9055999999999999e-05 2024-04-20T01:48:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795763+0000", 2024-04-20T01:48:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6430000000000007e-06 2024-04-20T01:48:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.802666+0000", 2024-04-20T01:48:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069029149999999999 2024-04-20T01:48:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.802701+0000", 2024-04-20T01:48:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5664999999999998e-05 2024-04-20T01:48:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.172 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 63/30 ECSubWrite(tid=296, reqid=client.4643.0:575, at_version=63'80, trim_to=0'0, roll_forward_to=62'77))", 2024-04-20T01:48:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:04.798493+0000", 2024-04-20T01:48:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 11.312383284999999, 2024-04-20T01:48:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069235080000000001, 2024-04-20T01:48:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798493+0000", 2024-04-20T01:48:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798493+0000", 2024-04-20T01:48:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798493+0000", 2024-04-20T01:48:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798862+0000", 2024-04-20T01:48:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00036897900000000002 2024-04-20T01:48:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798876+0000", 2024-04-20T01:48:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4321999999999999e-05 2024-04-20T01:48:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798884+0000", 2024-04-20T01:48:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.157e-06 2024-04-20T01:48:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798936+0000", 2024-04-20T01:48:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1879000000000003e-05 2024-04-20T01:48:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798949+0000", 2024-04-20T01:48:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3359e-05 2024-04-20T01:48:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.805386+0000", 2024-04-20T01:48:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064363470000000002 2024-04-20T01:48:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.805417+0000", 2024-04-20T01:48:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1260999999999998e-05 2024-04-20T01:48:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.178 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 63/30 ECSubWrite(tid=297, reqid=client.4643.0:576, at_version=63'81, trim_to=0'0, roll_forward_to=62'77))", 2024-04-20T01:48:16.178 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:04.801031+0000", 2024-04-20T01:48:16.178 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 11.309845613, 2024-04-20T01:48:16.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051312559999999998, 2024-04-20T01:48:16.178 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.178 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.178 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801031+0000", 2024-04-20T01:48:16.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801031+0000", 2024-04-20T01:48:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801030+0000", 2024-04-20T01:48:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801318+0000", 2024-04-20T01:48:16.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00028803100000000001 2024-04-20T01:48:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801334+0000", 2024-04-20T01:48:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5420000000000001e-05 2024-04-20T01:48:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801343+0000", 2024-04-20T01:48:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5049999999999996e-06 2024-04-20T01:48:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801396+0000", 2024-04-20T01:48:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2604e-05 2024-04-20T01:48:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.801407+0000", 2024-04-20T01:48:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1269e-05 2024-04-20T01:48:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.182 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.806131+0000", 2024-04-20T01:48:16.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047239389999999999 2024-04-20T01:48:16.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.182 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.182 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.182 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.806162+0000", 2024-04-20T01:48:16.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1321e-05 2024-04-20T01:48:16.182 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.182 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.182 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.182 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 68/30 ECSubWrite(tid=334, reqid=client.4643.0:660, at_version=68'108, trim_to=0'0, roll_forward_to=68'106))", 2024-04-20T01:48:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:09.932199+0000", 2024-04-20T01:48:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.1786779569999997, 2024-04-20T01:48:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051865849999999996, 2024-04-20T01:48:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.932199+0000", 2024-04-20T01:48:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.932199+0000", 2024-04-20T01:48:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.932197+0000", 2024-04-20T01:48:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.933682+0000", 2024-04-20T01:48:16.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014853500000000001 2024-04-20T01:48:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.933694+0000", 2024-04-20T01:48:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1664999999999999e-05 2024-04-20T01:48:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.933701+0000", 2024-04-20T01:48:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7739999999999999e-06 2024-04-20T01:48:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.933768+0000", 2024-04-20T01:48:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7275999999999998e-05 2024-04-20T01:48:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.933787+0000", 2024-04-20T01:48:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8868e-05 2024-04-20T01:48:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.937365+0000", 2024-04-20T01:48:16.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035783070000000002 2024-04-20T01:48:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.937385+0000", 2024-04-20T01:48:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0058999999999999e-05 2024-04-20T01:48:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:717 3.5s0 3:ae9f92b8:::smithi16579500-22:head [read 0~2129920,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.113707+0000", 2024-04-20T01:48:16.187 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.997169456, 2024-04-20T01:48:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083649959999999995, 2024-04-20T01:48:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 717 2024-04-20T01:48:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113707+0000", 2024-04-20T01:48:16.188 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113707+0000", 2024-04-20T01:48:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113706+0000", 2024-04-20T01:48:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113714+0000", 2024-04-20T01:48:16.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.371e-06 2024-04-20T01:48:16.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113726+0000", 2024-04-20T01:48:16.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1572e-05 2024-04-20T01:48:16.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113732+0000", 2024-04-20T01:48:16.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2870000000000001e-06 2024-04-20T01:48:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113785+0000", 2024-04-20T01:48:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2648000000000001e-05 2024-04-20T01:48:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113900+0000", 2024-04-20T01:48:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011510699999999999 2024-04-20T01:48:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.192 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.122072+0000", 2024-04-20T01:48:16.192 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0081722930000000006 2024-04-20T01:48:16.192 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.192 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.192 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.192 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.192 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.192 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:718 3.5s0 3:ae9f92b8:::smithi16579500-22:head [read 0~2129920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:16.192 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.113742+0000", 2024-04-20T01:48:16.192 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.9971343849999998, 2024-04-20T01:48:16.192 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095008990000000001, 2024-04-20T01:48:16.192 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.192 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 718 2024-04-20T01:48:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113742+0000", 2024-04-20T01:48:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.193 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.194 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113742+0000", 2024-04-20T01:48:16.194 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.194 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.194 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.194 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.194 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113742+0000", 2024-04-20T01:48:16.194 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.194 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.194 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113744+0000", 2024-04-20T01:48:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4219999999999999e-06 2024-04-20T01:48:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113749+0000", 2024-04-20T01:48:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.313e-06 2024-04-20T01:48:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113752+0000", 2024-04-20T01:48:16.195 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2449999999999998e-06 2024-04-20T01:48:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.117251+0000", 2024-04-20T01:48:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034990360000000001 2024-04-20T01:48:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.117402+0000", 2024-04-20T01:48:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000151099 2024-04-20T01:48:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.196 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.123243+0000", 2024-04-20T01:48:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058410800000000002 2024-04-20T01:48:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:719 3.5s0 3:ae9f92b8:::smithi16579500-22:head [read 0~2129920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.113770+0000", 2024-04-20T01:48:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.997106767, 2024-04-20T01:48:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010243805999999999, 2024-04-20T01:48:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.197 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:16.198 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:16.198 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:16.198 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:16.198 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 719 2024-04-20T01:48:16.198 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.198 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.198 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.198 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.198 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113770+0000", 2024-04-20T01:48:16.198 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.198 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113770+0000", 2024-04-20T01:48:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113770+0000", 2024-04-20T01:48:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.199 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113772+0000", 2024-04-20T01:48:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1270000000000001e-06 2024-04-20T01:48:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113777+0000", 2024-04-20T01:48:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4410000000000001e-06 2024-04-20T01:48:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113780+0000", 2024-04-20T01:48:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.869e-06 2024-04-20T01:48:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.200 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.120794+0000", 2024-04-20T01:48:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070135659999999997 2024-04-20T01:48:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.120909+0000", 2024-04-20T01:48:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011497099999999999 2024-04-20T01:48:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.124014+0000", 2024-04-20T01:48:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00310509 2024-04-20T01:48:16.201 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.202 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.202 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.202 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.202 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.202 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:16.204 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-20T01:48:16.204 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:16.216+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:16.304 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:16.318+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:16.373 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:16.373 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:16.373 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:48:16.373 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:16.382 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-20T01:48:16.405 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:16.418+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:16.506 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:16.520+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:16.509 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 30, 2024-04-20T01:48:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-20T01:48:16.510 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:16.520 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-20T01:48:16.606 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:16.620+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:16.648 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:16.648 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-20T01:48:16.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-20T01:48:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-20T01:48:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.5s3 32/30 ECSubWrite(tid=1, reqid=client.4643.0:11, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:48:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.754397+0000", 2024-04-20T01:48:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.907419976, 2024-04-20T01:48:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01012041, 2024-04-20T01:48:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.649 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754397+0000", 2024-04-20T01:48:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754397+0000", 2024-04-20T01:48:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754396+0000", 2024-04-20T01:48:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:16.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754423+0000", 2024-04-20T01:48:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7032000000000001e-05 2024-04-20T01:48:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754438+0000", 2024-04-20T01:48:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4946999999999999e-05 2024-04-20T01:48:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754447+0000", 2024-04-20T01:48:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8550000000000008e-06 2024-04-20T01:48:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754496+0000", 2024-04-20T01:48:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9018999999999999e-05 2024-04-20T01:48:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754506+0000", 2024-04-20T01:48:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7079999999999992e-06 2024-04-20T01:48:16.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764493+0000", 2024-04-20T01:48:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0099871460000000006 2024-04-20T01:48:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764517+0000", 2024-04-20T01:48:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4467000000000001e-05 2024-04-20T01:48:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.653 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.9s4 32/30 ECSubWrite(tid=2, reqid=client.4643.0:7, at_version=32'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:48:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.755700+0000", 2024-04-20T01:48:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.906116595, 2024-04-20T01:48:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0088510410000000005, 2024-04-20T01:48:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755700+0000", 2024-04-20T01:48:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755700+0000", 2024-04-20T01:48:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755700+0000", 2024-04-20T01:48:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755735+0000", 2024-04-20T01:48:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5524000000000001e-05 2024-04-20T01:48:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755750+0000", 2024-04-20T01:48:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4609e-05 2024-04-20T01:48:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755759+0000", 2024-04-20T01:48:16.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3700000000000001e-06 2024-04-20T01:48:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755799+0000", 2024-04-20T01:48:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9560000000000001e-05 2024-04-20T01:48:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755804+0000", 2024-04-20T01:48:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3149999999999999e-06 2024-04-20T01:48:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764532+0000", 2024-04-20T01:48:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087277770000000008 2024-04-20T01:48:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764551+0000", 2024-04-20T01:48:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9582e-05 2024-04-20T01:48:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.9s4 32/30 ECSubWrite(tid=3, reqid=client.4643.0:8, at_version=32'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:48:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.755948+0000", 2024-04-20T01:48:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.905869418999998, 2024-04-20T01:48:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0086164850000000001, 2024-04-20T01:48:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755948+0000", 2024-04-20T01:48:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755948+0000", 2024-04-20T01:48:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755947+0000", 2024-04-20T01:48:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755963+0000", 2024-04-20T01:48:16.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6013000000000001e-05 2024-04-20T01:48:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755975+0000", 2024-04-20T01:48:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2347000000000001e-05 2024-04-20T01:48:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.755985+0000", 2024-04-20T01:48:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8730000000000002e-06 2024-04-20T01:48:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.756022+0000", 2024-04-20T01:48:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7039000000000002e-05 2024-04-20T01:48:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.756027+0000", 2024-04-20T01:48:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.006e-06 2024-04-20T01:48:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764554+0000", 2024-04-20T01:48:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0085269609999999996 2024-04-20T01:48:16.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764564+0000", 2024-04-20T01:48:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8740000000000007e-06 2024-04-20T01:48:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s3 33/30 ECSubWrite(tid=157, reqid=client.4643.0:16, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:48:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.757106+0000", 2024-04-20T01:48:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.904710735999998, 2024-04-20T01:48:16.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0086461160000000006, 2024-04-20T01:48:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757106+0000", 2024-04-20T01:48:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757106+0000", 2024-04-20T01:48:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757106+0000", 2024-04-20T01:48:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757173+0000", 2024-04-20T01:48:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6734000000000002e-05 2024-04-20T01:48:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.665 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757184+0000", 2024-04-20T01:48:16.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0896999999999999e-05 2024-04-20T01:48:16.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.666 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.666 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757192+0000", 2024-04-20T01:48:16.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4460000000000001e-06 2024-04-20T01:48:16.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.666 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.666 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757248+0000", 2024-04-20T01:48:16.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5949999999999998e-05 2024-04-20T01:48:16.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.757257+0000", 2024-04-20T01:48:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.935e-06 2024-04-20T01:48:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.765715+0000", 2024-04-20T01:48:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0084578169999999994 2024-04-20T01:48:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.765752+0000", 2024-04-20T01:48:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7608000000000003e-05 2024-04-20T01:48:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s3 33/30 ECSubWrite(tid=168, reqid=client.4643.0:67, at_version=33'3, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:48:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.784794+0000", 2024-04-20T01:48:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.877022900999997, 2024-04-20T01:48:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011714966, 2024-04-20T01:48:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.668 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.669 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.669 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.669 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784794+0000", 2024-04-20T01:48:16.669 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.669 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.669 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.669 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784794+0000", 2024-04-20T01:48:16.669 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.670 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.670 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784794+0000", 2024-04-20T01:48:16.670 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.670 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.670 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784957+0000", 2024-04-20T01:48:16.670 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016303800000000001 2024-04-20T01:48:16.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.671 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.671 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784970+0000", 2024-04-20T01:48:16.671 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3305000000000001e-05 2024-04-20T01:48:16.671 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.671 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.671 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.671 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784979+0000", 2024-04-20T01:48:16.671 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7129999999999994e-06 2024-04-20T01:48:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785046+0000", 2024-04-20T01:48:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7108999999999997e-05 2024-04-20T01:48:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785055+0000", 2024-04-20T01:48:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.234e-06 2024-04-20T01:48:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.672 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.673 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796482+0000", 2024-04-20T01:48:16.673 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011427196000000001 2024-04-20T01:48:16.673 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.673 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.673 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.673 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796509+0000", 2024-04-20T01:48:16.673 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6999e-05 2024-04-20T01:48:16.673 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.673 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.673 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s3 33/30 ECSubWrite(tid=169, reqid=client.4643.0:68, at_version=33'4, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:48:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.785553+0000", 2024-04-20T01:48:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.876264063999997, 2024-04-20T01:48:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010979604, 2024-04-20T01:48:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.674 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785553+0000", 2024-04-20T01:48:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785553+0000", 2024-04-20T01:48:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785552+0000", 2024-04-20T01:48:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.675 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.676 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.676 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785579+0000", 2024-04-20T01:48:16.676 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6492999999999999e-05 2024-04-20T01:48:16.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.676 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.676 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.676 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785589+0000", 2024-04-20T01:48:16.676 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0353000000000001e-05 2024-04-20T01:48:16.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.676 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.676 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.785598+0000", 2024-04-20T01:48:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8130000000000002e-06 2024-04-20T01:48:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788372+0000", 2024-04-20T01:48:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027740410000000001 2024-04-20T01:48:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.788381+0000", 2024-04-20T01:48:16.677 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5930000000000006e-06 2024-04-20T01:48:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796518+0000", 2024-04-20T01:48:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0081374710000000003 2024-04-20T01:48:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796533+0000", 2024-04-20T01:48:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.451e-05 2024-04-20T01:48:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.678 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s4 33/30 ECSubWrite(tid=10, reqid=client.4643.0:70, at_version=33'7, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:48:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.787329+0000", 2024-04-20T01:48:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.874488513999999, 2024-04-20T01:48:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092166409999999994, 2024-04-20T01:48:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.679 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787329+0000", 2024-04-20T01:48:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787329+0000", 2024-04-20T01:48:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787328+0000", 2024-04-20T01:48:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.680 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787448+0000", 2024-04-20T01:48:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011992199999999999 2024-04-20T01:48:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787458+0000", 2024-04-20T01:48:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0037000000000001e-05 2024-04-20T01:48:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787466+0000", 2024-04-20T01:48:16.681 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6270000000000006e-06 2024-04-20T01:48:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787517+0000", 2024-04-20T01:48:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0890000000000002e-05 2024-04-20T01:48:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.787528+0000", 2024-04-20T01:48:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0363000000000001e-05 2024-04-20T01:48:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796522+0000", 2024-04-20T01:48:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0089942619999999994 2024-04-20T01:48:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.796545+0000", 2024-04-20T01:48:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3213999999999998e-05 2024-04-20T01:48:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.683 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:108 3.cs0 3:3f280175:::smithi16579500-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.824485+0000", 2024-04-20T01:48:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.837332492999998, 2024-04-20T01:48:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0099600550000000006, 2024-04-20T01:48:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 108 2024-04-20T01:48:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.684 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824485+0000", 2024-04-20T01:48:16.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.685 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.685 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824485+0000", 2024-04-20T01:48:16.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.685 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.685 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824484+0000", 2024-04-20T01:48:16.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.686 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.686 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.686 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824487+0000", 2024-04-20T01:48:16.686 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4889999999999998e-06 2024-04-20T01:48:16.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.686 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.686 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.686 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824493+0000", 2024-04-20T01:48:16.686 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1709999999999999e-06 2024-04-20T01:48:16.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.686 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.824498+0000", 2024-04-20T01:48:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1569999999999999e-06 2024-04-20T01:48:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.826844+0000", 2024-04-20T01:48:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023460249999999998 2024-04-20T01:48:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.826938+0000", 2024-04-20T01:48:16.687 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3547000000000004e-05 2024-04-20T01:48:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.833332+0000", 2024-04-20T01:48:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006394846 2024-04-20T01:48:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.833419+0000", 2024-04-20T01:48:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6304999999999994e-05 2024-04-20T01:48:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.688 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.834445+0000", 2024-04-20T01:48:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001025946 2024-04-20T01:48:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:325 3.cs0 3:3f280175:::smithi16579500-36:head [append 0~425984] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.949234+0000", 2024-04-20T01:48:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.712583311000003, 2024-04-20T01:48:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092546179999999992, 2024-04-20T01:48:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.689 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 325 2024-04-20T01:48:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.949234+0000", 2024-04-20T01:48:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.690 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.949234+0000", 2024-04-20T01:48:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.949232+0000", 2024-04-20T01:48:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.949737+0000", 2024-04-20T01:48:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00050513299999999995 2024-04-20T01:48:16.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.949751+0000", 2024-04-20T01:48:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3966e-05 2024-04-20T01:48:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.949776+0000", 2024-04-20T01:48:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4946000000000001e-05 2024-04-20T01:48:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.692 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.949831+0000", 2024-04-20T01:48:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4591e-05 2024-04-20T01:48:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.949953+0000", 2024-04-20T01:48:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012193600000000001 2024-04-20T01:48:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.951143+0000", 2024-04-20T01:48:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001190387 2024-04-20T01:48:16.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.952463+0000", 2024-04-20T01:48:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013197879999999999 2024-04-20T01:48:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.958405+0000", 2024-04-20T01:48:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005941956 2024-04-20T01:48:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.958488+0000", 2024-04-20T01:48:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3463999999999999e-05 2024-04-20T01:48:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:328 3.cs0 3:3f280175:::smithi16579500-36:head [setxattr _header (56) in=7b] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.953789+0000", 2024-04-20T01:48:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.708027649999998, 2024-04-20T01:48:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0088952739999999999, 2024-04-20T01:48:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.695 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 328 2024-04-20T01:48:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953789+0000", 2024-04-20T01:48:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.696 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953789+0000", 2024-04-20T01:48:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953789+0000", 2024-04-20T01:48:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953793+0000", 2024-04-20T01:48:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3460000000000004e-06 2024-04-20T01:48:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953797+0000", 2024-04-20T01:48:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7570000000000002e-06 2024-04-20T01:48:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953800+0000", 2024-04-20T01:48:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0809999999999998e-06 2024-04-20T01:48:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.698 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957330+0000", 2024-04-20T01:48:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035297430000000001 2024-04-20T01:48:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957432+0000", 2024-04-20T01:48:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010186799999999999 2024-04-20T01:48:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.957890+0000", 2024-04-20T01:48:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00045805100000000001 2024-04-20T01:48:16.699 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.962457+0000", 2024-04-20T01:48:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045676720000000001 2024-04-20T01:48:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.962618+0000", 2024-04-20T01:48:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016040600000000001 2024-04-20T01:48:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.700 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.962685+0000", 2024-04-20T01:48:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6853999999999994e-05 2024-04-20T01:48:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:329 3.cs0 3:3f280175:::smithi16579500-36:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.953803+0000", 2024-04-20T01:48:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.708014392000003, 2024-04-20T01:48:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011991458999999999, 2024-04-20T01:48:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.701 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 329 2024-04-20T01:48:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953803+0000", 2024-04-20T01:48:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953803+0000", 2024-04-20T01:48:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953802+0000", 2024-04-20T01:48:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953805+0000", 2024-04-20T01:48:16.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.553e-06 2024-04-20T01:48:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953809+0000", 2024-04-20T01:48:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.715e-06 2024-04-20T01:48:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.953812+0000", 2024-04-20T01:48:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0529999999999998e-06 2024-04-20T01:48:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.958685+0000", 2024-04-20T01:48:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048727739999999999 2024-04-20T01:48:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.958748+0000", 2024-04-20T01:48:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3748e-05 2024-04-20T01:48:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.962966+0000", 2024-04-20T01:48:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042174300000000003 2024-04-20T01:48:16.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.963059+0000", 2024-04-20T01:48:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3437999999999995e-05 2024-04-20T01:48:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.965794+0000", 2024-04-20T01:48:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027349990000000001 2024-04-20T01:48:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.706 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s4 48/30 ECSubWrite(tid=253, reqid=osd.13.0:253, at_version=48'44, trim_to=0'0, roll_forward_to=47'40))", 2024-04-20T01:48:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.711403+0000", 2024-04-20T01:48:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.950413775000001, 2024-04-20T01:48:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0093226750000000008, 2024-04-20T01:48:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711403+0000", 2024-04-20T01:48:16.707 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711403+0000", 2024-04-20T01:48:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711403+0000", 2024-04-20T01:48:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711422+0000", 2024-04-20T01:48:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9680000000000001e-05 2024-04-20T01:48:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711432+0000", 2024-04-20T01:48:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6330000000000008e-06 2024-04-20T01:48:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711440+0000", 2024-04-20T01:48:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1699999999999997e-06 2024-04-20T01:48:16.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711486+0000", 2024-04-20T01:48:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6118000000000001e-05 2024-04-20T01:48:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711496+0000", 2024-04-20T01:48:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9890000000000004e-06 2024-04-20T01:48:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.720699+0000", 2024-04-20T01:48:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092022619999999992 2024-04-20T01:48:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.720726+0000", 2024-04-20T01:48:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7378000000000001e-05 2024-04-20T01:48:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.711 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s4 55/30 ECSubWrite(tid=47, reqid=client.4643.0:462, at_version=55'36, trim_to=0'0, roll_forward_to=48'33))", 2024-04-20T01:48:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:56.740180+0000", 2024-04-20T01:48:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.921637326999999, 2024-04-20T01:48:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0088424850000000006, 2024-04-20T01:48:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740180+0000", 2024-04-20T01:48:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740180+0000", 2024-04-20T01:48:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740179+0000", 2024-04-20T01:48:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.713 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740203+0000", 2024-04-20T01:48:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4267999999999999e-05 2024-04-20T01:48:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740222+0000", 2024-04-20T01:48:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9032e-05 2024-04-20T01:48:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740233+0000", 2024-04-20T01:48:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1381e-05 2024-04-20T01:48:16.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740307+0000", 2024-04-20T01:48:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3559000000000005e-05 2024-04-20T01:48:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.740320+0000", 2024-04-20T01:48:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3339000000000001e-05 2024-04-20T01:48:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.748994+0000", 2024-04-20T01:48:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0086742150000000007 2024-04-20T01:48:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:56.749022+0000", 2024-04-20T01:48:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7746000000000002e-05 2024-04-20T01:48:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s4 63/30 ECSubWrite(tid=295, reqid=client.4643.0:574, at_version=63'79, trim_to=0'0, roll_forward_to=62'77))", 2024-04-20T01:48:16.716 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:04.794956+0000", 2024-04-20T01:48:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 11.866861401, 2024-04-20T01:48:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010386685, 2024-04-20T01:48:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.794956+0000", 2024-04-20T01:48:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.717 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.794956+0000", 2024-04-20T01:48:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.794955+0000", 2024-04-20T01:48:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795372+0000", 2024-04-20T01:48:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00041694200000000001 2024-04-20T01:48:16.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795389+0000", 2024-04-20T01:48:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7224999999999999e-05 2024-04-20T01:48:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795400+0000", 2024-04-20T01:48:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0284e-05 2024-04-20T01:48:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795488+0000", 2024-04-20T01:48:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8015000000000003e-05 2024-04-20T01:48:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795503+0000", 2024-04-20T01:48:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5713999999999999e-05 2024-04-20T01:48:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.805304+0000", 2024-04-20T01:48:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0098005669999999996 2024-04-20T01:48:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.805342+0000", 2024-04-20T01:48:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8411000000000002e-05 2024-04-20T01:48:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:693 3.cs0 3:3b24310b:::smithi16579500-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.023681+0000", 2024-04-20T01:48:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.6381355989999999, 2024-04-20T01:48:16.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011501219999999999, 2024-04-20T01:48:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 693 2024-04-20T01:48:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.023681+0000", 2024-04-20T01:48:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.023681+0000", 2024-04-20T01:48:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.023681+0000", 2024-04-20T01:48:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:48:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.723 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.023683+0000", 2024-04-20T01:48:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0229999999999999e-06 2024-04-20T01:48:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.023688+0000", 2024-04-20T01:48:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3320000000000002e-06 2024-04-20T01:48:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.023690+0000", 2024-04-20T01:48:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8049999999999998e-06 2024-04-20T01:48:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.026043+0000", 2024-04-20T01:48:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023528910000000002 2024-04-20T01:48:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.026092+0000", 2024-04-20T01:48:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8772e-05 2024-04-20T01:48:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.031230+0000", 2024-04-20T01:48:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051380499999999999 2024-04-20T01:48:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.031335+0000", 2024-04-20T01:48:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105377 2024-04-20T01:48:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.035183+0000", 2024-04-20T01:48:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038471920000000001 2024-04-20T01:48:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:695 3.cs0 3:3fb6e452:::smithi16579500-47:head [read 0~3031040,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.023709+0000", 2024-04-20T01:48:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.6381079569999999, 2024-04-20T01:48:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0091659710000000002, 2024-04-20T01:48:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 695 2024-04-20T01:48:16.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.023709+0000", 2024-04-20T01:48:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.023709+0000", 2024-04-20T01:48:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.728 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.023709+0000", 2024-04-20T01:48:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.023711+0000", 2024-04-20T01:48:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7630000000000001e-06 2024-04-20T01:48:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.023715+0000", 2024-04-20T01:48:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9550000000000002e-06 2024-04-20T01:48:16.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.023717+0000", 2024-04-20T01:48:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6129999999999998e-06 2024-04-20T01:48:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.026843+0000", 2024-04-20T01:48:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031254479999999999 2024-04-20T01:48:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.026902+0000", 2024-04-20T01:48:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9485999999999999e-05 2024-04-20T01:48:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.032875+0000", 2024-04-20T01:48:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059729479999999996 2024-04-20T01:48:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:696 3.cs0 3:3fb6e452:::smithi16579500-47:head [read 0~3031040,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.023719+0000", 2024-04-20T01:48:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.6380978099999997, 2024-04-20T01:48:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010753235999999999, 2024-04-20T01:48:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 696 2024-04-20T01:48:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.023719+0000", 2024-04-20T01:48:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.023719+0000", 2024-04-20T01:48:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.023719+0000", 2024-04-20T01:48:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:48:16.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.023721+0000", 2024-04-20T01:48:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5629999999999999e-06 2024-04-20T01:48:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.023724+0000", 2024-04-20T01:48:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7869999999999999e-06 2024-04-20T01:48:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.023727+0000", 2024-04-20T01:48:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7319999999999998e-06 2024-04-20T01:48:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.027540+0000", 2024-04-20T01:48:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038128229999999999 2024-04-20T01:48:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.027639+0000", 2024-04-20T01:48:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9079999999999993e-05 2024-04-20T01:48:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.034472+0000", 2024-04-20T01:48:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068334609999999999 2024-04-20T01:48:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:737 3.cs0 3:3b24310b:::smithi16579500-50:head [append 0~81920] snapc 26=[26,25,1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-20T01:48:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:11.858037+0000", 2024-04-20T01:48:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.8037797849999997, 2024-04-20T01:48:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0086784529999999992, 2024-04-20T01:48:16.736 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 737 2024-04-20T01:48:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.858037+0000", 2024-04-20T01:48:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.858037+0000", 2024-04-20T01:48:16.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.858037+0000", 2024-04-20T01:48:16.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.858082+0000", 2024-04-20T01:48:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5754000000000003e-05 2024-04-20T01:48:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.858089+0000", 2024-04-20T01:48:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2219999999999999e-06 2024-04-20T01:48:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.858093+0000", 2024-04-20T01:48:16.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3259999999999997e-06 2024-04-20T01:48:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.860023+0000", 2024-04-20T01:48:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019302259999999999 2024-04-20T01:48:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.860112+0000", 2024-04-20T01:48:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9003000000000001e-05 2024-04-20T01:48:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.860656+0000", 2024-04-20T01:48:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00054367499999999997 2024-04-20T01:48:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.863955+0000", 2024-04-20T01:48:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032994309999999998 2024-04-20T01:48:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.866668+0000", 2024-04-20T01:48:16.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002712604 2024-04-20T01:48:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.866716+0000", 2024-04-20T01:48:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7794999999999999e-05 2024-04-20T01:48:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:738 3.cs0 3:3b24310b:::smithi16579500-50:head [setxattr _header (57) in=7b] snapc 26=[26,25,1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-20T01:48:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:11.858097+0000", 2024-04-20T01:48:16.742 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.8037195779999999, 2024-04-20T01:48:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011560422000000001, 2024-04-20T01:48:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 738 2024-04-20T01:48:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.858097+0000", 2024-04-20T01:48:16.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.858097+0000", 2024-04-20T01:48:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.858097+0000", 2024-04-20T01:48:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.858100+0000", 2024-04-20T01:48:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8509999999999999e-06 2024-04-20T01:48:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.858103+0000", 2024-04-20T01:48:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1820000000000002e-06 2024-04-20T01:48:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.858112+0000", 2024-04-20T01:48:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5200000000000003e-06 2024-04-20T01:48:16.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861431+0000", 2024-04-20T01:48:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003318648 2024-04-20T01:48:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861526+0000", 2024-04-20T01:48:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4765000000000005e-05 2024-04-20T01:48:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:16.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861898+0000", 2024-04-20T01:48:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037221699999999999 2024-04-20T01:48:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.866450+0000", 2024-04-20T01:48:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045523769999999998 2024-04-20T01:48:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.869597+0000", 2024-04-20T01:48:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031461810000000001 2024-04-20T01:48:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.869658+0000", 2024-04-20T01:48:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1330999999999999e-05 2024-04-20T01:48:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:739 3.cs0 3:3b24310b:::smithi16579500-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e70)", 2024-04-20T01:48:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:11.858115+0000", 2024-04-20T01:48:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.8037016079999999, 2024-04-20T01:48:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012921861, 2024-04-20T01:48:16.748 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 739 2024-04-20T01:48:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.858115+0000", 2024-04-20T01:48:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.858115+0000", 2024-04-20T01:48:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.858115+0000", 2024-04-20T01:48:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.858117+0000", 2024-04-20T01:48:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0329999999999998e-06 2024-04-20T01:48:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.858121+0000", 2024-04-20T01:48:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8519999999999997e-06 2024-04-20T01:48:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.858125+0000", 2024-04-20T01:48:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3039999999999998e-06 2024-04-20T01:48:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.862157+0000", 2024-04-20T01:48:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040315259999999997 2024-04-20T01:48:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.862223+0000", 2024-04-20T01:48:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6489000000000007e-05 2024-04-20T01:48:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.869981+0000", 2024-04-20T01:48:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077577119999999999 2024-04-20T01:48:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.870048+0000", 2024-04-20T01:48:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6649999999999994e-05 2024-04-20T01:48:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.871037+0000", 2024-04-20T01:48:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00098958500000000007 2024-04-20T01:48:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:16.753 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:16.753 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:16.754 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-20T01:48:16.757 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-20T01:48:16.758 INFO:tasks.rados.rados.0.smithi165.stdout:301: done (0 left) 2024-04-20T01:48:16.758 INFO:tasks.rados.rados.0.smithi165.stdout:302: copy_from oid 30 from oid 29 current snap is 22 2024-04-20T01:48:16.758 INFO:tasks.rados.rados.0.smithi165.stdout:303: setattr oid 31 current snap is 22 2024-04-20T01:48:16.758 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 47 current snap is 22 2024-04-20T01:48:16.758 INFO:tasks.rados.rados.0.smithi165.stdout:304: seq_num 135 ranges {3031040=720896,3751936=622592,4374528=688128,5062656=245760} 2024-04-20T01:48:16.758 INFO:tasks.rados.rados.0.smithi165.stdout:304: writing smithi16579500-47 from 3031040 to 3751936 tid 1 2024-04-20T01:48:16.759 INFO:tasks.rados.rados.0.smithi165.stdout:304: writing smithi16579500-47 from 3751936 to 4374528 tid 2 2024-04-20T01:48:16.759 INFO:tasks.rados.rados.0.smithi165.stdout:304: writing smithi16579500-47 from 4374528 to 5062656 tid 3 2024-04-20T01:48:16.759 INFO:tasks.rados.rados.0.smithi165.stdout:304: writing smithi16579500-47 from 5062656 to 5308416 tid 4 2024-04-20T01:48:16.759 INFO:tasks.rados.rados.0.smithi165.stdout:305: copy_from oid 6 from oid 41 current snap is 22 2024-04-20T01:48:16.759 INFO:tasks.rados.rados.0.smithi165.stdout:302: finishing copy_from racing read to smithi16579500-30 2024-04-20T01:48:16.759 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 31 v 137 (ObjNum 127 snap 18 seq_num 127) dirty exists 2024-04-20T01:48:16.759 INFO:tasks.rados.rados.0.smithi165.stdout:303: done (3 left) 2024-04-20T01:48:16.759 INFO:tasks.rados.rados.0.smithi165.stdout:306: rollback oid 21 current snap is 22 2024-04-20T01:48:16.759 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 21 to 20 2024-04-20T01:48:16.759 INFO:tasks.rados.rados.0.smithi165.stdout:307: read oid 23 snap -1 2024-04-20T01:48:16.759 INFO:tasks.rados.rados.0.smithi165.stdout:307: expect (ObjNum 22 snap 0 seq_num 22) 2024-04-20T01:48:16.759 INFO:tasks.rados.rados.0.smithi165.stdout:308: setattr oid 17 current snap is 22 2024-04-20T01:48:16.759 INFO:tasks.rados.rados.0.smithi165.stdout:302: finishing copy_from to smithi16579500-30 2024-04-20T01:48:16.760 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 30 v 10 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-04-20T01:48:16.760 INFO:tasks.rados.rados.0.smithi165.stdout:304: finishing write tid 1 to smithi16579500-47 2024-04-20T01:48:16.760 INFO:tasks.rados.rados.0.smithi165.stdout:302: done (5 left) 2024-04-20T01:48:16.760 INFO:tasks.rados.rados.0.smithi165.stdout:309: read oid 8 snap 20 2024-04-20T01:48:16.760 INFO:tasks.rados.rados.0.smithi165.stdout:309: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-20T01:48:16.760 INFO:tasks.rados.rados.0.smithi165.stdout:310: snap_remove snap 21 2024-04-20T01:48:16.760 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:16.768+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:16.856 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:16.871+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:16.893 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:16.894 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:16.894 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-04-20T01:48:16.894 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:16.903 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-20T01:48:16.957 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:16.972+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:17.031 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:17.031 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:17.031 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:48:17.032 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:48:17.032 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:17.042 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-20T01:48:17.085 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:17.099+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:17.158 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:17.173+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:17.171 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:17.171 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:48:17.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:48:17.171 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:48:17.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.172 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:234 3.0s0 3:0c1ed9f1:::smithi16579500-46:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:17.172 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.787453+0000", 2024-04-20T01:48:17.172 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 38.397227512000001, 2024-04-20T01:48:17.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016795529, 2024-04-20T01:48:17.172 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.172 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:17.172 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:17.172 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:17.172 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:17.172 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 234 2024-04-20T01:48:17.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.172 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787453+0000", 2024-04-20T01:48:17.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787453+0000", 2024-04-20T01:48:17.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787451+0000", 2024-04-20T01:48:17.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:17.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787462+0000", 2024-04-20T01:48:17.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1083e-05 2024-04-20T01:48:17.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787483+0000", 2024-04-20T01:48:17.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1656999999999999e-05 2024-04-20T01:48:17.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787503+0000", 2024-04-20T01:48:17.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9623000000000002e-05 2024-04-20T01:48:17.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787595+0000", 2024-04-20T01:48:17.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2417000000000006e-05 2024-04-20T01:48:17.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:17.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787604+0000", 2024-04-20T01:48:17.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.952e-06 2024-04-20T01:48:17.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.791584+0000", 2024-04-20T01:48:17.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039792250000000003 2024-04-20T01:48:17.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:17.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.791699+0000", 2024-04-20T01:48:17.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115692 2024-04-20T01:48:17.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.792530+0000", 2024-04-20T01:48:17.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083077900000000002 2024-04-20T01:48:17.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.798777+0000", 2024-04-20T01:48:17.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062472379999999996 2024-04-20T01:48:17.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:17.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.804175+0000", 2024-04-20T01:48:17.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005397158 2024-04-20T01:48:17.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.804248+0000", 2024-04-20T01:48:17.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3682000000000004e-05 2024-04-20T01:48:17.178 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.178 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.178 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.178 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:235 3.0s0 3:0c1ed9f1:::smithi16579500-46:head [rollback 5] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:17.178 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.787517+0000", 2024-04-20T01:48:17.178 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 38.397163683000002, 2024-04-20T01:48:17.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019202061999999999, 2024-04-20T01:48:17.178 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.178 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:17.179 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:17.179 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:17.179 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:17.179 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 235 2024-04-20T01:48:17.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.179 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787517+0000", 2024-04-20T01:48:17.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787517+0000", 2024-04-20T01:48:17.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787516+0000", 2024-04-20T01:48:17.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:17.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787521+0000", 2024-04-20T01:48:17.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2599999999999996e-06 2024-04-20T01:48:17.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787534+0000", 2024-04-20T01:48:17.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2717e-05 2024-04-20T01:48:17.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787538+0000", 2024-04-20T01:48:17.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3540000000000002e-06 2024-04-20T01:48:17.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787673+0000", 2024-04-20T01:48:17.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013480799999999999 2024-04-20T01:48:17.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:17.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787678+0000", 2024-04-20T01:48:17.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5229999999999997e-06 2024-04-20T01:48:17.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793075+0000", 2024-04-20T01:48:17.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053977549999999997 2024-04-20T01:48:17.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:17.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793190+0000", 2024-04-20T01:48:17.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114775 2024-04-20T01:48:17.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793760+0000", 2024-04-20T01:48:17.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00056945599999999998 2024-04-20T01:48:17.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799198+0000", 2024-04-20T01:48:17.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0054385589999999999 2024-04-20T01:48:17.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:17.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806653+0000", 2024-04-20T01:48:17.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0074547739999999999 2024-04-20T01:48:17.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806719+0000", 2024-04-20T01:48:17.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5732e-05 2024-04-20T01:48:17.184 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.184 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.184 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.184 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:236 3.0s0 3:0c1ed9f1:::smithi16579500-46:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:17.184 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:48:17.184 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 38.397104069000001, 2024-04-20T01:48:17.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019273109, 2024-04-20T01:48:17.184 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.185 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:17.185 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:17.185 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:17.185 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:17.185 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 236 2024-04-20T01:48:17.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.185 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:48:17.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:48:17.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:48:17.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:17.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787579+0000", 2024-04-20T01:48:17.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6380000000000002e-06 2024-04-20T01:48:17.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787586+0000", 2024-04-20T01:48:17.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.81e-06 2024-04-20T01:48:17.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787592+0000", 2024-04-20T01:48:17.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5180000000000004e-06 2024-04-20T01:48:17.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787705+0000", 2024-04-20T01:48:17.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011387000000000001 2024-04-20T01:48:17.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:17.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787712+0000", 2024-04-20T01:48:17.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6229999999999999e-06 2024-04-20T01:48:17.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794765+0000", 2024-04-20T01:48:17.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0070530519999999998 2024-04-20T01:48:17.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:17.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794873+0000", 2024-04-20T01:48:17.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107861 2024-04-20T01:48:17.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795365+0000", 2024-04-20T01:48:17.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049192300000000001 2024-04-20T01:48:17.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799201+0000", 2024-04-20T01:48:17.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003836454 2024-04-20T01:48:17.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:17.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806808+0000", 2024-04-20T01:48:17.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076070119999999998 2024-04-20T01:48:17.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.190 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806849+0000", 2024-04-20T01:48:17.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0852999999999997e-05 2024-04-20T01:48:17.190 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.190 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.190 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.190 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:443 3.0s0 3:06dd530f:::smithi16579500-30:head [append 0~720896] snapc 15=[15,14,13,b] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:17.190 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.714463+0000", 2024-04-20T01:48:17.190 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 22.470216960999998, 2024-04-20T01:48:17.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014549549, 2024-04-20T01:48:17.190 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.190 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:17.190 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:17.191 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:17.191 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:17.191 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 443 2024-04-20T01:48:17.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.191 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.191 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.191 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714463+0000", 2024-04-20T01:48:17.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.191 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.191 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714463+0000", 2024-04-20T01:48:17.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714461+0000", 2024-04-20T01:48:17.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:17.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714916+0000", 2024-04-20T01:48:17.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00045487499999999998 2024-04-20T01:48:17.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714934+0000", 2024-04-20T01:48:17.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7586000000000001e-05 2024-04-20T01:48:17.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714962+0000", 2024-04-20T01:48:17.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7976000000000001e-05 2024-04-20T01:48:17.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.715010+0000", 2024-04-20T01:48:17.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8535999999999997e-05 2024-04-20T01:48:17.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:17.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.715016+0000", 2024-04-20T01:48:17.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8309999999999997e-06 2024-04-20T01:48:17.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717469+0000", 2024-04-20T01:48:17.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024527529999999998 2024-04-20T01:48:17.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:17.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717568+0000", 2024-04-20T01:48:17.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8708000000000004e-05 2024-04-20T01:48:17.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718717+0000", 2024-04-20T01:48:17.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0011493510000000001 2024-04-20T01:48:17.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724443+0000", 2024-04-20T01:48:17.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057254280000000003 2024-04-20T01:48:17.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:17.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.728929+0000", 2024-04-20T01:48:17.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044864919999999999 2024-04-20T01:48:17.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.729013+0000", 2024-04-20T01:48:17.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3770999999999995e-05 2024-04-20T01:48:17.196 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.196 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.196 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.196 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:444 3.0s0 3:06dd530f:::smithi16579500-30:head [append 0~442368] snapc 15=[15,14,13,b] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:17.196 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.716986+0000", 2024-04-20T01:48:17.196 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 22.467694696999999, 2024-04-20T01:48:17.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014745457999999999, 2024-04-20T01:48:17.196 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.196 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:17.196 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:17.196 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:17.196 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:17.196 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 444 2024-04-20T01:48:17.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.197 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.197 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.197 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.716986+0000", 2024-04-20T01:48:17.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.197 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.197 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.716986+0000", 2024-04-20T01:48:17.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.197 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.716984+0000", 2024-04-20T01:48:17.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:17.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.198 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717309+0000", 2024-04-20T01:48:17.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032479500000000001 2024-04-20T01:48:17.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.198 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717321+0000", 2024-04-20T01:48:17.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2177e-05 2024-04-20T01:48:17.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.199 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717330+0000", 2024-04-20T01:48:17.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0399999999999998e-06 2024-04-20T01:48:17.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.199 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.199 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.720236+0000", 2024-04-20T01:48:17.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002905992 2024-04-20T01:48:17.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.199 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:17.199 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.720367+0000", 2024-04-20T01:48:17.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000130488 2024-04-20T01:48:17.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.199 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721328+0000", 2024-04-20T01:48:17.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00096118699999999996 2024-04-20T01:48:17.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.729125+0000", 2024-04-20T01:48:17.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077972429999999997 2024-04-20T01:48:17.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:17.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731687+0000", 2024-04-20T01:48:17.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00256201 2024-04-20T01:48:17.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731731+0000", 2024-04-20T01:48:17.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4005999999999998e-05 2024-04-20T01:48:17.201 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.201 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.201 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.201 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:445 3.0s0 3:06dd530f:::smithi16579500-30:head [append 0~327680] snapc 15=[15,14,13,b] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:17.201 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.718952+0000", 2024-04-20T01:48:17.201 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 22.465728033000001, 2024-04-20T01:48:17.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012930084, 2024-04-20T01:48:17.201 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.201 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:17.202 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:17.202 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:17.202 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:17.202 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 445 2024-04-20T01:48:17.202 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.202 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.202 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.202 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.202 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718952+0000", 2024-04-20T01:48:17.202 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.202 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.202 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.202 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.202 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718952+0000", 2024-04-20T01:48:17.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.203 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.203 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718950+0000", 2024-04-20T01:48:17.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:17.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.203 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.203 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719225+0000", 2024-04-20T01:48:17.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00027498300000000002 2024-04-20T01:48:17.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.203 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.203 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719241+0000", 2024-04-20T01:48:17.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6036999999999999e-05 2024-04-20T01:48:17.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719251+0000", 2024-04-20T01:48:17.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.027e-05 2024-04-20T01:48:17.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722012+0000", 2024-04-20T01:48:17.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027604309999999998 2024-04-20T01:48:17.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:17.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722121+0000", 2024-04-20T01:48:17.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109207 2024-04-20T01:48:17.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722804+0000", 2024-04-20T01:48:17.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00068250299999999999 2024-04-20T01:48:17.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.729128+0000", 2024-04-20T01:48:17.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0063243079999999998 2024-04-20T01:48:17.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:17.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731817+0000", 2024-04-20T01:48:17.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0026888239999999998 2024-04-20T01:48:17.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731882+0000", 2024-04-20T01:48:17.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5499000000000004e-05 2024-04-20T01:48:17.206 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.206 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.206 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.206 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:446 3.0s0 3:06dd530f:::smithi16579500-30:head [setxattr _header (57) in=7b] snapc 15=[15,14,13,b] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:17.206 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.719261+0000", 2024-04-20T01:48:17.207 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 22.465419696000001, 2024-04-20T01:48:17.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012992831, 2024-04-20T01:48:17.207 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.207 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:17.207 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:17.207 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:17.207 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:17.207 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 446 2024-04-20T01:48:17.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.207 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.207 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.207 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719261+0000", 2024-04-20T01:48:17.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719261+0000", 2024-04-20T01:48:17.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719260+0000", 2024-04-20T01:48:17.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:17.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719264+0000", 2024-04-20T01:48:17.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6179999999999997e-06 2024-04-20T01:48:17.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719271+0000", 2024-04-20T01:48:17.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9949999999999999e-06 2024-04-20T01:48:17.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719277+0000", 2024-04-20T01:48:17.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6999999999999996e-06 2024-04-20T01:48:17.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723396+0000", 2024-04-20T01:48:17.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0041186599999999997 2024-04-20T01:48:17.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.210 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:17.210 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723458+0000", 2024-04-20T01:48:17.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2786000000000005e-05 2024-04-20T01:48:17.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.210 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.210 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723867+0000", 2024-04-20T01:48:17.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040805300000000002 2024-04-20T01:48:17.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.210 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.210 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731913+0000", 2024-04-20T01:48:17.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080461330000000005 2024-04-20T01:48:17.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:17.211 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731951+0000", 2024-04-20T01:48:17.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8707000000000001e-05 2024-04-20T01:48:17.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.211 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.732253+0000", 2024-04-20T01:48:17.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030198999999999997 2024-04-20T01:48:17.211 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.211 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.211 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.212 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:447 3.0s0 3:06dd530f:::smithi16579500-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:17.212 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.719282+0000", 2024-04-20T01:48:17.212 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 22.465398129, 2024-04-20T01:48:17.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01483317, 2024-04-20T01:48:17.212 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.212 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:17.212 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:17.212 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:17.212 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:17.212 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 447 2024-04-20T01:48:17.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.212 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719282+0000", 2024-04-20T01:48:17.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719282+0000", 2024-04-20T01:48:17.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719282+0000", 2024-04-20T01:48:17.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:17.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.214 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719294+0000", 2024-04-20T01:48:17.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2747e-05 2024-04-20T01:48:17.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.214 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719302+0000", 2024-04-20T01:48:17.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0960000000000001e-06 2024-04-20T01:48:17.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.214 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719308+0000", 2024-04-20T01:48:17.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.066e-06 2024-04-20T01:48:17.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724217+0000", 2024-04-20T01:48:17.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0049083549999999997 2024-04-20T01:48:17.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:17.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724333+0000", 2024-04-20T01:48:17.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011662099999999999 2024-04-20T01:48:17.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.732307+0000", 2024-04-20T01:48:17.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0079733059999999994 2024-04-20T01:48:17.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:17.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.732409+0000", 2024-04-20T01:48:17.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010193599999999999 2024-04-20T01:48:17.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.734115+0000", 2024-04-20T01:48:17.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0017065520000000001 2024-04-20T01:48:17.216 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.216 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.216 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.216 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:781 3.0s0 3:06dd530f:::smithi16579500-30:head [copy-from ver 10] snapc 2a=[2a,28] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-04-20T01:48:17.217 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.717982+0000", 2024-04-20T01:48:17.217 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46669836799999997, 2024-04-20T01:48:17.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.036464862000000001, 2024-04-20T01:48:17.217 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.217 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:17.217 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:17.217 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:17.217 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:17.217 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 781 2024-04-20T01:48:17.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.217 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.717982+0000", 2024-04-20T01:48:17.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.717982+0000", 2024-04-20T01:48:17.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.717980+0000", 2024-04-20T01:48:17.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:17.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.717990+0000", 2024-04-20T01:48:17.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9790000000000005e-06 2024-04-20T01:48:17.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718003+0000", 2024-04-20T01:48:17.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.342e-05 2024-04-20T01:48:17.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718014+0000", 2024-04-20T01:48:17.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0258000000000001e-05 2024-04-20T01:48:17.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718122+0000", 2024-04-20T01:48:17.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108208 2024-04-20T01:48:17.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.220 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:17.220 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718130+0000", 2024-04-20T01:48:17.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2120000000000003e-06 2024-04-20T01:48:17.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.220 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.220 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722371+0000", 2024-04-20T01:48:17.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042406049999999997 2024-04-20T01:48:17.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.220 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:17.220 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722507+0000", 2024-04-20T01:48:17.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000136136 2024-04-20T01:48:17.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.734773+0000", 2024-04-20T01:48:17.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012266325999999999 2024-04-20T01:48:17.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.742440+0000", 2024-04-20T01:48:17.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076668489999999999 2024-04-20T01:48:17.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:17.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.754352+0000", 2024-04-20T01:48:17.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011911880999999999 2024-04-20T01:48:17.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.754447+0000", 2024-04-20T01:48:17.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4827999999999994e-05 2024-04-20T01:48:17.222 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.222 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.223 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.223 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:782 3.0s0 3:06dd530f:::smithi16579500-30:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e75)", 2024-04-20T01:48:17.223 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.718032+0000", 2024-04-20T01:48:17.223 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46664837199999998, 2024-04-20T01:48:17.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020311961999999999, 2024-04-20T01:48:17.223 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.223 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:17.223 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:17.223 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:17.223 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:17.223 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 782 2024-04-20T01:48:17.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.223 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718032+0000", 2024-04-20T01:48:17.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718032+0000", 2024-04-20T01:48:17.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718031+0000", 2024-04-20T01:48:17.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:17.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718038+0000", 2024-04-20T01:48:17.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0990000000000001e-06 2024-04-20T01:48:17.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718047+0000", 2024-04-20T01:48:17.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2520000000000005e-06 2024-04-20T01:48:17.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718059+0000", 2024-04-20T01:48:17.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1965e-05 2024-04-20T01:48:17.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718210+0000", 2024-04-20T01:48:17.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150519 2024-04-20T01:48:17.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:17.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718216+0000", 2024-04-20T01:48:17.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3539999999999996e-06 2024-04-20T01:48:17.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722660+0000", 2024-04-20T01:48:17.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.004443687 2024-04-20T01:48:17.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:17.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722692+0000", 2024-04-20T01:48:17.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1761000000000003e-05 2024-04-20T01:48:17.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.738085+0000", 2024-04-20T01:48:17.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015393333 2024-04-20T01:48:17.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:17.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.738203+0000", 2024-04-20T01:48:17.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011827100000000001 2024-04-20T01:48:17.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.738344+0000", 2024-04-20T01:48:17.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014056600000000001 2024-04-20T01:48:17.228 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.228 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.228 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.228 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs1 75/30 ECSubWrite(tid=65, reqid=osd.2.0:65, at_version=75'50, trim_to=0'0, roll_forward_to=73'46))", 2024-04-20T01:48:17.229 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.720810+0000", 2024-04-20T01:48:17.229 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46387003300000001, 2024-04-20T01:48:17.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013232499, 2024-04-20T01:48:17.229 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.229 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.229 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720810+0000", 2024-04-20T01:48:17.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720810+0000", 2024-04-20T01:48:17.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720809+0000", 2024-04-20T01:48:17.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:17.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720840+0000", 2024-04-20T01:48:17.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0954999999999998e-05 2024-04-20T01:48:17.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720853+0000", 2024-04-20T01:48:17.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2748000000000001e-05 2024-04-20T01:48:17.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720863+0000", 2024-04-20T01:48:17.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4409999999999999e-06 2024-04-20T01:48:17.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722832+0000", 2024-04-20T01:48:17.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019689899999999999 2024-04-20T01:48:17.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722841+0000", 2024-04-20T01:48:17.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8320000000000001e-06 2024-04-20T01:48:17.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.734017+0000", 2024-04-20T01:48:17.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011175314 2024-04-20T01:48:17.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.734043+0000", 2024-04-20T01:48:17.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5962999999999999e-05 2024-04-20T01:48:17.232 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.232 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.233 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.233 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s1 75/30 ECSubWrite(tid=351, reqid=osd.13.0:351, at_version=75'121, trim_to=0'0, roll_forward_to=73'119))", 2024-04-20T01:48:17.233 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.721483+0000", 2024-04-20T01:48:17.233 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.463197523, 2024-04-20T01:48:17.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012358233, 2024-04-20T01:48:17.233 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.233 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.233 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721483+0000", 2024-04-20T01:48:17.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721483+0000", 2024-04-20T01:48:17.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721482+0000", 2024-04-20T01:48:17.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:17.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721517+0000", 2024-04-20T01:48:17.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4520999999999997e-05 2024-04-20T01:48:17.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721530+0000", 2024-04-20T01:48:17.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2928000000000001e-05 2024-04-20T01:48:17.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721540+0000", 2024-04-20T01:48:17.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0276e-05 2024-04-20T01:48:17.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721654+0000", 2024-04-20T01:48:17.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113496 2024-04-20T01:48:17.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721665+0000", 2024-04-20T01:48:17.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1796e-05 2024-04-20T01:48:17.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.733806+0000", 2024-04-20T01:48:17.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012140656999999999 2024-04-20T01:48:17.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.733841+0000", 2024-04-20T01:48:17.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4820999999999997e-05 2024-04-20T01:48:17.237 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.237 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.237 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.237 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s1 75/30 ECSubWrite(tid=352, reqid=osd.13.0:352, at_version=75'123, trim_to=0'0, roll_forward_to=73'119))", 2024-04-20T01:48:17.237 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.723743+0000", 2024-04-20T01:48:17.237 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.460936862, 2024-04-20T01:48:17.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013134636999999999, 2024-04-20T01:48:17.237 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.237 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.237 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723743+0000", 2024-04-20T01:48:17.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723743+0000", 2024-04-20T01:48:17.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723742+0000", 2024-04-20T01:48:17.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:17.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723769+0000", 2024-04-20T01:48:17.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6086999999999999e-05 2024-04-20T01:48:17.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723782+0000", 2024-04-20T01:48:17.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3675e-05 2024-04-20T01:48:17.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723794+0000", 2024-04-20T01:48:17.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1965e-05 2024-04-20T01:48:17.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.240 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.240 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.240 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725734+0000", 2024-04-20T01:48:17.240 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019396439999999999 2024-04-20T01:48:17.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.240 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.240 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.240 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725743+0000", 2024-04-20T01:48:17.240 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8659999999999995e-06 2024-04-20T01:48:17.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.240 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.240 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.240 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736860+0000", 2024-04-20T01:48:17.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011117311 2024-04-20T01:48:17.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736878+0000", 2024-04-20T01:48:17.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7938999999999999e-05 2024-04-20T01:48:17.241 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.241 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.241 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.241 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs3 75/30 ECSubWrite(tid=112, reqid=osd.14.0:112, at_version=75'91, trim_to=0'0, roll_forward_to=74'89))", 2024-04-20T01:48:17.241 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.724406+0000", 2024-04-20T01:48:17.241 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46027417399999998, 2024-04-20T01:48:17.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012721514999999999, 2024-04-20T01:48:17.242 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.242 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.242 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724406+0000", 2024-04-20T01:48:17.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724406+0000", 2024-04-20T01:48:17.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724406+0000", 2024-04-20T01:48:17.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:17.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724444+0000", 2024-04-20T01:48:17.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8659999999999999e-05 2024-04-20T01:48:17.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724455+0000", 2024-04-20T01:48:17.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0468999999999999e-05 2024-04-20T01:48:17.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724469+0000", 2024-04-20T01:48:17.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4498000000000001e-05 2024-04-20T01:48:17.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724894+0000", 2024-04-20T01:48:17.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00042428799999999999 2024-04-20T01:48:17.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724905+0000", 2024-04-20T01:48:17.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1415e-05 2024-04-20T01:48:17.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.737103+0000", 2024-04-20T01:48:17.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012198293000000001 2024-04-20T01:48:17.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.737128+0000", 2024-04-20T01:48:17.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4178000000000001e-05 2024-04-20T01:48:17.245 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.245 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.245 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.246 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs3 75/30 ECSubWrite(tid=113, reqid=osd.14.0:113, at_version=75'93, trim_to=0'0, roll_forward_to=74'89))", 2024-04-20T01:48:17.246 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.724486+0000", 2024-04-20T01:48:17.246 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46019448099999999, 2024-04-20T01:48:17.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012679546999999999, 2024-04-20T01:48:17.246 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.246 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.246 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724486+0000", 2024-04-20T01:48:17.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724486+0000", 2024-04-20T01:48:17.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724485+0000", 2024-04-20T01:48:17.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:17.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724545+0000", 2024-04-20T01:48:17.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9892000000000002e-05 2024-04-20T01:48:17.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724556+0000", 2024-04-20T01:48:17.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0706999999999999e-05 2024-04-20T01:48:17.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724564+0000", 2024-04-20T01:48:17.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5129999999999996e-06 2024-04-20T01:48:17.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725568+0000", 2024-04-20T01:48:17.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001003547 2024-04-20T01:48:17.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725593+0000", 2024-04-20T01:48:17.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4604e-05 2024-04-20T01:48:17.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.737155+0000", 2024-04-20T01:48:17.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011562915 2024-04-20T01:48:17.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.737165+0000", 2024-04-20T01:48:17.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7909999999999999e-06 2024-04-20T01:48:17.249 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.249 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.249 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.250 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s3 75/30 ECSubWrite(tid=101, reqid=osd.5.0:101, at_version=75'82, trim_to=0'0, roll_forward_to=68'78))", 2024-04-20T01:48:17.250 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.724669+0000", 2024-04-20T01:48:17.250 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46001154700000002, 2024-04-20T01:48:17.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012107346, 2024-04-20T01:48:17.250 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.250 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.250 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.250 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.250 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724669+0000", 2024-04-20T01:48:17.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724669+0000", 2024-04-20T01:48:17.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724668+0000", 2024-04-20T01:48:17.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:17.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724693+0000", 2024-04-20T01:48:17.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4289000000000001e-05 2024-04-20T01:48:17.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724707+0000", 2024-04-20T01:48:17.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4489e-05 2024-04-20T01:48:17.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724715+0000", 2024-04-20T01:48:17.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.174e-06 2024-04-20T01:48:17.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725729+0000", 2024-04-20T01:48:17.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0010136870000000001 2024-04-20T01:48:17.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725739+0000", 2024-04-20T01:48:17.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0064000000000001e-05 2024-04-20T01:48:17.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736749+0000", 2024-04-20T01:48:17.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011009778 2024-04-20T01:48:17.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736776+0000", 2024-04-20T01:48:17.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7220999999999999e-05 2024-04-20T01:48:17.254 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.254 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.254 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.254 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.es5 75/30 ECSubWrite(tid=101, reqid=osd.11.0:101, at_version=75'78, trim_to=0'0, roll_forward_to=73'74))", 2024-04-20T01:48:17.254 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.726511+0000", 2024-04-20T01:48:17.254 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45816933900000001, 2024-04-20T01:48:17.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010295505, 2024-04-20T01:48:17.254 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.254 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.254 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.255 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726511+0000", 2024-04-20T01:48:17.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.255 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726511+0000", 2024-04-20T01:48:17.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.255 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726510+0000", 2024-04-20T01:48:17.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:17.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726539+0000", 2024-04-20T01:48:17.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9207999999999999e-05 2024-04-20T01:48:17.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726553+0000", 2024-04-20T01:48:17.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3706000000000001e-05 2024-04-20T01:48:17.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726562+0000", 2024-04-20T01:48:17.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7739999999999999e-06 2024-04-20T01:48:17.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726620+0000", 2024-04-20T01:48:17.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7071000000000003e-05 2024-04-20T01:48:17.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726630+0000", 2024-04-20T01:48:17.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0781999999999999e-05 2024-04-20T01:48:17.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736789+0000", 2024-04-20T01:48:17.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010158284 2024-04-20T01:48:17.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736806+0000", 2024-04-20T01:48:17.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7759999999999999e-05 2024-04-20T01:48:17.258 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.258 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.258 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.258 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.1s5 75/30 ECSubWrite(tid=253, reqid=osd.4.0:253, at_version=75'37, trim_to=0'0, roll_forward_to=73'35))", 2024-04-20T01:48:17.258 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.726600+0000", 2024-04-20T01:48:17.258 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45807999300000002, 2024-04-20T01:48:17.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010233638, 2024-04-20T01:48:17.259 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.259 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.259 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726600+0000", 2024-04-20T01:48:17.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726600+0000", 2024-04-20T01:48:17.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726600+0000", 2024-04-20T01:48:17.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:17.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726616+0000", 2024-04-20T01:48:17.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6293000000000001e-05 2024-04-20T01:48:17.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726624+0000", 2024-04-20T01:48:17.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.011e-06 2024-04-20T01:48:17.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.261 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726631+0000", 2024-04-20T01:48:17.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6939999999999998e-06 2024-04-20T01:48:17.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.261 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726743+0000", 2024-04-20T01:48:17.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011179300000000001 2024-04-20T01:48:17.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.261 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726768+0000", 2024-04-20T01:48:17.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5007e-05 2024-04-20T01:48:17.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736818+0000", 2024-04-20T01:48:17.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010050211 2024-04-20T01:48:17.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736834+0000", 2024-04-20T01:48:17.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5787e-05 2024-04-20T01:48:17.262 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.262 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.262 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.263 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s5 75/57 ECSubWrite(tid=187, reqid=osd.8.0:187, at_version=75'130, trim_to=0'0, roll_forward_to=75'128))", 2024-04-20T01:48:17.263 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.729669+0000", 2024-04-20T01:48:17.263 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45501139699999998, 2024-04-20T01:48:17.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010590518, 2024-04-20T01:48:17.263 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.263 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.263 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.729669+0000", 2024-04-20T01:48:17.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.729669+0000", 2024-04-20T01:48:17.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.729668+0000", 2024-04-20T01:48:17.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:17.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.729700+0000", 2024-04-20T01:48:17.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2382000000000003e-05 2024-04-20T01:48:17.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.729712+0000", 2024-04-20T01:48:17.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2099000000000001e-05 2024-04-20T01:48:17.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.729722+0000", 2024-04-20T01:48:17.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1020000000000003e-06 2024-04-20T01:48:17.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.730508+0000", 2024-04-20T01:48:17.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00078608300000000003 2024-04-20T01:48:17.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.730517+0000", 2024-04-20T01:48:17.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5300000000000002e-06 2024-04-20T01:48:17.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.740230+0000", 2024-04-20T01:48:17.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097130830000000008 2024-04-20T01:48:17.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.740259+0000", 2024-04-20T01:48:17.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9074e-05 2024-04-20T01:48:17.266 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.266 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.266 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.267 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s5 75/57 ECSubWrite(tid=188, reqid=osd.8.0:188, at_version=75'132, trim_to=0'0, roll_forward_to=75'128))", 2024-04-20T01:48:17.267 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.730596+0000", 2024-04-20T01:48:17.267 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45408429099999997, 2024-04-20T01:48:17.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011737587000000001, 2024-04-20T01:48:17.267 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.267 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.267 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.730596+0000", 2024-04-20T01:48:17.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.730596+0000", 2024-04-20T01:48:17.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.730595+0000", 2024-04-20T01:48:17.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:17.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.730622+0000", 2024-04-20T01:48:17.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6631999999999998e-05 2024-04-20T01:48:17.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.730632+0000", 2024-04-20T01:48:17.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0086e-05 2024-04-20T01:48:17.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.730641+0000", 2024-04-20T01:48:17.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0080000000000008e-06 2024-04-20T01:48:17.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.734756+0000", 2024-04-20T01:48:17.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0041146760000000003 2024-04-20T01:48:17.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.734765+0000", 2024-04-20T01:48:17.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7120000000000006e-06 2024-04-20T01:48:17.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.742310+0000", 2024-04-20T01:48:17.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075449940000000002 2024-04-20T01:48:17.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.742333+0000", 2024-04-20T01:48:17.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3975e-05 2024-04-20T01:48:17.270 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.270 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.271 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.271 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.271 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.271 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:17.272 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-20T01:48:17.273 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:17.286+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:17.373 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:17.387+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:17.442 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:17.442 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:17.442 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-04-20T01:48:17.442 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:17.452 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-20T01:48:17.493 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:48:17.507+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:48:17.575 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:17.590+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:17.620 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:17.620 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:17.620 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:48:17.620 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:48:17.620 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:17.630 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-20T01:48:17.656 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-20T01:48:17.676 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:17.690+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:17.697 INFO:tasks.rados.rados.0.smithi165.stdout:310: done (6 left) 2024-04-20T01:48:17.698 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 48 current snap is 22 2024-04-20T01:48:17.698 INFO:tasks.rados.rados.0.smithi165.stdout:311: seq_num 136 ranges {3964928=737280,4702208=770048,5472256=540672,6012928=245760} 2024-04-20T01:48:17.705 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is set 2024-04-20T01:48:17.723 INFO:tasks.rados.rados.0.smithi165.stdout:311: writing smithi16579500-48 from 3964928 to 4702208 tid 1 2024-04-20T01:48:17.728 INFO:tasks.rados.rados.0.smithi165.stdout:311: writing smithi16579500-48 from 4702208 to 5472256 tid 2 2024-04-20T01:48:17.731 INFO:tasks.rados.rados.0.smithi165.stdout:311: writing smithi16579500-48 from 5472256 to 6012928 tid 3 2024-04-20T01:48:17.733 INFO:tasks.rados.rados.0.smithi165.stdout:311: writing smithi16579500-48 from 6012928 to 6258688 tid 4 2024-04-20T01:48:17.733 INFO:tasks.rados.rados.0.smithi165.stdout:312: snap_create 2024-04-20T01:48:17.733 INFO:tasks.rados.rados.0.smithi165.stdout:304: finishing write tid 2 to smithi16579500-47 2024-04-20T01:48:17.733 INFO:tasks.rados.rados.0.smithi165.stdout:307: expect (ObjNum 22 snap 0 seq_num 22) 2024-04-20T01:48:17.744 INFO:tasks.rados.rados.0.smithi165.stdout:304: finishing write tid 3 to smithi16579500-47 2024-04-20T01:48:17.744 INFO:tasks.rados.rados.0.smithi165.stdout:306: finishing rollback tid 0 to smithi16579500-21 2024-04-20T01:48:17.744 INFO:tasks.rados.rados.0.smithi165.stdout:306: finishing rollback tid 1 to smithi16579500-21 2024-04-20T01:48:17.744 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 17 v 129 (ObjNum 120 snap 18 seq_num 120) dirty exists 2024-04-20T01:48:17.744 INFO:tasks.rados.rados.0.smithi165.stdout:306: finishing rollback tid 2 to smithi16579500-21 2024-04-20T01:48:17.744 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 21 v 41 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-04-20T01:48:17.745 INFO:tasks.rados.rados.0.smithi165.stdout:304: finishing write tid 4 to smithi16579500-47 2024-04-20T01:48:17.745 INFO:tasks.rados.rados.0.smithi165.stdout:304: finishing write tid 5 to smithi16579500-47 2024-04-20T01:48:17.745 INFO:tasks.rados.rados.0.smithi165.stdout:304: finishing write tid 6 to smithi16579500-47 2024-04-20T01:48:17.745 INFO:tasks.rados.rados.0.smithi165.stdout:304: oid 47 updating version 0 to 98 2024-04-20T01:48:17.745 INFO:tasks.rados.rados.0.smithi165.stdout:304: oid 47 updating version 98 to 101 2024-04-20T01:48:17.745 INFO:tasks.rados.rados.0.smithi165.stdout:304: oid 47 version 101 is already newer than 100 2024-04-20T01:48:17.745 INFO:tasks.rados.rados.0.smithi165.stdout:304: oid 47 version 101 is already newer than 97 2024-04-20T01:48:17.745 INFO:tasks.rados.rados.0.smithi165.stdout:304: oid 47 version 101 is already newer than 99 2024-04-20T01:48:17.745 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 47 v 101 (ObjNum 135 snap 22 seq_num 135) dirty exists 2024-04-20T01:48:17.745 INFO:tasks.rados.rados.0.smithi165.stdout:304: left oid 47 (ObjNum 135 snap 22 seq_num 135) 2024-04-20T01:48:17.745 INFO:tasks.rados.rados.0.smithi165.stdout:309: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-20T01:48:17.760 INFO:tasks.rados.rados.0.smithi165.stdout:305: finishing copy_from racing read to smithi16579500-6 2024-04-20T01:48:17.761 INFO:tasks.rados.rados.0.smithi165.stdout:305: finishing copy_from to smithi16579500-6 2024-04-20T01:48:17.761 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 6 v 101 (ObjNum 75 snap 7 seq_num 75) dirty exists 2024-04-20T01:48:17.761 INFO:tasks.rados.rados.0.smithi165.stdout:311: finishing write tid 1 to smithi16579500-48 2024-04-20T01:48:17.761 INFO:tasks.rados.rados.0.smithi165.stdout:311: finishing write tid 2 to smithi16579500-48 2024-04-20T01:48:17.761 INFO:tasks.rados.rados.0.smithi165.stdout:311: finishing write tid 3 to smithi16579500-48 2024-04-20T01:48:17.761 INFO:tasks.rados.rados.0.smithi165.stdout:311: finishing write tid 4 to smithi16579500-48 2024-04-20T01:48:17.761 INFO:tasks.rados.rados.0.smithi165.stdout:311: finishing write tid 5 to smithi16579500-48 2024-04-20T01:48:17.761 INFO:tasks.rados.rados.0.smithi165.stdout:311: finishing write tid 6 to smithi16579500-48 2024-04-20T01:48:17.761 INFO:tasks.rados.rados.0.smithi165.stdout:311: oid 48 updating version 0 to 82 2024-04-20T01:48:17.761 INFO:tasks.rados.rados.0.smithi165.stdout:311: oid 48 version 82 is already newer than 80 2024-04-20T01:48:17.761 INFO:tasks.rados.rados.0.smithi165.stdout:311: oid 48 version 82 is already newer than 81 2024-04-20T01:48:17.761 INFO:tasks.rados.rados.0.smithi165.stdout:311: oid 48 updating version 82 to 84 2024-04-20T01:48:17.761 INFO:tasks.rados.rados.0.smithi165.stdout:311: oid 48 version 84 is already newer than 83 2024-04-20T01:48:17.761 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 48 v 84 (ObjNum 136 snap 22 seq_num 136) dirty exists 2024-04-20T01:48:17.762 INFO:tasks.rados.rados.0.smithi165.stdout:311: left oid 48 (ObjNum 136 snap 22 seq_num 136) 2024-04-20T01:48:17.773 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:17.773 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:48:17.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:48:17.773 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:48:17.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.773 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:289 3.9s0 3:94399c02:::smithi16579500-3:head [copy-from ver 6] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:17.774 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:43.807269+0000", 2024-04-20T01:48:17.774 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 33.979181007999998, 2024-04-20T01:48:17.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016170202000000002, 2024-04-20T01:48:17.774 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.774 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:17.774 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:17.774 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:17.775 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:17.775 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 289 2024-04-20T01:48:17.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.775 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807269+0000", 2024-04-20T01:48:17.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807269+0000", 2024-04-20T01:48:17.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807267+0000", 2024-04-20T01:48:17.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:17.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807280+0000", 2024-04-20T01:48:17.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3227999999999999e-05 2024-04-20T01:48:17.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.777 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807303+0000", 2024-04-20T01:48:17.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2991000000000001e-05 2024-04-20T01:48:17.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.777 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807318+0000", 2024-04-20T01:48:17.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4321999999999999e-05 2024-04-20T01:48:17.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807385+0000", 2024-04-20T01:48:17.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7403000000000005e-05 2024-04-20T01:48:17.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:17.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807394+0000", 2024-04-20T01:48:17.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7010000000000002e-06 2024-04-20T01:48:17.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.809246+0000", 2024-04-20T01:48:17.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0018521 2024-04-20T01:48:17.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:17.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.809314+0000", 2024-04-20T01:48:17.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7818999999999996e-05 2024-04-20T01:48:17.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.817661+0000", 2024-04-20T01:48:17.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0083471710000000005 2024-04-20T01:48:17.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.819085+0000", 2024-04-20T01:48:17.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0014239739999999999 2024-04-20T01:48:17.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:17.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.823350+0000", 2024-04-20T01:48:17.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042652990000000002 2024-04-20T01:48:17.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.823439+0000", 2024-04-20T01:48:17.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9424000000000003e-05 2024-04-20T01:48:17.783 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.783 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.784 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.784 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:290 3.9s0 3:94399c02:::smithi16579500-3:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:17.784 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:43.807329+0000", 2024-04-20T01:48:17.784 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 33.979121087000003, 2024-04-20T01:48:17.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011459461000000001, 2024-04-20T01:48:17.785 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.785 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:17.785 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:17.785 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:17.785 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:17.785 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 290 2024-04-20T01:48:17.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.786 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807329+0000", 2024-04-20T01:48:17.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807329+0000", 2024-04-20T01:48:17.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807329+0000", 2024-04-20T01:48:17.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:17.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807332+0000", 2024-04-20T01:48:17.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.349e-06 2024-04-20T01:48:17.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807343+0000", 2024-04-20T01:48:17.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1270999999999999e-05 2024-04-20T01:48:17.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807347+0000", 2024-04-20T01:48:17.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-04-20T01:48:17.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807453+0000", 2024-04-20T01:48:17.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106162 2024-04-20T01:48:17.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:17.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807460+0000", 2024-04-20T01:48:17.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9809999999999997e-06 2024-04-20T01:48:17.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.809441+0000", 2024-04-20T01:48:17.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019806060000000002 2024-04-20T01:48:17.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:17.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.809484+0000", 2024-04-20T01:48:17.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3019000000000003e-05 2024-04-20T01:48:17.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.818614+0000", 2024-04-20T01:48:17.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091302699999999994 2024-04-20T01:48:17.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:17.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.818713+0000", 2024-04-20T01:48:17.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8776000000000004e-05 2024-04-20T01:48:17.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.818789+0000", 2024-04-20T01:48:17.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5944000000000003e-05 2024-04-20T01:48:17.792 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.792 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.792 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.792 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 47/30 ECSubWrite(tid=38, reqid=osd.14.0:38, at_version=47'31, trim_to=0'0, roll_forward_to=46'29))", 2024-04-20T01:48:17.792 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.711869+0000", 2024-04-20T01:48:17.793 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 29.074581199000001, 2024-04-20T01:48:17.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016896644999999998, 2024-04-20T01:48:17.793 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.793 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.793 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711869+0000", 2024-04-20T01:48:17.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711869+0000", 2024-04-20T01:48:17.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711868+0000", 2024-04-20T01:48:17.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:17.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711907+0000", 2024-04-20T01:48:17.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8828000000000002e-05 2024-04-20T01:48:17.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711927+0000", 2024-04-20T01:48:17.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9656e-05 2024-04-20T01:48:17.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711937+0000", 2024-04-20T01:48:17.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0277e-05 2024-04-20T01:48:17.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711991+0000", 2024-04-20T01:48:17.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3980000000000002e-05 2024-04-20T01:48:17.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.712002+0000", 2024-04-20T01:48:17.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719e-05 2024-04-20T01:48:17.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.728747+0000", 2024-04-20T01:48:17.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016745466000000001 2024-04-20T01:48:17.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.728766+0000", 2024-04-20T01:48:17.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8607999999999999e-05 2024-04-20T01:48:17.797 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.798 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.798 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.798 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 47/30 ECSubWrite(tid=39, reqid=osd.14.0:39, at_version=47'33, trim_to=0'0, roll_forward_to=46'29))", 2024-04-20T01:48:17.798 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.713021+0000", 2024-04-20T01:48:17.798 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 29.073428992, 2024-04-20T01:48:17.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020945089, 2024-04-20T01:48:17.799 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.799 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.799 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713021+0000", 2024-04-20T01:48:17.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713021+0000", 2024-04-20T01:48:17.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713020+0000", 2024-04-20T01:48:17.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:17.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713046+0000", 2024-04-20T01:48:17.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5554e-05 2024-04-20T01:48:17.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713064+0000", 2024-04-20T01:48:17.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7635999999999999e-05 2024-04-20T01:48:17.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713074+0000", 2024-04-20T01:48:17.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9529999999999994e-06 2024-04-20T01:48:17.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713140+0000", 2024-04-20T01:48:17.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6235999999999994e-05 2024-04-20T01:48:17.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713155+0000", 2024-04-20T01:48:17.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5647000000000001e-05 2024-04-20T01:48:17.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733953+0000", 2024-04-20T01:48:17.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020797936 2024-04-20T01:48:17.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733966+0000", 2024-04-20T01:48:17.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3067999999999999e-05 2024-04-20T01:48:17.804 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.804 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.804 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.804 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s1 47/30 ECSubWrite(tid=30, reqid=osd.3.0:30, at_version=47'19, trim_to=0'0, roll_forward_to=44'17))", 2024-04-20T01:48:17.804 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.713663+0000", 2024-04-20T01:48:17.804 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 29.072787232, 2024-04-20T01:48:17.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0202712, 2024-04-20T01:48:17.804 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.804 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.804 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713663+0000", 2024-04-20T01:48:17.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713663+0000", 2024-04-20T01:48:17.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713662+0000", 2024-04-20T01:48:17.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:17.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713693+0000", 2024-04-20T01:48:17.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1031999999999997e-05 2024-04-20T01:48:17.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713712+0000", 2024-04-20T01:48:17.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8615999999999999e-05 2024-04-20T01:48:17.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713726+0000", 2024-04-20T01:48:17.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4477e-05 2024-04-20T01:48:17.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713779+0000", 2024-04-20T01:48:17.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3090999999999997e-05 2024-04-20T01:48:17.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713791+0000", 2024-04-20T01:48:17.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1921e-05 2024-04-20T01:48:17.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733906+0000", 2024-04-20T01:48:17.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020114522999999999 2024-04-20T01:48:17.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733934+0000", 2024-04-20T01:48:17.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8357999999999999e-05 2024-04-20T01:48:17.808 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.809 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.809 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.809 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 47/30 ECSubWrite(tid=39, reqid=osd.5.0:39, at_version=47'27, trim_to=0'0, roll_forward_to=46'25))", 2024-04-20T01:48:17.809 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.718432+0000", 2024-04-20T01:48:17.809 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 29.068018575, 2024-04-20T01:48:17.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015468671, 2024-04-20T01:48:17.809 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.809 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.809 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718432+0000", 2024-04-20T01:48:17.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718432+0000", 2024-04-20T01:48:17.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718431+0000", 2024-04-20T01:48:17.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:17.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718455+0000", 2024-04-20T01:48:17.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3085e-05 2024-04-20T01:48:17.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718471+0000", 2024-04-20T01:48:17.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6172999999999999e-05 2024-04-20T01:48:17.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718480+0000", 2024-04-20T01:48:17.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5719999999999992e-06 2024-04-20T01:48:17.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718592+0000", 2024-04-20T01:48:17.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111517 2024-04-20T01:48:17.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718602+0000", 2024-04-20T01:48:17.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.064e-05 2024-04-20T01:48:17.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733869+0000", 2024-04-20T01:48:17.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015266798 2024-04-20T01:48:17.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733900+0000", 2024-04-20T01:48:17.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.116e-05 2024-04-20T01:48:17.813 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.814 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.814 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.814 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 47/30 ECSubWrite(tid=40, reqid=osd.5.0:40, at_version=47'29, trim_to=0'0, roll_forward_to=46'25))", 2024-04-20T01:48:17.814 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.719277+0000", 2024-04-20T01:48:17.814 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 29.067172879000001, 2024-04-20T01:48:17.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017279452000000001, 2024-04-20T01:48:17.814 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.814 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.814 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719277+0000", 2024-04-20T01:48:17.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719277+0000", 2024-04-20T01:48:17.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719277+0000", 2024-04-20T01:48:17.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:17.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719306+0000", 2024-04-20T01:48:17.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9771999999999999e-05 2024-04-20T01:48:17.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719322+0000", 2024-04-20T01:48:17.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6035000000000001e-05 2024-04-20T01:48:17.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719331+0000", 2024-04-20T01:48:17.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1130000000000007e-06 2024-04-20T01:48:17.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.728810+0000", 2024-04-20T01:48:17.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094789090000000006 2024-04-20T01:48:17.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.728819+0000", 2024-04-20T01:48:17.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8890000000000008e-06 2024-04-20T01:48:17.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.736530+0000", 2024-04-20T01:48:17.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077106129999999998 2024-04-20T01:48:17.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.736557+0000", 2024-04-20T01:48:17.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6974999999999998e-05 2024-04-20T01:48:17.818 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.819 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.819 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.819 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 47/30 ECSubWrite(tid=240, reqid=osd.13.0:240, at_version=47'33, trim_to=0'0, roll_forward_to=44'31))", 2024-04-20T01:48:17.819 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.722777+0000", 2024-04-20T01:48:17.819 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 29.063673165000001, 2024-04-20T01:48:17.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008649894, 2024-04-20T01:48:17.819 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.819 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.820 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.820 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.820 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722777+0000", 2024-04-20T01:48:17.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.820 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.820 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722777+0000", 2024-04-20T01:48:17.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722777+0000", 2024-04-20T01:48:17.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:17.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722805+0000", 2024-04-20T01:48:17.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7881e-05 2024-04-20T01:48:17.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722820+0000", 2024-04-20T01:48:17.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5387e-05 2024-04-20T01:48:17.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.822 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.822 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722831+0000", 2024-04-20T01:48:17.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813e-05 2024-04-20T01:48:17.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722915+0000", 2024-04-20T01:48:17.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3578000000000005e-05 2024-04-20T01:48:17.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722929+0000", 2024-04-20T01:48:17.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4256999999999999e-05 2024-04-20T01:48:17.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.731402+0000", 2024-04-20T01:48:17.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008472785 2024-04-20T01:48:17.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.731427+0000", 2024-04-20T01:48:17.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5355e-05 2024-04-20T01:48:17.825 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.826 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.826 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.826 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 47/30 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=47'35, trim_to=0'0, roll_forward_to=44'31))", 2024-04-20T01:48:17.827 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.723066+0000", 2024-04-20T01:48:17.827 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 29.063384418999998, 2024-04-20T01:48:17.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010874728, 2024-04-20T01:48:17.827 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.827 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.827 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723066+0000", 2024-04-20T01:48:17.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.829 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723066+0000", 2024-04-20T01:48:17.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.829 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.829 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723065+0000", 2024-04-20T01:48:17.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:17.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723089+0000", 2024-04-20T01:48:17.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3529000000000001e-05 2024-04-20T01:48:17.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723101+0000", 2024-04-20T01:48:17.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2143e-05 2024-04-20T01:48:17.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723110+0000", 2024-04-20T01:48:17.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9809999999999992e-06 2024-04-20T01:48:17.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.725642+0000", 2024-04-20T01:48:17.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002532337 2024-04-20T01:48:17.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.725655+0000", 2024-04-20T01:48:17.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2707e-05 2024-04-20T01:48:17.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733911+0000", 2024-04-20T01:48:17.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082559369999999997 2024-04-20T01:48:17.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733941+0000", 2024-04-20T01:48:17.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9798000000000002e-05 2024-04-20T01:48:17.835 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.835 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.835 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.835 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:396 3.9s0 3:9a2fbd15:::smithi16579500-1:head [copy-from ver 32] snapc d=[c,b] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-20T01:48:17.835 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.746088+0000", 2024-04-20T01:48:17.835 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 29.040362241, 2024-04-20T01:48:17.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01355685, 2024-04-20T01:48:17.836 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.836 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:17.836 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:17.836 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:17.836 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:17.836 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 396 2024-04-20T01:48:17.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.836 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746088+0000", 2024-04-20T01:48:17.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746088+0000", 2024-04-20T01:48:17.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746086+0000", 2024-04-20T01:48:17.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:17.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746094+0000", 2024-04-20T01:48:17.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.272e-06 2024-04-20T01:48:17.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746109+0000", 2024-04-20T01:48:17.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5563999999999999e-05 2024-04-20T01:48:17.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746131+0000", 2024-04-20T01:48:17.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1515999999999999e-05 2024-04-20T01:48:17.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746178+0000", 2024-04-20T01:48:17.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7472000000000002e-05 2024-04-20T01:48:17.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:17.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746261+0000", 2024-04-20T01:48:17.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2412999999999999e-05 2024-04-20T01:48:17.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.752312+0000", 2024-04-20T01:48:17.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0060515120000000002 2024-04-20T01:48:17.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.754299+0000", 2024-04-20T01:48:17.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019870690000000002 2024-04-20T01:48:17.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:17.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.759594+0000", 2024-04-20T01:48:17.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0052944070000000001 2024-04-20T01:48:17.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.759645+0000", 2024-04-20T01:48:17.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1236000000000002e-05 2024-04-20T01:48:17.841 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.841 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.841 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.841 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 48/30 ECSubWrite(tid=44, reqid=osd.5.0:44, at_version=48'33, trim_to=0'0, roll_forward_to=47'31))", 2024-04-20T01:48:17.842 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.716311+0000", 2024-04-20T01:48:17.842 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.070138836000002, 2024-04-20T01:48:17.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093701210000000004, 2024-04-20T01:48:17.842 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.842 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.842 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716311+0000", 2024-04-20T01:48:17.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716311+0000", 2024-04-20T01:48:17.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716311+0000", 2024-04-20T01:48:17.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:17.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716336+0000", 2024-04-20T01:48:17.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5216e-05 2024-04-20T01:48:17.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716348+0000", 2024-04-20T01:48:17.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1369000000000001e-05 2024-04-20T01:48:17.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716358+0000", 2024-04-20T01:48:17.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9019999999999994e-06 2024-04-20T01:48:17.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716410+0000", 2024-04-20T01:48:17.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1974000000000001e-05 2024-04-20T01:48:17.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716419+0000", 2024-04-20T01:48:17.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4439999999999997e-06 2024-04-20T01:48:17.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.725652+0000", 2024-04-20T01:48:17.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092323909999999995 2024-04-20T01:48:17.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.725682+0000", 2024-04-20T01:48:17.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0049999999999999e-05 2024-04-20T01:48:17.846 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.846 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.847 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.847 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 48/30 ECSubWrite(tid=256, reqid=client.4643.0:403, at_version=48'46, trim_to=0'0, roll_forward_to=48'44))", 2024-04-20T01:48:17.847 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.789372+0000", 2024-04-20T01:48:17.847 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.997078259999999, 2024-04-20T01:48:17.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086922490000000008, 2024-04-20T01:48:17.847 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.847 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.847 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789372+0000", 2024-04-20T01:48:17.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789372+0000", 2024-04-20T01:48:17.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789371+0000", 2024-04-20T01:48:17.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:17.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789968+0000", 2024-04-20T01:48:17.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00059678300000000004 2024-04-20T01:48:17.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789979+0000", 2024-04-20T01:48:17.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1499e-05 2024-04-20T01:48:17.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789986+0000", 2024-04-20T01:48:17.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2179999999999996e-06 2024-04-20T01:48:17.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.790029+0000", 2024-04-20T01:48:17.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2876000000000001e-05 2024-04-20T01:48:17.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.790036+0000", 2024-04-20T01:48:17.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7870000000000002e-06 2024-04-20T01:48:17.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.798040+0000", 2024-04-20T01:48:17.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080038720000000004 2024-04-20T01:48:17.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.798064+0000", 2024-04-20T01:48:17.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4037999999999999e-05 2024-04-20T01:48:17.853 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.853 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.853 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.853 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 48/30 ECSubWrite(tid=257, reqid=client.4643.0:404, at_version=48'47, trim_to=0'0, roll_forward_to=48'44))", 2024-04-20T01:48:17.853 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.791622+0000", 2024-04-20T01:48:17.853 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.994828274, 2024-04-20T01:48:17.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0088453630000000002, 2024-04-20T01:48:17.853 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.853 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.854 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.791622+0000", 2024-04-20T01:48:17.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.791622+0000", 2024-04-20T01:48:17.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.791621+0000", 2024-04-20T01:48:17.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:17.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792215+0000", 2024-04-20T01:48:17.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000593689 2024-04-20T01:48:17.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792227+0000", 2024-04-20T01:48:17.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2037e-05 2024-04-20T01:48:17.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792236+0000", 2024-04-20T01:48:17.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8240000000000006e-06 2024-04-20T01:48:17.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792538+0000", 2024-04-20T01:48:17.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000301668 2024-04-20T01:48:17.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792549+0000", 2024-04-20T01:48:17.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0915e-05 2024-04-20T01:48:17.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.800443+0000", 2024-04-20T01:48:17.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0078945300000000003 2024-04-20T01:48:17.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.800467+0000", 2024-04-20T01:48:17.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4309000000000001e-05 2024-04-20T01:48:17.858 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.858 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.858 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.858 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s1 53/30 ECSubWrite(tid=33, reqid=client.4643.0:444, at_version=53'22, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:17.858 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.721415+0000", 2024-04-20T01:48:17.858 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 23.065035497, 2024-04-20T01:48:17.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097484879999999996, 2024-04-20T01:48:17.859 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.859 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.859 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721415+0000", 2024-04-20T01:48:17.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721415+0000", 2024-04-20T01:48:17.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721413+0000", 2024-04-20T01:48:17.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:17.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721522+0000", 2024-04-20T01:48:17.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108445 2024-04-20T01:48:17.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721540+0000", 2024-04-20T01:48:17.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8128e-05 2024-04-20T01:48:17.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721555+0000", 2024-04-20T01:48:17.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5228999999999999e-05 2024-04-20T01:48:17.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721639+0000", 2024-04-20T01:48:17.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3803999999999999e-05 2024-04-20T01:48:17.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721650+0000", 2024-04-20T01:48:17.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0702e-05 2024-04-20T01:48:17.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731131+0000", 2024-04-20T01:48:17.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094816929999999994 2024-04-20T01:48:17.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731163+0000", 2024-04-20T01:48:17.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1946999999999998e-05 2024-04-20T01:48:17.863 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.863 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.863 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.863 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 58/30 ECSubWrite(tid=52, reqid=client.4643.0:489, at_version=58'41, trim_to=0'0, roll_forward_to=55'37))", 2024-04-20T01:48:17.863 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:59.769343+0000", 2024-04-20T01:48:17.863 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.017107158999998, 2024-04-20T01:48:17.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094380780000000008, 2024-04-20T01:48:17.863 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.864 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.864 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769343+0000", 2024-04-20T01:48:17.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769343+0000", 2024-04-20T01:48:17.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769343+0000", 2024-04-20T01:48:17.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:17.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769462+0000", 2024-04-20T01:48:17.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011929 2024-04-20T01:48:17.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769474+0000", 2024-04-20T01:48:17.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1857e-05 2024-04-20T01:48:17.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769482+0000", 2024-04-20T01:48:17.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0809999999999994e-06 2024-04-20T01:48:17.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769531+0000", 2024-04-20T01:48:17.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9168000000000001e-05 2024-04-20T01:48:17.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769538+0000", 2024-04-20T01:48:17.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5930000000000002e-06 2024-04-20T01:48:17.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.778749+0000", 2024-04-20T01:48:17.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092115670000000004 2024-04-20T01:48:17.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.778781+0000", 2024-04-20T01:48:17.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2140000000000001e-05 2024-04-20T01:48:17.868 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.868 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.868 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.868 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:733 3.9s0 3:94399c02:::smithi16579500-3:head [rmxattr Pp[amim]nQkdfb][ca]Rhonh_mr`cpapUabYUfb\\ in=40b,rmxattr QP`igcZgYZ`X]j_`ObrUNcR_gQZW^mQs_T`XhhQp in=40b,rmxattr QXqRngrniZSrdNdQaffOsm_g_pYQoedrXoNqT[mn in=40b,rmxattr V[thp]nUrl^Xata^al[oh[d`ddg`p[siljQWZ^\\X in=40b,rmxattr XgcaedbaNamUmoa^d[dhg[arrjeikfUlToSjsbY` in=40b,rmxattr ZcXlR_O_^fXfTQrt`cZfOgVlmhepod\\^\\`socsW` in=40b,rmxattr ][ifNPWgaZd[^RnfpRhWbfS\\^a]m[\\[ienetlUUZ in=40b,rmxattr `U`PSXddYml`lYXX[sSNnPfg`VgWbXsYnftUQ`it in=40b,rmxattr aR in=2b,rmxattr eNsVcjeRcj\\m]Q[VhYdSqNcmPhhR[UPR^ORSk_Vd in=40b,rmxattr eRqQZNtlPRVjcgdjmbSmOsUSjSlU]tSoVjr`^oRN in=40b,rmxattr fVRh^qlag^rjpdaqVk`^r]eR`P[XUlr]]agZjWq[ in=40b,rmxattr frZXQX\\isoq]gmjs`rk]PiQ\\fV[RTNPNYN]YV[mq in=40b,rmxattr h\\bUdlZ]kknhkaaYeTcclaXatghTgbVrhNQblto_ in=40b,rmxattr hgq\\iqsggNPTRPQok in=17b,rmxattr jecnmafmfg__r^\\S_sU]bi]kYaplTcttSSX`\\prR in=40b,rmxattr jmliR in=5b,rmxattr mnhUacnNajpOrNecmPZT[OUj[XaWmU in=30b,rmxattr peUYXiNb_UsOaXnhlX_s_[QTW_SoUaN]Wal`rkao in=40b,rmxattr rZrXsckW]Y`iWQiRn in=17b] snapc 26=[26,25,1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-20T01:48:17.868 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:11.823678+0000", 2024-04-20T01:48:17.869 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.9627727930000001, 2024-04-20T01:48:17.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0098521919999999992, 2024-04-20T01:48:17.869 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.869 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:17.869 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:17.869 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:17.869 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:17.869 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 733 2024-04-20T01:48:17.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.869 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823678+0000", 2024-04-20T01:48:17.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823678+0000", 2024-04-20T01:48:17.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823675+0000", 2024-04-20T01:48:17.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:17.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823691+0000", 2024-04-20T01:48:17.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5954999999999999e-05 2024-04-20T01:48:17.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823715+0000", 2024-04-20T01:48:17.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4113000000000001e-05 2024-04-20T01:48:17.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823730+0000", 2024-04-20T01:48:17.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4694e-05 2024-04-20T01:48:17.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823786+0000", 2024-04-20T01:48:17.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6419000000000003e-05 2024-04-20T01:48:17.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:17.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823913+0000", 2024-04-20T01:48:17.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012677500000000001 2024-04-20T01:48:17.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.824989+0000", 2024-04-20T01:48:17.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0010762199999999999 2024-04-20T01:48:17.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.828014+0000", 2024-04-20T01:48:17.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0030246190000000001 2024-04-20T01:48:17.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:17.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.833442+0000", 2024-04-20T01:48:17.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0054285490000000004 2024-04-20T01:48:17.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.833530+0000", 2024-04-20T01:48:17.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7462999999999999e-05 2024-04-20T01:48:17.875 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.875 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.875 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.875 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 75/30 ECSubWrite(tid=100, reqid=osd.5.0:100, at_version=75'80, trim_to=0'0, roll_forward_to=68'78))", 2024-04-20T01:48:17.875 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.720432+0000", 2024-04-20T01:48:17.875 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.066018047, 2024-04-20T01:48:17.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010779983, 2024-04-20T01:48:17.875 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.875 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.876 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720432+0000", 2024-04-20T01:48:17.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720432+0000", 2024-04-20T01:48:17.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720432+0000", 2024-04-20T01:48:17.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:17.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720457+0000", 2024-04-20T01:48:17.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5272000000000001e-05 2024-04-20T01:48:17.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720469+0000", 2024-04-20T01:48:17.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2193e-05 2024-04-20T01:48:17.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720480+0000", 2024-04-20T01:48:17.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0291e-05 2024-04-20T01:48:17.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723263+0000", 2024-04-20T01:48:17.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002783003 2024-04-20T01:48:17.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723273+0000", 2024-04-20T01:48:17.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.01e-05 2024-04-20T01:48:17.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731180+0000", 2024-04-20T01:48:17.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0079069929999999993 2024-04-20T01:48:17.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731212+0000", 2024-04-20T01:48:17.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2421e-05 2024-04-20T01:48:17.880 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.880 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.880 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.880 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 75/30 ECSubWrite(tid=112, reqid=osd.14.0:112, at_version=75'91, trim_to=0'0, roll_forward_to=74'89))", 2024-04-20T01:48:17.880 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.720882+0000", 2024-04-20T01:48:17.880 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.0655685180000001, 2024-04-20T01:48:17.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010362481999999999, 2024-04-20T01:48:17.880 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.880 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.880 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720882+0000", 2024-04-20T01:48:17.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720882+0000", 2024-04-20T01:48:17.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720881+0000", 2024-04-20T01:48:17.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:17.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720905+0000", 2024-04-20T01:48:17.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3683e-05 2024-04-20T01:48:17.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720917+0000", 2024-04-20T01:48:17.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1651000000000001e-05 2024-04-20T01:48:17.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720926+0000", 2024-04-20T01:48:17.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2040000000000003e-06 2024-04-20T01:48:17.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724280+0000", 2024-04-20T01:48:17.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003354363 2024-04-20T01:48:17.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724291+0000", 2024-04-20T01:48:17.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1004e-05 2024-04-20T01:48:17.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731213+0000", 2024-04-20T01:48:17.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069213670000000003 2024-04-20T01:48:17.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731244+0000", 2024-04-20T01:48:17.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1563000000000003e-05 2024-04-20T01:48:17.885 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.885 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.885 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.885 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 75/30 ECSubWrite(tid=101, reqid=osd.5.0:101, at_version=75'82, trim_to=0'0, roll_forward_to=68'78))", 2024-04-20T01:48:17.886 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.723236+0000", 2024-04-20T01:48:17.886 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.063214146, 2024-04-20T01:48:17.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010789544, 2024-04-20T01:48:17.886 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.886 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.886 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723236+0000", 2024-04-20T01:48:17.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723236+0000", 2024-04-20T01:48:17.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723235+0000", 2024-04-20T01:48:17.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:17.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723261+0000", 2024-04-20T01:48:17.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5381e-05 2024-04-20T01:48:17.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723279+0000", 2024-04-20T01:48:17.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7796e-05 2024-04-20T01:48:17.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723290+0000", 2024-04-20T01:48:17.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1739000000000001e-05 2024-04-20T01:48:17.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726374+0000", 2024-04-20T01:48:17.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003084023 2024-04-20T01:48:17.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726389+0000", 2024-04-20T01:48:17.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5041e-05 2024-04-20T01:48:17.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.733997+0000", 2024-04-20T01:48:17.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076079329999999999 2024-04-20T01:48:17.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.734026+0000", 2024-04-20T01:48:17.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8394e-05 2024-04-20T01:48:17.890 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.890 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.890 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.891 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 75/30 ECSubWrite(tid=113, reqid=osd.14.0:113, at_version=75'93, trim_to=0'0, roll_forward_to=74'89))", 2024-04-20T01:48:17.891 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.724246+0000", 2024-04-20T01:48:17.891 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.062204302, 2024-04-20T01:48:17.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097217950000000001, 2024-04-20T01:48:17.891 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:17.891 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:17.891 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:17.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:17.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724246+0000", 2024-04-20T01:48:17.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:17.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724246+0000", 2024-04-20T01:48:17.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:17.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:17.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724245+0000", 2024-04-20T01:48:17.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:17.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:17.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724270+0000", 2024-04-20T01:48:17.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4536999999999999e-05 2024-04-20T01:48:17.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:17.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724284+0000", 2024-04-20T01:48:17.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4421e-05 2024-04-20T01:48:17.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:17.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724292+0000", 2024-04-20T01:48:17.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6009999999999997e-06 2024-04-20T01:48:17.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:17.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726433+0000", 2024-04-20T01:48:17.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021418399999999999 2024-04-20T01:48:17.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:17.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726444+0000", 2024-04-20T01:48:17.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0671e-05 2024-04-20T01:48:17.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:17.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.733939+0000", 2024-04-20T01:48:17.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0074953249999999997 2024-04-20T01:48:17.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:17.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:17.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:17.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.733968+0000", 2024-04-20T01:48:17.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8368e-05 2024-04-20T01:48:17.895 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.896 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.896 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.896 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:17.896 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:17.896 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:17.898 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:48:17.898 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:48:17.898 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:48:17.899 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-20T01:48:17.899 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:17.911+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:17.901 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:48:17.921 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:48:17.931 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-20T01:48:17.931 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-20T01:48:17.931 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-04-20T01:48:17.932 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.226008 2024-04-20T01:48:17.932 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0.22600759438199503 2024-04-20T01:48:17.942 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:48:17.963 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:48:17.984 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:48:17.998 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:18.013+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:18.004 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:48:18.025 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:48:18.047 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:48:18.068 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:48:18.070 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:18.070 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:18.080 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-20T01:48:18.091 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:48:18.099 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:18.114+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:18.112 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:48:18.133 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:48:18.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:48:18.148+0000 7f3460d66700 0 -- 172.21.15.44:0/1528495011 >> [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] conn(0x7f345c21f510 msgr2=0x7f345c221900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-20T01:48:18.201 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:18.215+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:18.209 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:18.209 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:18.219 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-20T01:48:18.302 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:18.316+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:18.356 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:48:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:48:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:48:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:48:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:48:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:48:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:48:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:48:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:48:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:48:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:48:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:48:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:48:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:48:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:48:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:48:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:48:18.389 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:18.389 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:18.399 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-20T01:48:18.402 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:18.417+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:18.504 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:18.518+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:18.528 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:18.528 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:18.538 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-20T01:48:18.604 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:48:18.618+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:48:18.666 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:18.666 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:18.677 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-20T01:48:18.698 INFO:tasks.rados.rados.0.smithi165.stdout:304: done (7 left) 2024-04-20T01:48:18.698 INFO:tasks.rados.rados.0.smithi165.stdout:305: done (6 left) 2024-04-20T01:48:18.698 INFO:tasks.rados.rados.0.smithi165.stdout:306: done (5 left) 2024-04-20T01:48:18.698 INFO:tasks.rados.rados.0.smithi165.stdout:307: done (4 left) 2024-04-20T01:48:18.698 INFO:tasks.rados.rados.0.smithi165.stdout:308: done (3 left) 2024-04-20T01:48:18.698 INFO:tasks.rados.rados.0.smithi165.stdout:309: done (2 left) 2024-04-20T01:48:18.698 INFO:tasks.rados.rados.0.smithi165.stdout:311: done (1 left) 2024-04-20T01:48:18.698 INFO:tasks.rados.rados.0.smithi165.stdout:312: done (0 left) 2024-04-20T01:48:18.698 INFO:tasks.rados.rados.0.smithi165.stdout:313: setattr oid 45 current snap is 23 2024-04-20T01:48:18.700 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 4 current snap is 23 2024-04-20T01:48:18.701 INFO:tasks.rados.rados.0.smithi165.stdout:314: seq_num 137 ranges {0=753664,753664=786432,1540096=655360,2195456=114688} 2024-04-20T01:48:18.703 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.2 primary-affinity to 0.226008 (8148112) 2024-04-20T01:48:18.704 INFO:tasks.rados.rados.0.smithi165.stdout:314: writing smithi16579500-4 from 0 to 753664 tid 1 2024-04-20T01:48:18.705 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:18.718+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:18.708 INFO:tasks.rados.rados.0.smithi165.stdout:314: writing smithi16579500-4 from 753664 to 1540096 tid 2 2024-04-20T01:48:18.712 INFO:tasks.rados.rados.0.smithi165.stdout:314: writing smithi16579500-4 from 1540096 to 2195456 tid 3 2024-04-20T01:48:18.713 INFO:tasks.rados.rados.0.smithi165.stdout:314: writing smithi16579500-4 from 2195456 to 2310144 tid 4 2024-04-20T01:48:18.713 INFO:tasks.rados.rados.0.smithi165.stdout:315: rollback oid 6 current snap is 23 2024-04-20T01:48:18.713 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 6 to 20 2024-04-20T01:48:18.713 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 45 v 149 (ObjNum 122 snap 18 seq_num 122) dirty exists 2024-04-20T01:48:18.714 INFO:tasks.rados.rados.0.smithi165.stdout:313: done (2 left) 2024-04-20T01:48:18.714 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 42 current snap is 23 2024-04-20T01:48:18.714 INFO:tasks.rados.rados.0.smithi165.stdout:316: seq_num 138 ranges {2834432=770048,3604480=540672,4145152=753664,4898816=262144} 2024-04-20T01:48:18.733 INFO:tasks.rados.rados.0.smithi165.stdout:316: writing smithi16579500-42 from 2834432 to 3604480 tid 1 2024-04-20T01:48:18.736 INFO:tasks.rados.rados.0.smithi165.stdout:316: writing smithi16579500-42 from 3604480 to 4145152 tid 2 2024-04-20T01:48:18.741 INFO:tasks.rados.rados.0.smithi165.stdout:316: writing smithi16579500-42 from 4145152 to 4898816 tid 3 2024-04-20T01:48:18.743 INFO:tasks.rados.rados.0.smithi165.stdout:316: writing smithi16579500-42 from 4898816 to 5160960 tid 4 2024-04-20T01:48:18.743 INFO:tasks.rados.rados.0.smithi165.stdout:314: finishing write tid 1 to smithi16579500-4 2024-04-20T01:48:18.743 INFO:tasks.rados.rados.0.smithi165.stdout:314: finishing write tid 2 to smithi16579500-4 2024-04-20T01:48:18.743 INFO:tasks.rados.rados.0.smithi165.stdout:314: finishing write tid 3 to smithi16579500-4 2024-04-20T01:48:18.743 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 43 current snap is 23 2024-04-20T01:48:18.743 INFO:tasks.rados.rados.0.smithi165.stdout:314: finishing write tid 4 to smithi16579500-4 2024-04-20T01:48:18.744 INFO:tasks.rados.rados.0.smithi165.stdout:314: finishing write tid 5 to smithi16579500-4 2024-04-20T01:48:18.744 INFO:tasks.rados.rados.0.smithi165.stdout:315: finishing rollback tid 0 to smithi16579500-6 2024-04-20T01:48:18.744 INFO:tasks.rados.rados.0.smithi165.stdout:317: seq_num 139 ranges {0=786432,786432=688128,1474560=475136,1949696=393216} 2024-04-20T01:48:18.748 INFO:tasks.rados.rados.0.smithi165.stdout:317: writing smithi16579500-43 from 0 to 786432 tid 1 2024-04-20T01:48:18.752 INFO:tasks.rados.rados.0.smithi165.stdout:317: writing smithi16579500-43 from 786432 to 1474560 tid 2 2024-04-20T01:48:18.755 INFO:tasks.rados.rados.0.smithi165.stdout:317: writing smithi16579500-43 from 1474560 to 1949696 tid 3 2024-04-20T01:48:18.758 INFO:tasks.rados.rados.0.smithi165.stdout:317: writing smithi16579500-43 from 1949696 to 2342912 tid 4 2024-04-20T01:48:18.758 INFO:tasks.rados.rados.0.smithi165.stdout:318: read oid 18 snap -1 2024-04-20T01:48:18.758 INFO:tasks.rados.rados.0.smithi165.stdout:318: expect (ObjNum 133 snap 21 seq_num 133) 2024-04-20T01:48:18.758 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 11 current snap is 23 2024-04-20T01:48:18.758 INFO:tasks.rados.rados.0.smithi165.stdout:319: seq_num 140 ranges {0=802816,802816=425984,1228800=720896,1949696=409600} 2024-04-20T01:48:18.763 INFO:tasks.rados.rados.0.smithi165.stdout:319: writing smithi16579500-11 from 0 to 802816 tid 1 2024-04-20T01:48:18.765 INFO:tasks.rados.rados.0.smithi165.stdout:319: writing smithi16579500-11 from 802816 to 1228800 tid 2 2024-04-20T01:48:18.770 INFO:tasks.rados.rados.0.smithi165.stdout:319: writing smithi16579500-11 from 1228800 to 1949696 tid 3 2024-04-20T01:48:18.773 INFO:tasks.rados.rados.0.smithi165.stdout:319: writing smithi16579500-11 from 1949696 to 2359296 tid 4 2024-04-20T01:48:18.773 INFO:tasks.rados.rados.0.smithi165.stdout:320: copy_from oid 50 from oid 27 current snap is 23 2024-04-20T01:48:18.773 INFO:tasks.rados.rados.0.smithi165.stdout:321: read oid 33 snap -1 2024-04-20T01:48:18.773 INFO:tasks.rados.rados.0.smithi165.stdout:321: expect (ObjNum 116 snap 18 seq_num 116) 2024-04-20T01:48:18.773 INFO:tasks.rados.rados.0.smithi165.stdout:322: delete oid 12 current snap is 23 2024-04-20T01:48:18.773 INFO:tasks.rados.rados.0.smithi165.stdout:315: finishing rollback tid 1 to smithi16579500-6 2024-04-20T01:48:18.774 INFO:tasks.rados.rados.0.smithi165.stdout:315: finishing rollback tid 2 to smithi16579500-6 2024-04-20T01:48:18.774 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 6 v 104 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-04-20T01:48:18.774 INFO:tasks.rados.rados.0.smithi165.stdout:314: finishing write tid 6 to smithi16579500-4 2024-04-20T01:48:18.774 INFO:tasks.rados.rados.0.smithi165.stdout:314: oid 4 updating version 0 to 29 2024-04-20T01:48:18.774 INFO:tasks.rados.rados.0.smithi165.stdout:314: oid 4 updating version 29 to 30 2024-04-20T01:48:18.774 INFO:tasks.rados.rados.0.smithi165.stdout:314: oid 4 updating version 30 to 33 2024-04-20T01:48:18.774 INFO:tasks.rados.rados.0.smithi165.stdout:314: oid 4 version 33 is already newer than 32 2024-04-20T01:48:18.774 INFO:tasks.rados.rados.0.smithi165.stdout:314: oid 4 version 33 is already newer than 31 2024-04-20T01:48:18.774 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 4 v 33 (ObjNum 137 snap 23 seq_num 137) dirty exists 2024-04-20T01:48:18.774 INFO:tasks.rados.rados.0.smithi165.stdout:314: left oid 4 (ObjNum 137 snap 23 seq_num 137) 2024-04-20T01:48:18.774 INFO:tasks.rados.rados.0.smithi165.stdout:316: finishing write tid 1 to smithi16579500-42 2024-04-20T01:48:18.774 INFO:tasks.rados.rados.0.smithi165.stdout:316: finishing write tid 2 to smithi16579500-42 2024-04-20T01:48:18.774 INFO:tasks.rados.rados.0.smithi165.stdout:316: finishing write tid 3 to smithi16579500-42 2024-04-20T01:48:18.774 INFO:tasks.rados.rados.0.smithi165.stdout:316: finishing write tid 4 to smithi16579500-42 2024-04-20T01:48:18.775 INFO:tasks.rados.rados.0.smithi165.stdout:316: finishing write tid 5 to smithi16579500-42 2024-04-20T01:48:18.775 INFO:tasks.rados.rados.0.smithi165.stdout:316: finishing write tid 6 to smithi16579500-42 2024-04-20T01:48:18.775 INFO:tasks.rados.rados.0.smithi165.stdout:316: oid 42 updating version 0 to 54 2024-04-20T01:48:18.775 INFO:tasks.rados.rados.0.smithi165.stdout:316: oid 42 updating version 54 to 55 2024-04-20T01:48:18.775 INFO:tasks.rados.rados.0.smithi165.stdout:316: oid 42 version 55 is already newer than 53 2024-04-20T01:48:18.775 INFO:tasks.rados.rados.0.smithi165.stdout:316: oid 42 updating version 55 to 56 2024-04-20T01:48:18.775 INFO:tasks.rados.rados.0.smithi165.stdout:316: oid 42 version 56 is already newer than 52 2024-04-20T01:48:18.775 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 42 v 56 (ObjNum 138 snap 23 seq_num 138) dirty exists 2024-04-20T01:48:18.775 INFO:tasks.rados.rados.0.smithi165.stdout:316: left oid 42 (ObjNum 138 snap 23 seq_num 138) 2024-04-20T01:48:18.775 INFO:tasks.rados.rados.0.smithi165.stdout:317: finishing write tid 1 to smithi16579500-43 2024-04-20T01:48:18.775 INFO:tasks.rados.rados.0.smithi165.stdout:317: finishing write tid 2 to smithi16579500-43 2024-04-20T01:48:18.775 INFO:tasks.rados.rados.0.smithi165.stdout:317: finishing write tid 3 to smithi16579500-43 2024-04-20T01:48:18.775 INFO:tasks.rados.rados.0.smithi165.stdout:317: finishing write tid 4 to smithi16579500-43 2024-04-20T01:48:18.775 INFO:tasks.rados.rados.0.smithi165.stdout:317: finishing write tid 5 to smithi16579500-43 2024-04-20T01:48:18.775 INFO:tasks.rados.rados.0.smithi165.stdout:317: finishing write tid 6 to smithi16579500-43 2024-04-20T01:48:18.776 INFO:tasks.rados.rados.0.smithi165.stdout:317: oid 43 updating version 0 to 137 2024-04-20T01:48:18.776 INFO:tasks.rados.rados.0.smithi165.stdout:317: oid 43 version 137 is already newer than 135 2024-04-20T01:48:18.776 INFO:tasks.rados.rados.0.smithi165.stdout:317: oid 43 version 137 is already newer than 133 2024-04-20T01:48:18.776 INFO:tasks.rados.rados.0.smithi165.stdout:317: oid 43 version 137 is already newer than 136 2024-04-20T01:48:18.776 INFO:tasks.rados.rados.0.smithi165.stdout:317: oid 43 version 137 is already newer than 134 2024-04-20T01:48:18.776 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 43 v 137 (ObjNum 139 snap 23 seq_num 139) dirty exists 2024-04-20T01:48:18.776 INFO:tasks.rados.rados.0.smithi165.stdout:317: left oid 43 (ObjNum 139 snap 23 seq_num 139) 2024-04-20T01:48:18.777 INFO:tasks.rados.rados.0.smithi165.stdout:319: finishing write tid 1 to smithi16579500-11 2024-04-20T01:48:18.781 INFO:tasks.rados.rados.0.smithi165.stdout:314: done (8 left) 2024-04-20T01:48:18.781 INFO:tasks.rados.rados.0.smithi165.stdout:315: done (7 left) 2024-04-20T01:48:18.781 INFO:tasks.rados.rados.0.smithi165.stdout:316: done (6 left) 2024-04-20T01:48:18.781 INFO:tasks.rados.rados.0.smithi165.stdout:317: done (5 left) 2024-04-20T01:48:18.781 INFO:tasks.rados.rados.0.smithi165.stdout:322: done (4 left) 2024-04-20T01:48:18.782 INFO:tasks.rados.rados.0.smithi165.stdout:323: setattr oid 2 current snap is 23 2024-04-20T01:48:18.782 INFO:tasks.rados.rados.0.smithi165.stdout:319: finishing write tid 2 to smithi16579500-11 2024-04-20T01:48:18.782 INFO:tasks.rados.rados.0.smithi165.stdout:319: finishing write tid 3 to smithi16579500-11 2024-04-20T01:48:18.782 INFO:tasks.rados.rados.0.smithi165.stdout:319: finishing write tid 4 to smithi16579500-11 2024-04-20T01:48:18.782 INFO:tasks.rados.rados.0.smithi165.stdout:319: finishing write tid 5 to smithi16579500-11 2024-04-20T01:48:18.783 INFO:tasks.rados.rados.0.smithi165.stdout:324: read oid 24 snap 20 2024-04-20T01:48:18.783 INFO:tasks.rados.rados.0.smithi165.stdout:324: expect deleted 2024-04-20T01:48:18.783 INFO:tasks.rados.rados.0.smithi165.stdout:325: setattr oid 21 current snap is 23 2024-04-20T01:48:18.785 INFO:tasks.rados.rados.0.smithi165.stdout:319: finishing write tid 6 to smithi16579500-11 2024-04-20T01:48:18.785 INFO:tasks.rados.rados.0.smithi165.stdout:319: oid 11 updating version 0 to 91 2024-04-20T01:48:18.785 INFO:tasks.rados.rados.0.smithi165.stdout:319: oid 11 version 91 is already newer than 87 2024-04-20T01:48:18.785 INFO:tasks.rados.rados.0.smithi165.stdout:319: oid 11 version 91 is already newer than 88 2024-04-20T01:48:18.785 INFO:tasks.rados.rados.0.smithi165.stdout:319: oid 11 version 91 is already newer than 89 2024-04-20T01:48:18.786 INFO:tasks.rados.rados.0.smithi165.stdout:319: oid 11 version 91 is already newer than 90 2024-04-20T01:48:18.786 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 11 v 91 (ObjNum 140 snap 23 seq_num 140) dirty exists 2024-04-20T01:48:18.786 INFO:tasks.rados.rados.0.smithi165.stdout:319: left oid 11 (ObjNum 140 snap 23 seq_num 140) 2024-04-20T01:48:18.786 INFO:tasks.rados.rados.0.smithi165.stdout:319: done (6 left) 2024-04-20T01:48:18.786 INFO:tasks.rados.rados.0.smithi165.stdout:324: done (5 left) 2024-04-20T01:48:18.786 INFO:tasks.rados.rados.0.smithi165.stdout:326: rollback oid 10 current snap is 23 2024-04-20T01:48:18.786 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 10 to 20 2024-04-20T01:48:18.786 INFO:tasks.rados.rados.0.smithi165.stdout:327: read oid 9 snap -1 2024-04-20T01:48:18.786 INFO:tasks.rados.rados.0.smithi165.stdout:327: expect (ObjNum 123 snap 18 seq_num 123) 2024-04-20T01:48:18.787 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 5 current snap is 23 2024-04-20T01:48:18.787 INFO:tasks.rados.rados.0.smithi165.stdout:328: seq_num 142 ranges {1589248=442368,2031616=720896,2752512=540672,3293184=688128} 2024-04-20T01:48:18.798 INFO:tasks.rados.rados.0.smithi165.stdout:328: writing smithi16579500-5 from 1589248 to 2031616 tid 1 2024-04-20T01:48:18.803 INFO:tasks.rados.rados.0.smithi165.stdout:328: writing smithi16579500-5 from 2031616 to 2752512 tid 2 2024-04-20T01:48:18.805 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:18.819+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:18.806 INFO:tasks.rados.rados.0.smithi165.stdout:328: writing smithi16579500-5 from 2752512 to 3293184 tid 3 2024-04-20T01:48:18.811 INFO:tasks.rados.rados.0.smithi165.stdout:328: writing smithi16579500-5 from 3293184 to 3981312 tid 4 2024-04-20T01:48:18.811 INFO:tasks.rados.rados.0.smithi165.stdout:329: rmattr oid 44 current snap is 23 2024-04-20T01:48:18.811 INFO:tasks.rados.rados.0.smithi165.stdout:320: finishing copy_from racing read to smithi16579500-50 2024-04-20T01:48:18.811 INFO:tasks.rados.rados.0.smithi165.stdout:318: expect (ObjNum 133 snap 21 seq_num 133) 2024-04-20T01:48:18.854 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:18.854 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:18.865 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-20T01:48:18.906 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:18.921+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:18.935 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 2 v 139 (ObjNum 104 snap 16 seq_num 104) dirty exists 2024-04-20T01:48:18.935 INFO:tasks.rados.rados.0.smithi165.stdout:321: expect (ObjNum 116 snap 18 seq_num 116) 2024-04-20T01:48:18.982 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 21 v 45 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-04-20T01:48:18.982 INFO:tasks.rados.rados.0.smithi165.stdout:326: finishing rollback tid 0 to smithi16579500-10 2024-04-20T01:48:18.982 INFO:tasks.rados.rados.0.smithi165.stdout:326: finishing rollback tid 1 to smithi16579500-10 2024-04-20T01:48:18.982 INFO:tasks.rados.rados.0.smithi165.stdout:326: finishing rollback tid 2 to smithi16579500-10 2024-04-20T01:48:18.983 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 10 v 143 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-04-20T01:48:18.983 INFO:tasks.rados.rados.0.smithi165.stdout:320: finishing copy_from to smithi16579500-50 2024-04-20T01:48:18.983 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 50 v 32 (ObjNum 117 snap 18 seq_num 117) dirty exists 2024-04-20T01:48:18.983 INFO:tasks.rados.rados.0.smithi165.stdout:328: finishing write tid 1 to smithi16579500-5 2024-04-20T01:48:18.983 INFO:tasks.rados.rados.0.smithi165.stdout:318: done (8 left) 2024-04-20T01:48:18.983 INFO:tasks.rados.rados.0.smithi165.stdout:320: done (7 left) 2024-04-20T01:48:18.983 INFO:tasks.rados.rados.0.smithi165.stdout:321: done (6 left) 2024-04-20T01:48:18.983 INFO:tasks.rados.rados.0.smithi165.stdout:323: done (5 left) 2024-04-20T01:48:18.983 INFO:tasks.rados.rados.0.smithi165.stdout:325: done (4 left) 2024-04-20T01:48:18.983 INFO:tasks.rados.rados.0.smithi165.stdout:326: done (3 left) 2024-04-20T01:48:18.983 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 20 current snap is 23 2024-04-20T01:48:18.983 INFO:tasks.rados.rados.0.smithi165.stdout:327: expect (ObjNum 123 snap 18 seq_num 123) 2024-04-20T01:48:19.006 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:19.021+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:19.018 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:19.018 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:19.030 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-20T01:48:19.086 INFO:tasks.rados.rados.0.smithi165.stdout:328: finishing write tid 2 to smithi16579500-5 2024-04-20T01:48:19.086 INFO:tasks.rados.rados.0.smithi165.stdout:328: finishing write tid 3 to smithi16579500-5 2024-04-20T01:48:19.086 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 44 v 101 (ObjNum 57 snap 2 seq_num 57) dirty exists 2024-04-20T01:48:19.086 INFO:tasks.rados.rados.0.smithi165.stdout:328: finishing write tid 4 to smithi16579500-5 2024-04-20T01:48:19.086 INFO:tasks.rados.rados.0.smithi165.stdout:328: finishing write tid 5 to smithi16579500-5 2024-04-20T01:48:19.086 INFO:tasks.rados.rados.0.smithi165.stdout:328: finishing write tid 6 to smithi16579500-5 2024-04-20T01:48:19.086 INFO:tasks.rados.rados.0.smithi165.stdout:328: oid 5 updating version 0 to 49 2024-04-20T01:48:19.086 INFO:tasks.rados.rados.0.smithi165.stdout:328: oid 5 version 49 is already newer than 47 2024-04-20T01:48:19.087 INFO:tasks.rados.rados.0.smithi165.stdout:328: oid 5 version 49 is already newer than 45 2024-04-20T01:48:19.087 INFO:tasks.rados.rados.0.smithi165.stdout:328: oid 5 version 49 is already newer than 46 2024-04-20T01:48:19.087 INFO:tasks.rados.rados.0.smithi165.stdout:328: oid 5 version 49 is already newer than 48 2024-04-20T01:48:19.087 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 5 v 49 (ObjNum 142 snap 23 seq_num 142) dirty exists 2024-04-20T01:48:19.087 INFO:tasks.rados.rados.0.smithi165.stdout:328: left oid 5 (ObjNum 142 snap 23 seq_num 142) 2024-04-20T01:48:19.087 INFO:tasks.rados.rados.0.smithi165.stdout:330: seq_num 143 ranges {1916928=458752,2375680=475136,2850816=655360,3506176=671744,4177920=147456} 2024-04-20T01:48:19.098 INFO:tasks.rados.rados.0.smithi165.stdout:330: writing smithi16579500-20 from 1916928 to 2375680 tid 1 2024-04-20T01:48:19.101 INFO:tasks.rados.rados.0.smithi165.stdout:330: writing smithi16579500-20 from 2375680 to 2850816 tid 2 2024-04-20T01:48:19.104 INFO:tasks.rados.rados.0.smithi165.stdout:330: writing smithi16579500-20 from 2850816 to 3506176 tid 3 2024-04-20T01:48:19.108 INFO:tasks.rados.rados.0.smithi165.stdout:330: writing smithi16579500-20 from 3506176 to 4177920 tid 4 2024-04-20T01:48:19.110 INFO:tasks.rados.rados.0.smithi165.stdout:330: writing smithi16579500-20 from 4177920 to 4325376 tid 5 2024-04-20T01:48:19.110 INFO:tasks.rados.rados.0.smithi165.stdout:327: done (3 left) 2024-04-20T01:48:19.110 INFO:tasks.rados.rados.0.smithi165.stdout:328: done (2 left) 2024-04-20T01:48:19.110 INFO:tasks.rados.rados.0.smithi165.stdout:329: done (1 left) 2024-04-20T01:48:19.110 INFO:tasks.rados.rados.0.smithi165.stdout:331: read oid 13 snap -1 2024-04-20T01:48:19.110 INFO:tasks.rados.rados.0.smithi165.stdout:331: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-20T01:48:19.111 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 47 current snap is 23 2024-04-20T01:48:19.111 INFO:tasks.rados.rados.0.smithi165.stdout:330: finishing write tid 1 to smithi16579500-20 2024-04-20T01:48:19.111 INFO:tasks.rados.rados.0.smithi165.stdout:330: finishing write tid 2 to smithi16579500-20 2024-04-20T01:48:19.111 INFO:tasks.rados.rados.0.smithi165.stdout:332: seq_num 144 ranges {5308416=16384} 2024-04-20T01:48:19.114 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:19.129+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:19.139 INFO:tasks.rados.rados.0.smithi165.stdout:332: writing smithi16579500-47 from 5308416 to 5324800 tid 1 2024-04-20T01:48:19.139 INFO:tasks.rados.rados.0.smithi165.stdout:333: rmattr oid 6 current snap is 23 2024-04-20T01:48:19.139 INFO:tasks.rados.rados.0.smithi165.stdout:330: finishing write tid 3 to smithi16579500-20 2024-04-20T01:48:19.139 INFO:tasks.rados.rados.0.smithi165.stdout:330: finishing write tid 4 to smithi16579500-20 2024-04-20T01:48:19.139 INFO:tasks.rados.rados.0.smithi165.stdout:330: finishing write tid 5 to smithi16579500-20 2024-04-20T01:48:19.139 INFO:tasks.rados.rados.0.smithi165.stdout:330: finishing write tid 6 to smithi16579500-20 2024-04-20T01:48:19.139 INFO:tasks.rados.rados.0.smithi165.stdout:331: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-20T01:48:19.155 INFO:tasks.rados.rados.0.smithi165.stdout:330: finishing write tid 7 to smithi16579500-20 2024-04-20T01:48:19.156 INFO:tasks.rados.rados.0.smithi165.stdout:330: oid 20 updating version 0 to 154 2024-04-20T01:48:19.156 INFO:tasks.rados.rados.0.smithi165.stdout:330: oid 20 version 154 is already newer than 151 2024-04-20T01:48:19.156 INFO:tasks.rados.rados.0.smithi165.stdout:330: oid 20 updating version 154 to 156 2024-04-20T01:48:19.156 INFO:tasks.rados.rados.0.smithi165.stdout:330: oid 20 version 156 is already newer than 153 2024-04-20T01:48:19.156 INFO:tasks.rados.rados.0.smithi165.stdout:330: oid 20 version 156 is already newer than 155 2024-04-20T01:48:19.156 INFO:tasks.rados.rados.0.smithi165.stdout:330: oid 20 version 156 is already newer than 152 2024-04-20T01:48:19.156 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 20 v 156 (ObjNum 143 snap 23 seq_num 143) dirty exists 2024-04-20T01:48:19.156 INFO:tasks.rados.rados.0.smithi165.stdout:330: left oid 20 (ObjNum 143 snap 23 seq_num 143) 2024-04-20T01:48:19.156 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 6 v 105 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-04-20T01:48:19.156 INFO:tasks.rados.rados.0.smithi165.stdout:330: done (3 left) 2024-04-20T01:48:19.156 INFO:tasks.rados.rados.0.smithi165.stdout:331: done (2 left) 2024-04-20T01:48:19.156 INFO:tasks.rados.rados.0.smithi165.stdout:333: done (1 left) 2024-04-20T01:48:19.156 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 38 current snap is 23 2024-04-20T01:48:19.157 INFO:tasks.rados.rados.0.smithi165.stdout:332: finishing write tid 1 to smithi16579500-47 2024-04-20T01:48:19.157 INFO:tasks.rados.rados.0.smithi165.stdout:334: seq_num 145 ranges {2883584=32768} 2024-04-20T01:48:19.171 INFO:tasks.rados.rados.0.smithi165.stdout:334: writing smithi16579500-38 from 2883584 to 2916352 tid 1 2024-04-20T01:48:19.171 INFO:tasks.rados.rados.0.smithi165.stdout:335: read oid 1 snap -1 2024-04-20T01:48:19.171 INFO:tasks.rados.rados.0.smithi165.stdout:335: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-20T01:48:19.172 INFO:tasks.rados.rados.0.smithi165.stdout:336: read oid 7 snap -1 2024-04-20T01:48:19.172 INFO:tasks.rados.rados.0.smithi165.stdout:336: expect deleted 2024-04-20T01:48:19.172 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 45 current snap is 23 2024-04-20T01:48:19.172 INFO:tasks.rados.rados.0.smithi165.stdout:332: finishing write tid 2 to smithi16579500-47 2024-04-20T01:48:19.172 INFO:tasks.rados.rados.0.smithi165.stdout:332: finishing write tid 3 to smithi16579500-47 2024-04-20T01:48:19.172 INFO:tasks.rados.rados.0.smithi165.stdout:332: oid 47 updating version 0 to 108 2024-04-20T01:48:19.172 INFO:tasks.rados.rados.0.smithi165.stdout:332: oid 47 updating version 108 to 109 2024-04-20T01:48:19.172 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 47 v 109 (ObjNum 144 snap 23 seq_num 144) dirty exists 2024-04-20T01:48:19.172 INFO:tasks.rados.rados.0.smithi165.stdout:332: left oid 47 (ObjNum 144 snap 23 seq_num 144) 2024-04-20T01:48:19.172 INFO:tasks.rados.rados.0.smithi165.stdout:337: seq_num 146 ranges {4898816=49152} 2024-04-20T01:48:19.198 INFO:tasks.rados.rados.0.smithi165.stdout:337: writing smithi16579500-45 from 4898816 to 4947968 tid 1 2024-04-20T01:48:19.198 INFO:tasks.rados.rados.0.smithi165.stdout:332: done (4 left) 2024-04-20T01:48:19.199 INFO:tasks.rados.rados.0.smithi165.stdout:338: read oid 33 snap -1 2024-04-20T01:48:19.199 INFO:tasks.rados.rados.0.smithi165.stdout:338: expect (ObjNum 116 snap 18 seq_num 116) 2024-04-20T01:48:19.199 INFO:tasks.rados.rados.0.smithi165.stdout:339: delete oid 32 current snap is 23 2024-04-20T01:48:19.199 INFO:tasks.rados.rados.0.smithi165.stdout:335: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-20T01:48:19.209 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:19.223+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:19.215 INFO:tasks.rados.rados.0.smithi165.stdout:334: finishing write tid 1 to smithi16579500-38 2024-04-20T01:48:19.215 INFO:tasks.rados.rados.0.smithi165.stdout:334: finishing write tid 2 to smithi16579500-38 2024-04-20T01:48:19.215 INFO:tasks.rados.rados.0.smithi165.stdout:334: finishing write tid 3 to smithi16579500-38 2024-04-20T01:48:19.216 INFO:tasks.rados.rados.0.smithi165.stdout:334: oid 38 updating version 0 to 133 2024-04-20T01:48:19.216 INFO:tasks.rados.rados.0.smithi165.stdout:334: oid 38 updating version 133 to 134 2024-04-20T01:48:19.216 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 38 v 134 (ObjNum 145 snap 23 seq_num 145) dirty exists 2024-04-20T01:48:19.216 INFO:tasks.rados.rados.0.smithi165.stdout:334: left oid 38 (ObjNum 145 snap 23 seq_num 145) 2024-04-20T01:48:19.216 INFO:tasks.rados.rados.0.smithi165.stdout:337: finishing write tid 1 to smithi16579500-45 2024-04-20T01:48:19.216 INFO:tasks.rados.rados.0.smithi165.stdout:337: finishing write tid 2 to smithi16579500-45 2024-04-20T01:48:19.216 INFO:tasks.rados.rados.0.smithi165.stdout:338: expect (ObjNum 116 snap 18 seq_num 116) 2024-04-20T01:48:19.228 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:19.228 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:19.240 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-20T01:48:19.263 INFO:tasks.rados.rados.0.smithi165.stdout:337: finishing write tid 3 to smithi16579500-45 2024-04-20T01:48:19.263 INFO:tasks.rados.rados.0.smithi165.stdout:337: oid 45 updating version 0 to 158 2024-04-20T01:48:19.263 INFO:tasks.rados.rados.0.smithi165.stdout:337: oid 45 version 158 is already newer than 157 2024-04-20T01:48:19.263 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 45 v 158 (ObjNum 146 snap 23 seq_num 146) dirty exists 2024-04-20T01:48:19.263 INFO:tasks.rados.rados.0.smithi165.stdout:337: left oid 45 (ObjNum 146 snap 23 seq_num 146) 2024-04-20T01:48:19.263 INFO:tasks.rados.rados.0.smithi165.stdout:334: done (5 left) 2024-04-20T01:48:19.263 INFO:tasks.rados.rados.0.smithi165.stdout:335: done (4 left) 2024-04-20T01:48:19.263 INFO:tasks.rados.rados.0.smithi165.stdout:336: done (3 left) 2024-04-20T01:48:19.263 INFO:tasks.rados.rados.0.smithi165.stdout:337: done (2 left) 2024-04-20T01:48:19.263 INFO:tasks.rados.rados.0.smithi165.stdout:338: done (1 left) 2024-04-20T01:48:19.263 INFO:tasks.rados.rados.0.smithi165.stdout:339: done (0 left) 2024-04-20T01:48:19.263 INFO:tasks.rados.rados.0.smithi165.stdout:340: snap_create 2024-04-20T01:48:19.309 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:19.324+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:19.410 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:19.425+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:19.428 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:19.428 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:19.440 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-20T01:48:19.512 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:19.526+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:19.584 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:19.584 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:19.596 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-20T01:48:19.612 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:19.627+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:19.700 INFO:tasks.rados.rados.0.smithi165.stdout:340: done (0 left) 2024-04-20T01:48:19.700 INFO:tasks.rados.rados.0.smithi165.stdout:341: read oid 46 snap -1 2024-04-20T01:48:19.700 INFO:tasks.rados.rados.0.smithi165.stdout:341: expect (ObjNum 45 snap 0 seq_num 45) 2024-04-20T01:48:19.700 INFO:tasks.rados.rados.0.smithi165.stdout:342: snap_create 2024-04-20T01:48:19.705 INFO:tasks.rados.rados.0.smithi165.stdout:341: expect (ObjNum 45 snap 0 seq_num 45) 2024-04-20T01:48:19.713 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:19.727+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:19.723 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-20T01:48:19.743 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:19.743 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:19.756 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-20T01:48:19.814 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:19.828+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:19.915 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:19.929+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:19.943 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:19.943 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:19.958 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-20T01:48:20.016 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:20.029+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:20.104 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:20.104 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:20.116 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-20T01:48:20.116 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:20.130+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:20.217 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:20.231+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:20.261 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:20.261 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:20.273 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-20T01:48:20.357 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:20.371+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:20.370 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-20T01:48:20.418 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:20.432+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:20.462 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:20.462 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:20.474 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-20T01:48:20.519 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:20.533+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:20.537 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:48:20.559 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:48:20.581 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:48:20.603 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:48:20.620 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:20.634+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:20.622 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:20.622 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:20.625 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:48:20.635 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-20T01:48:20.649 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:48:20.673 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:48:20.694 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:48:20.701 INFO:tasks.rados.rados.0.smithi165.stdout:341: done (1 left) 2024-04-20T01:48:20.702 INFO:tasks.rados.rados.0.smithi165.stdout:342: done (0 left) 2024-04-20T01:48:20.702 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 49 current snap is 25 2024-04-20T01:48:20.702 INFO:tasks.rados.rados.0.smithi165.stdout:343: seq_num 148 ranges {1916928=81920} 2024-04-20T01:48:20.706 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is set 2024-04-20T01:48:20.713 INFO:tasks.rados.rados.0.smithi165.stdout:343: writing smithi16579500-49 from 1916928 to 1998848 tid 1 2024-04-20T01:48:20.713 INFO:tasks.rados.rados.0.smithi165.stdout:344: copy_from oid 39 from oid 22 current snap is 25 2024-04-20T01:48:20.713 INFO:tasks.rados.rados.0.smithi165.stdout:345: snap_remove snap 22 2024-04-20T01:48:20.721 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:20.735+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:20.724 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:48:20.749 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:48:20.770 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:48:20.792 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:48:20.797 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:20.797 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:20.809 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-20T01:48:20.812 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:48:20.833 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:48:20.834 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:20.848+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:20.856 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:48:20.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:48:20.871+0000 7f4f8db88700 0 -- 172.21.15.44:0/977631 >> [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] conn(0x7f4f8821f340 msgr2=0x7f4f88221730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-20T01:48:20.923 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:20.937+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:20.958 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:20.958 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-04-20T01:48:20.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-04-20T01:48:20.958 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-04-20T01:48:20.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.958 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.5s4 33/30 ECSubWrite(tid=2, reqid=client.4643.0:11, at_version=0'0, trim_to=0'0, roll_forward_to=32'2))", 2024-04-20T01:48:20.958 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.765877+0000", 2024-04-20T01:48:20.958 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 46.205632602999998, 2024-04-20T01:48:20.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0075032049999999998, 2024-04-20T01:48:20.959 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:20.959 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:20.959 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:20.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:20.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765877+0000", 2024-04-20T01:48:20.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:20.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:20.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765877+0000", 2024-04-20T01:48:20.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:20.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:20.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765877+0000", 2024-04-20T01:48:20.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:20.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:20.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765904+0000", 2024-04-20T01:48:20.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7461e-05 2024-04-20T01:48:20.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:20.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765927+0000", 2024-04-20T01:48:20.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2697999999999999e-05 2024-04-20T01:48:20.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:20.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765939+0000", 2024-04-20T01:48:20.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2142e-05 2024-04-20T01:48:20.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:20.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.768205+0000", 2024-04-20T01:48:20.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022662239999999998 2024-04-20T01:48:20.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:20.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.768221+0000", 2024-04-20T01:48:20.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6161e-05 2024-04-20T01:48:20.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:20.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.773346+0000", 2024-04-20T01:48:20.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051250469999999998 2024-04-20T01:48:20.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:20.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.773380+0000", 2024-04-20T01:48:20.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3760999999999997e-05 2024-04-20T01:48:20.962 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:20.962 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:20.962 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:20.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.963 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/30 ECSubWrite(tid=4, reqid=client.4643.0:58, at_version=33'3, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:48:20.963 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.780211+0000", 2024-04-20T01:48:20.963 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 46.191299014000002, 2024-04-20T01:48:20.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0098815680000000003, 2024-04-20T01:48:20.963 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:20.963 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:20.963 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:20.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:20.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780211+0000", 2024-04-20T01:48:20.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:20.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:20.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780211+0000", 2024-04-20T01:48:20.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:20.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:20.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780209+0000", 2024-04-20T01:48:20.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:20.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:20.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780307+0000", 2024-04-20T01:48:20.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8575999999999999e-05 2024-04-20T01:48:20.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:20.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780328+0000", 2024-04-20T01:48:20.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0537999999999999e-05 2024-04-20T01:48:20.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:20.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780340+0000", 2024-04-20T01:48:20.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1997999999999999e-05 2024-04-20T01:48:20.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.965 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:20.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780423+0000", 2024-04-20T01:48:20.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2812999999999995e-05 2024-04-20T01:48:20.965 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:20.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780434+0000", 2024-04-20T01:48:20.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1028e-05 2024-04-20T01:48:20.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:20.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790048+0000", 2024-04-20T01:48:20.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096147339999999998 2024-04-20T01:48:20.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.966 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.966 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:20.966 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790092+0000", 2024-04-20T01:48:20.966 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3649999999999997e-05 2024-04-20T01:48:20.966 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:20.967 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:20.967 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:20.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.967 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/30 ECSubWrite(tid=5, reqid=client.4643.0:59, at_version=33'4, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:48:20.967 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.781201+0000", 2024-04-20T01:48:20.967 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 46.190308776000002, 2024-04-20T01:48:20.967 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0089164740000000006, 2024-04-20T01:48:20.967 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:20.967 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:20.967 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:20.967 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.967 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:20.967 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781201+0000", 2024-04-20T01:48:20.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:20.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:20.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781201+0000", 2024-04-20T01:48:20.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:20.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:20.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781200+0000", 2024-04-20T01:48:20.968 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:20.968 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.968 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.968 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:20.968 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781219+0000", 2024-04-20T01:48:20.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8748000000000001e-05 2024-04-20T01:48:20.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:20.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781236+0000", 2024-04-20T01:48:20.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7251000000000001e-05 2024-04-20T01:48:20.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:20.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781246+0000", 2024-04-20T01:48:20.969 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0026e-05 2024-04-20T01:48:20.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.969 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.969 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:20.969 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.782742+0000", 2024-04-20T01:48:20.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001496079 2024-04-20T01:48:20.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:20.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.782753+0000", 2024-04-20T01:48:20.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0767999999999999e-05 2024-04-20T01:48:20.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:20.970 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790103+0000", 2024-04-20T01:48:20.970 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073499259999999997 2024-04-20T01:48:20.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.970 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.970 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:20.971 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790117+0000", 2024-04-20T01:48:20.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4729000000000001e-05 2024-04-20T01:48:20.971 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:20.971 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:20.971 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:20.971 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.971 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.971 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.5s4 33/30 ECSubWrite(tid=5, reqid=client.4643.0:65, at_version=33'4, trim_to=0'0, roll_forward_to=32'2))", 2024-04-20T01:48:20.971 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.783666+0000", 2024-04-20T01:48:20.971 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 46.187843755000003, 2024-04-20T01:48:20.971 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0063804899999999999, 2024-04-20T01:48:20.971 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:20.971 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:20.971 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:20.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:20.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783666+0000", 2024-04-20T01:48:20.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:20.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:20.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783666+0000", 2024-04-20T01:48:20.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:20.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.972 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.972 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:20.972 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783665+0000", 2024-04-20T01:48:20.972 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:20.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:20.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783685+0000", 2024-04-20T01:48:20.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9735e-05 2024-04-20T01:48:20.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:20.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783701+0000", 2024-04-20T01:48:20.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6175e-05 2024-04-20T01:48:20.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.973 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.973 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:20.973 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783710+0000", 2024-04-20T01:48:20.973 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3649999999999993e-06 2024-04-20T01:48:20.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:20.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783800+0000", 2024-04-20T01:48:20.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0061000000000004e-05 2024-04-20T01:48:20.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:20.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783809+0000", 2024-04-20T01:48:20.974 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0750000000000004e-06 2024-04-20T01:48:20.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.974 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:20.974 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790003+0000", 2024-04-20T01:48:20.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0061941590000000003 2024-04-20T01:48:20.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.975 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:20.975 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790046+0000", 2024-04-20T01:48:20.975 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2831999999999999e-05 2024-04-20T01:48:20.975 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:20.975 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:20.975 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:20.975 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.975 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.975 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs5 33/30 ECSubWrite(tid=0, reqid=client.4643.0:106, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:48:20.975 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.825844+0000", 2024-04-20T01:48:20.975 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 46.145665792999999, 2024-04-20T01:48:20.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0062702069999999999, 2024-04-20T01:48:20.976 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:20.976 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:20.976 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:20.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:20.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.825844+0000", 2024-04-20T01:48:20.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:20.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.976 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:20.976 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.825844+0000", 2024-04-20T01:48:20.976 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:20.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.976 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:20.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.825843+0000", 2024-04-20T01:48:20.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:20.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:20.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826258+0000", 2024-04-20T01:48:20.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00041487099999999998 2024-04-20T01:48:20.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.977 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.977 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:20.977 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826269+0000", 2024-04-20T01:48:20.977 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0264e-05 2024-04-20T01:48:20.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:20.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826275+0000", 2024-04-20T01:48:20.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3010000000000003e-06 2024-04-20T01:48:20.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:20.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826329+0000", 2024-04-20T01:48:20.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4036999999999998e-05 2024-04-20T01:48:20.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.978 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.978 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:20.978 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826337+0000", 2024-04-20T01:48:20.978 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7470000000000005e-06 2024-04-20T01:48:20.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:20.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.832089+0000", 2024-04-20T01:48:20.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0057520339999999996 2024-04-20T01:48:20.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.979 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:20.979 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.832114+0000", 2024-04-20T01:48:20.979 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5267999999999999e-05 2024-04-20T01:48:20.979 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:20.979 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:20.979 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:20.979 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.979 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.980 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.es6 33/30 ECSubWrite(tid=3, reqid=client.4643.0:104, at_version=33'4, trim_to=0'0, roll_forward_to=33'1))", 2024-04-20T01:48:20.980 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.966859+0000", 2024-04-20T01:48:20.980 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 46.004650095999999, 2024-04-20T01:48:20.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0063969730000000002, 2024-04-20T01:48:20.980 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:20.980 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:20.980 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:20.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:20.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966859+0000", 2024-04-20T01:48:20.980 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:20.980 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.980 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.980 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:20.980 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966859+0000", 2024-04-20T01:48:20.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:20.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:20.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966859+0000", 2024-04-20T01:48:20.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:20.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:20.981 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966876+0000", 2024-04-20T01:48:20.981 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7116999999999999e-05 2024-04-20T01:48:20.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.981 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.981 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:20.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966887+0000", 2024-04-20T01:48:20.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1508999999999999e-05 2024-04-20T01:48:20.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:20.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966894+0000", 2024-04-20T01:48:20.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4219999999999997e-06 2024-04-20T01:48:20.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:20.982 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.967342+0000", 2024-04-20T01:48:20.982 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00044805799999999998 2024-04-20T01:48:20.982 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.982 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.982 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:20.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.967352+0000", 2024-04-20T01:48:20.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0497e-05 2024-04-20T01:48:20.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:20.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.973223+0000", 2024-04-20T01:48:20.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0058706749999999997 2024-04-20T01:48:20.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.983 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.983 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:20.983 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.973256+0000", 2024-04-20T01:48:20.983 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3544999999999998e-05 2024-04-20T01:48:20.983 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:20.983 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:20.983 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:20.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.984 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 42/30 ECSubWrite(tid=98, reqid=client.4643.0:321, at_version=42'36, trim_to=0'0, roll_forward_to=42'35))", 2024-04-20T01:48:20.984 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:43.943374+0000", 2024-04-20T01:48:20.984 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.028135345000003, 2024-04-20T01:48:20.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0059029729999999997, 2024-04-20T01:48:20.984 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:20.984 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:20.984 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:20.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.984 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:20.984 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943374+0000", 2024-04-20T01:48:20.984 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:20.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.984 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:20.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943374+0000", 2024-04-20T01:48:20.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:20.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:20.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943372+0000", 2024-04-20T01:48:20.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:20.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.985 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.985 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:20.985 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943540+0000", 2024-04-20T01:48:20.985 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00016738300000000001 2024-04-20T01:48:20.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:20.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943564+0000", 2024-04-20T01:48:20.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4371000000000001e-05 2024-04-20T01:48:20.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:20.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943577+0000", 2024-04-20T01:48:20.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3254e-05 2024-04-20T01:48:20.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.986 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.986 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:20.986 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943661+0000", 2024-04-20T01:48:20.986 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3727000000000007e-05 2024-04-20T01:48:20.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:20.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.943674+0000", 2024-04-20T01:48:20.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3097e-05 2024-04-20T01:48:20.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:20.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.949246+0000", 2024-04-20T01:48:20.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055718210000000002 2024-04-20T01:48:20.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.987 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.987 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:20.987 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.949277+0000", 2024-04-20T01:48:20.987 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1226999999999998e-05 2024-04-20T01:48:20.988 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:20.988 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:20.988 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:20.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.988 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 47/30 ECSubWrite(tid=114, reqid=client.4643.0:392, at_version=47'49, trim_to=0'0, roll_forward_to=47'47))", 2024-04-20T01:48:20.988 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.746767+0000", 2024-04-20T01:48:20.988 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 32.224742591999998, 2024-04-20T01:48:20.988 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0061268629999999998, 2024-04-20T01:48:20.988 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:20.988 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:20.988 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:20.988 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.988 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:20.988 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746767+0000", 2024-04-20T01:48:20.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:20.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:20.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746767+0000", 2024-04-20T01:48:20.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:20.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:20.989 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746766+0000", 2024-04-20T01:48:20.989 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:20.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.989 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:20.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746823+0000", 2024-04-20T01:48:20.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6743999999999997e-05 2024-04-20T01:48:20.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:20.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746840+0000", 2024-04-20T01:48:20.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7300999999999999e-05 2024-04-20T01:48:20.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:20.990 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746850+0000", 2024-04-20T01:48:20.990 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0356999999999999e-05 2024-04-20T01:48:20.990 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.990 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.990 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:20.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746919+0000", 2024-04-20T01:48:20.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8180000000000001e-05 2024-04-20T01:48:20.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:20.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746927+0000", 2024-04-20T01:48:20.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3659999999999992e-06 2024-04-20T01:48:20.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.991 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:20.991 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.752855+0000", 2024-04-20T01:48:20.991 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0059277779999999999 2024-04-20T01:48:20.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.991 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.992 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:20.992 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.752894+0000", 2024-04-20T01:48:20.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9057999999999998e-05 2024-04-20T01:48:20.992 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:20.992 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:20.992 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:20.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.992 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.992 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 60/30 ECSubWrite(tid=144, reqid=client.4643.0:506, at_version=60'65, trim_to=0'0, roll_forward_to=60'62))", 2024-04-20T01:48:20.992 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:01.773724+0000", 2024-04-20T01:48:20.992 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.197785370999998, 2024-04-20T01:48:20.992 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0062789609999999996, 2024-04-20T01:48:20.992 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:20.992 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:20.992 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:20.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:20.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773724+0000", 2024-04-20T01:48:20.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:20.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:20.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773724+0000", 2024-04-20T01:48:20.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:20.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.993 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.993 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:20.993 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773723+0000", 2024-04-20T01:48:20.993 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:20.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:20.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773743+0000", 2024-04-20T01:48:20.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0032000000000001e-05 2024-04-20T01:48:20.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:20.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773755+0000", 2024-04-20T01:48:20.994 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1156e-05 2024-04-20T01:48:20.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.994 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.994 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:20.994 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773763+0000", 2024-04-20T01:48:20.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2870000000000004e-06 2024-04-20T01:48:20.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:20.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773830+0000", 2024-04-20T01:48:20.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7269999999999998e-05 2024-04-20T01:48:20.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:20.995 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773840+0000", 2024-04-20T01:48:20.995 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0284e-05 2024-04-20T01:48:20.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.995 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.995 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:20.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.779970+0000", 2024-04-20T01:48:20.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.006129364 2024-04-20T01:48:20.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.996 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:20.996 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.780003+0000", 2024-04-20T01:48:20.996 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3402000000000003e-05 2024-04-20T01:48:20.996 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:20.996 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:20.996 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:20.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.996 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.996 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 63/30 ECSubWrite(tid=165, reqid=client.4643.0:566, at_version=63'82, trim_to=0'0, roll_forward_to=62'79))", 2024-04-20T01:48:20.996 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:04.760184+0000", 2024-04-20T01:48:20.997 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 16.211325559999999, 2024-04-20T01:48:20.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0058193819999999997, 2024-04-20T01:48:20.997 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:20.997 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:20.997 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:20.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:20.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.760184+0000", 2024-04-20T01:48:20.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:20.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.997 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:20.997 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.760184+0000", 2024-04-20T01:48:20.997 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:20.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:20.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.760183+0000", 2024-04-20T01:48:20.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:20.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:20.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.760251+0000", 2024-04-20T01:48:20.998 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8096000000000007e-05 2024-04-20T01:48:20.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.998 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.998 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:20.998 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.760266+0000", 2024-04-20T01:48:20.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5370999999999999e-05 2024-04-20T01:48:20.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:20.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.760278+0000", 2024-04-20T01:48:20.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1208e-05 2024-04-20T01:48:20.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:20.999 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.760351+0000", 2024-04-20T01:48:20.999 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3227999999999998e-05 2024-04-20T01:48:20.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:20.999 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:20.999 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.760362+0000", 2024-04-20T01:48:21.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0922e-05 2024-04-20T01:48:21.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.000 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.765976+0000", 2024-04-20T01:48:21.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0056145140000000001 2024-04-20T01:48:21.000 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.000 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.000 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:21.000 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.766003+0000", 2024-04-20T01:48:21.000 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7081e-05 2024-04-20T01:48:21.000 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.000 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:21.001 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.001 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 65/30 ECSubWrite(tid=174, reqid=osd.4.0:174, at_version=65'88, trim_to=0'0, roll_forward_to=63'84))", 2024-04-20T01:48:21.001 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:06.763588+0000", 2024-04-20T01:48:21.001 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 14.207921452000001, 2024-04-20T01:48:21.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0081115820000000009, 2024-04-20T01:48:21.001 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:21.001 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.001 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:21.001 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.001 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:21.001 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.763588+0000", 2024-04-20T01:48:21.001 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:21.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:21.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.763588+0000", 2024-04-20T01:48:21.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:21.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:21.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.763587+0000", 2024-04-20T01:48:21.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:21.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.002 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.002 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:21.002 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.763612+0000", 2024-04-20T01:48:21.002 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5106999999999999e-05 2024-04-20T01:48:21.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:21.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.763624+0000", 2024-04-20T01:48:21.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1487e-05 2024-04-20T01:48:21.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.763631+0000", 2024-04-20T01:48:21.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0249999999999997e-06 2024-04-20T01:48:21.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.003 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:21.003 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.764079+0000", 2024-04-20T01:48:21.003 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00044799400000000001 2024-04-20T01:48:21.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.764088+0000", 2024-04-20T01:48:21.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5240000000000006e-06 2024-04-20T01:48:21.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.004 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.004 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.771664+0000", 2024-04-20T01:48:21.004 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0075759160000000002 2024-04-20T01:48:21.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.004 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.005 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:21.005 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.771700+0000", 2024-04-20T01:48:21.005 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5493000000000001e-05 2024-04-20T01:48:21.005 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.005 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:21.005 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.005 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.006 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 68/30 ECSubWrite(tid=209, reqid=client.4643.0:670, at_version=68'110, trim_to=0'0, roll_forward_to=68'107))", 2024-04-20T01:48:21.006 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.945938+0000", 2024-04-20T01:48:21.006 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.025572036, 2024-04-20T01:48:21.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.006638194, 2024-04-20T01:48:21.006 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:21.006 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.006 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:21.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.006 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:21.006 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.945938+0000", 2024-04-20T01:48:21.006 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:21.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.006 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:21.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.945938+0000", 2024-04-20T01:48:21.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:21.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:21.007 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.945937+0000", 2024-04-20T01:48:21.007 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:21.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.007 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.007 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:21.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.946003+0000", 2024-04-20T01:48:21.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6026999999999997e-05 2024-04-20T01:48:21.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.008 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:21.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.946019+0000", 2024-04-20T01:48:21.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6458000000000001e-05 2024-04-20T01:48:21.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.008 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.008 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.946029+0000", 2024-04-20T01:48:21.008 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.55e-06 2024-04-20T01:48:21.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.008 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:21.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.946077+0000", 2024-04-20T01:48:21.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8205999999999999e-05 2024-04-20T01:48:21.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.946089+0000", 2024-04-20T01:48:21.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2e-05 2024-04-20T01:48:21.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.009 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.009 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.009 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.952557+0000", 2024-04-20T01:48:21.009 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0064676919999999997 2024-04-20T01:48:21.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.010 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:21.010 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.952576+0000", 2024-04-20T01:48:21.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8838000000000002e-05 2024-04-20T01:48:21.010 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.010 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:21.010 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.010 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 68/30 ECSubWrite(tid=220, reqid=client.4643.0:703, at_version=68'117, trim_to=0'0, roll_forward_to=68'116))", 2024-04-20T01:48:21.010 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:10.059553+0000", 2024-04-20T01:48:21.010 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.911956329000001, 2024-04-20T01:48:21.010 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0062243860000000002, 2024-04-20T01:48:21.011 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:21.011 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.011 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:21.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:21.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059553+0000", 2024-04-20T01:48:21.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:21.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.011 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:21.011 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059553+0000", 2024-04-20T01:48:21.011 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:21.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.011 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:21.012 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059552+0000", 2024-04-20T01:48:21.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:21.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:21.012 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059733+0000", 2024-04-20T01:48:21.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018084899999999999 2024-04-20T01:48:21.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.012 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:21.012 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059773+0000", 2024-04-20T01:48:21.012 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0985999999999997e-05 2024-04-20T01:48:21.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.012 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059801+0000", 2024-04-20T01:48:21.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7727999999999999e-05 2024-04-20T01:48:21.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:21.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059897+0000", 2024-04-20T01:48:21.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5471999999999997e-05 2024-04-20T01:48:21.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.013 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.013 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.013 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059911+0000", 2024-04-20T01:48:21.013 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4644000000000001e-05 2024-04-20T01:48:21.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.014 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.014 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.065740+0000", 2024-04-20T01:48:21.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0058290709999999999 2024-04-20T01:48:21.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.014 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:21.014 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.065778+0000", 2024-04-20T01:48:21.014 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7234999999999998e-05 2024-04-20T01:48:21.014 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.014 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:21.014 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.014 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.015 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 68/30 ECSubWrite(tid=221, reqid=client.4643.0:704, at_version=68'118, trim_to=0'0, roll_forward_to=68'116))", 2024-04-20T01:48:21.015 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:10.064387+0000", 2024-04-20T01:48:21.015 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 10.907122467000001, 2024-04-20T01:48:21.015 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068350370000000004, 2024-04-20T01:48:21.015 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:21.015 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.015 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:21.015 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.015 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:21.015 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064387+0000", 2024-04-20T01:48:21.015 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:21.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.015 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.015 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:21.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064387+0000", 2024-04-20T01:48:21.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:21.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:21.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064386+0000", 2024-04-20T01:48:21.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:21.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:21.016 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064602+0000", 2024-04-20T01:48:21.016 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00021609999999999999 2024-04-20T01:48:21.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.016 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.016 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:21.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064651+0000", 2024-04-20T01:48:21.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8976e-05 2024-04-20T01:48:21.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064664+0000", 2024-04-20T01:48:21.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2632999999999999e-05 2024-04-20T01:48:21.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.017 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:21.017 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064732+0000", 2024-04-20T01:48:21.017 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8047999999999996e-05 2024-04-20T01:48:21.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.017 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.018 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.018 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064748+0000", 2024-04-20T01:48:21.018 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6640999999999999e-05 2024-04-20T01:48:21.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.018 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.018 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.071185+0000", 2024-04-20T01:48:21.018 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0064368109999999997 2024-04-20T01:48:21.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.018 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:21.018 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.071222+0000", 2024-04-20T01:48:21.018 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6942e-05 2024-04-20T01:48:21.018 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.019 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:21.019 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.019 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs5 75/30 ECSubWrite(tid=117, reqid=client.4643.0:785, at_version=75'98, trim_to=0'0, roll_forward_to=75'93))", 2024-04-20T01:48:21.019 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:16.750149+0000", 2024-04-20T01:48:21.019 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.2213609769999998, 2024-04-20T01:48:21.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0088344489999999994, 2024-04-20T01:48:21.019 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:21.019 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.019 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:21.019 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.019 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:21.019 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.750149+0000", 2024-04-20T01:48:21.019 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:21.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:21.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.750149+0000", 2024-04-20T01:48:21.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:21.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:21.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.750148+0000", 2024-04-20T01:48:21.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:21.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.020 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.020 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:21.020 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.750323+0000", 2024-04-20T01:48:21.020 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017526299999999999 2024-04-20T01:48:21.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:21.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.750340+0000", 2024-04-20T01:48:21.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7241999999999999e-05 2024-04-20T01:48:21.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.021 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.021 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.021 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.750351+0000", 2024-04-20T01:48:21.021 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0968000000000001e-05 2024-04-20T01:48:21.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:21.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.750419+0000", 2024-04-20T01:48:21.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7500000000000001e-05 2024-04-20T01:48:21.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.750432+0000", 2024-04-20T01:48:21.022 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3232e-05 2024-04-20T01:48:21.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.022 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.022 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.022 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.758945+0000", 2024-04-20T01:48:21.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0085134830000000005 2024-04-20T01:48:21.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.023 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:21.023 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.758983+0000", 2024-04-20T01:48:21.023 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7734000000000002e-05 2024-04-20T01:48:21.023 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.023 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:21.023 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.023 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.023 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.023 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs5 75/30 ECSubWrite(tid=118, reqid=client.4643.0:786, at_version=75'99, trim_to=0'0, roll_forward_to=75'95))", 2024-04-20T01:48:21.023 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:16.754332+0000", 2024-04-20T01:48:21.023 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.2171777070000003, 2024-04-20T01:48:21.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068657700000000002, 2024-04-20T01:48:21.024 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:21.024 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.024 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:21.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.024 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:21.024 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.754332+0000", 2024-04-20T01:48:21.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:21.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.024 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:21.024 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.754332+0000", 2024-04-20T01:48:21.024 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:21.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.024 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:21.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.754327+0000", 2024-04-20T01:48:21.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999952 2024-04-20T01:48:21.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:21.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.754656+0000", 2024-04-20T01:48:21.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00032840000000000001 2024-04-20T01:48:21.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.025 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.025 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:21.025 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.754677+0000", 2024-04-20T01:48:21.025 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1089000000000001e-05 2024-04-20T01:48:21.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.754687+0000", 2024-04-20T01:48:21.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0838e-05 2024-04-20T01:48:21.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:21.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.754736+0000", 2024-04-20T01:48:21.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8596e-05 2024-04-20T01:48:21.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.026 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.026 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.754743+0000", 2024-04-20T01:48:21.026 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0160000000000003e-06 2024-04-20T01:48:21.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.761161+0000", 2024-04-20T01:48:21.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0064178580000000002 2024-04-20T01:48:21.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.027 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:21.027 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.761198+0000", 2024-04-20T01:48:21.027 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6733999999999998e-05 2024-04-20T01:48:21.027 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.027 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:21.027 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.027 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.027 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.028 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs5 75/30 ECSubWrite(tid=119, reqid=client.4643.0:787, at_version=75'100, trim_to=0'0, roll_forward_to=75'97))", 2024-04-20T01:48:21.028 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:16.755830+0000", 2024-04-20T01:48:21.028 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.21567969, 2024-04-20T01:48:21.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0087352760000000002, 2024-04-20T01:48:21.028 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:21.028 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.028 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:21.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:21.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.755830+0000", 2024-04-20T01:48:21.028 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:21.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.028 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.028 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:21.028 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.755830+0000", 2024-04-20T01:48:21.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:21.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:21.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.755829+0000", 2024-04-20T01:48:21.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:21.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:21.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.755930+0000", 2024-04-20T01:48:21.029 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000100809 2024-04-20T01:48:21.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.029 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.029 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:21.029 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.755946+0000", 2024-04-20T01:48:21.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6334e-05 2024-04-20T01:48:21.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.755964+0000", 2024-04-20T01:48:21.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7442e-05 2024-04-20T01:48:21.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:21.030 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.756402+0000", 2024-04-20T01:48:21.030 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00043873000000000002 2024-04-20T01:48:21.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.030 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.030 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.756415+0000", 2024-04-20T01:48:21.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2687000000000001e-05 2024-04-20T01:48:21.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.764523+0000", 2024-04-20T01:48:21.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0081081409999999993 2024-04-20T01:48:21.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.031 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.031 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:21.031 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.764565+0000", 2024-04-20T01:48:21.031 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2067000000000002e-05 2024-04-20T01:48:21.031 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.031 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:21.032 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.032 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 75/30 ECSubWrite(tid=265, reqid=osd.4.0:261, at_version=0'0, trim_to=0'0, roll_forward_to=75'141))", 2024-04-20T01:48:21.032 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:16.757192+0000", 2024-04-20T01:48:21.032 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.2143179039999996, 2024-04-20T01:48:21.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0074381890000000004, 2024-04-20T01:48:21.032 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:21.032 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.032 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:21.032 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.032 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:21.032 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.757192+0000", 2024-04-20T01:48:21.032 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:21.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:21.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.757192+0000", 2024-04-20T01:48:21.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:21.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:21.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.757191+0000", 2024-04-20T01:48:21.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:21.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.033 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.033 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:21.033 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.757220+0000", 2024-04-20T01:48:21.033 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9322000000000001e-05 2024-04-20T01:48:21.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:21.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.757240+0000", 2024-04-20T01:48:21.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9558000000000001e-05 2024-04-20T01:48:21.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.757251+0000", 2024-04-20T01:48:21.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1151e-05 2024-04-20T01:48:21.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.034 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:21.034 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.757339+0000", 2024-04-20T01:48:21.034 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7825999999999996e-05 2024-04-20T01:48:21.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.757349+0000", 2024-04-20T01:48:21.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0633e-05 2024-04-20T01:48:21.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.764574+0000", 2024-04-20T01:48:21.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072251939999999999 2024-04-20T01:48:21.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.035 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.035 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:21.035 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.764630+0000", 2024-04-20T01:48:21.035 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5454999999999997e-05 2024-04-20T01:48:21.036 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.036 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:21.036 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.036 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 75/30 ECSubWrite(tid=267, reqid=osd.4.0:267, at_version=75'143, trim_to=0'0, roll_forward_to=75'141))", 2024-04-20T01:48:21.036 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:16.758765+0000", 2024-04-20T01:48:21.036 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.2127447519999999, 2024-04-20T01:48:21.036 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0058925849999999997, 2024-04-20T01:48:21.036 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:21.036 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.036 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:21.036 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.036 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:21.036 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.758765+0000", 2024-04-20T01:48:21.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:21.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:21.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.758765+0000", 2024-04-20T01:48:21.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:21.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.037 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:21.037 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.758764+0000", 2024-04-20T01:48:21.037 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:21.037 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.037 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:21.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.758798+0000", 2024-04-20T01:48:21.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4345999999999999e-05 2024-04-20T01:48:21.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:21.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.758811+0000", 2024-04-20T01:48:21.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2849e-05 2024-04-20T01:48:21.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.038 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.038 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.038 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.758832+0000", 2024-04-20T01:48:21.038 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0852999999999998e-05 2024-04-20T01:48:21.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:21.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.758885+0000", 2024-04-20T01:48:21.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3365000000000002e-05 2024-04-20T01:48:21.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.758897+0000", 2024-04-20T01:48:21.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1521e-05 2024-04-20T01:48:21.039 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.039 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.039 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.039 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.764634+0000", 2024-04-20T01:48:21.039 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0057369949999999999 2024-04-20T01:48:21.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.040 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:21.040 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.764657+0000", 2024-04-20T01:48:21.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3473e-05 2024-04-20T01:48:21.040 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.040 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:21.040 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.040 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.040 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 75/30 ECSubWrite(tid=268, reqid=osd.4.0:268, at_version=75'145, trim_to=0'0, roll_forward_to=75'141))", 2024-04-20T01:48:21.040 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:16.759662+0000", 2024-04-20T01:48:21.040 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.211847154, 2024-04-20T01:48:21.040 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0058269430000000002, 2024-04-20T01:48:21.041 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:21.041 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.041 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:21.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:21.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.759662+0000", 2024-04-20T01:48:21.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:21.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.041 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:21.041 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.759662+0000", 2024-04-20T01:48:21.041 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:21.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.041 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:21.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.759662+0000", 2024-04-20T01:48:21.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:21.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:21.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.759685+0000", 2024-04-20T01:48:21.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3459e-05 2024-04-20T01:48:21.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.042 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.042 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:21.042 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.759699+0000", 2024-04-20T01:48:21.042 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4151999999999999e-05 2024-04-20T01:48:21.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.759708+0000", 2024-04-20T01:48:21.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8060000000000001e-06 2024-04-20T01:48:21.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:21.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.759791+0000", 2024-04-20T01:48:21.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2906999999999998e-05 2024-04-20T01:48:21.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.043 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.043 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.043 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.759801+0000", 2024-04-20T01:48:21.043 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0489000000000001e-05 2024-04-20T01:48:21.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.765443+0000", 2024-04-20T01:48:21.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0056415579999999996 2024-04-20T01:48:21.044 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:21.044 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:21.044 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:21.044 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.765489+0000", 2024-04-20T01:48:21.044 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6356000000000001e-05 2024-04-20T01:48:21.044 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.044 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:21.044 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.044 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:21.044 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:21.044 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:21.046 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-20T01:48:21.047 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:21.061+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:21.080 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:48:21.081 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:48:21.081 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:48:21.081 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:48:21.082 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:48:21.082 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:48:21.082 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:48:21.082 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:48:21.082 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:48:21.082 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:48:21.082 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:48:21.082 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:48:21.082 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:48:21.082 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:48:21.082 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:48:21.082 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:48:21.082 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:48:21.147 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:21.161+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:21.188 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:21.188 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:21.188 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 0 2024-04-20T01:48:21.188 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:21.198 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-20T01:48:21.248 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:21.262+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:21.350 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:21.364+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:21.366 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:21.366 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:21.366 INFO:teuthology.orchestra.run.smithi121.stdout: "complaint_time": 30, 2024-04-20T01:48:21.366 INFO:teuthology.orchestra.run.smithi121.stdout: "num_blocked_ops": 0 2024-04-20T01:48:21.366 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:21.376 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-20T01:48:21.451 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:48:21.465+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:48:21.506 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:21.506 INFO:teuthology.orchestra.run.smithi121.stdout: "size": 20, 2024-04-20T01:48:21.506 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 600, 2024-04-20T01:48:21.506 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-04-20T01:48:21.506 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.506 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 47/30 ECSubWrite(tid=246, reqid=client.4643.0:388, at_version=47'38, trim_to=0'0, roll_forward_to=47'35))", 2024-04-20T01:48:21.506 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.745219+0000", 2024-04-20T01:48:21.506 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 32.774173040000001, 2024-04-20T01:48:21.506 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0073977050000000001, 2024-04-20T01:48:21.506 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.507 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.507 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.507 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.507 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.507 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745219+0000", 2024-04-20T01:48:21.507 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.507 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.507 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.507 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.507 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745219+0000", 2024-04-20T01:48:21.507 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.507 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.507 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.507 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.508 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745218+0000", 2024-04-20T01:48:21.508 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:21.508 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.508 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.508 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.508 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745266+0000", 2024-04-20T01:48:21.508 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.7012999999999999e-05 2024-04-20T01:48:21.508 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.508 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.508 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.508 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745277+0000", 2024-04-20T01:48:21.508 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1348999999999999e-05 2024-04-20T01:48:21.508 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.508 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.509 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.509 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745284+0000", 2024-04-20T01:48:21.509 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0060000000000003e-06 2024-04-20T01:48:21.509 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.509 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.509 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.509 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745340+0000", 2024-04-20T01:48:21.509 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6023000000000002e-05 2024-04-20T01:48:21.509 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.509 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.509 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.509 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745361+0000", 2024-04-20T01:48:21.509 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1356999999999999e-05 2024-04-20T01:48:21.510 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.510 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.510 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.510 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.752597+0000", 2024-04-20T01:48:21.510 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0072354910000000001 2024-04-20T01:48:21.510 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.510 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.510 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:21.510 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.752617+0000", 2024-04-20T01:48:21.510 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0341000000000001e-05 2024-04-20T01:48:21.510 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.510 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.510 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.510 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.511 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.511 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 48/30 ECSubWrite(tid=252, reqid=osd.13.0:252, at_version=48'42, trim_to=0'0, roll_forward_to=47'40))", 2024-04-20T01:48:21.511 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.710137+0000", 2024-04-20T01:48:21.511 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 31.809255191999998, 2024-04-20T01:48:21.511 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0082943410000000002, 2024-04-20T01:48:21.511 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.511 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.511 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.511 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.511 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.511 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710137+0000", 2024-04-20T01:48:21.511 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.511 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.512 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.512 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.512 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710137+0000", 2024-04-20T01:48:21.512 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.512 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.512 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.512 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.512 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710137+0000", 2024-04-20T01:48:21.512 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:21.512 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.512 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.512 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.512 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710162+0000", 2024-04-20T01:48:21.513 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4768000000000001e-05 2024-04-20T01:48:21.513 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.513 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.513 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.513 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710174+0000", 2024-04-20T01:48:21.513 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2150999999999999e-05 2024-04-20T01:48:21.513 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.513 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.513 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.513 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710183+0000", 2024-04-20T01:48:21.513 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8200000000000003e-06 2024-04-20T01:48:21.513 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.513 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.513 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.513 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710238+0000", 2024-04-20T01:48:21.514 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.4954999999999998e-05 2024-04-20T01:48:21.514 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.514 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.514 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.514 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710248+0000", 2024-04-20T01:48:21.514 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0222e-05 2024-04-20T01:48:21.514 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.514 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.514 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.514 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718408+0000", 2024-04-20T01:48:21.514 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0081605570000000006 2024-04-20T01:48:21.514 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.514 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.514 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:21.515 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718432+0000", 2024-04-20T01:48:21.515 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3129000000000002e-05 2024-04-20T01:48:21.515 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.515 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.515 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.515 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.515 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.515 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 48/30 ECSubWrite(tid=253, reqid=osd.13.0:253, at_version=48'44, trim_to=0'0, roll_forward_to=47'40))", 2024-04-20T01:48:21.515 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.711047+0000", 2024-04-20T01:48:21.515 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 31.808345773999999, 2024-04-20T01:48:21.515 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0093898609999999993, 2024-04-20T01:48:21.515 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.515 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.516 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.516 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.516 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.516 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711047+0000", 2024-04-20T01:48:21.516 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.516 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.516 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.516 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.516 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711047+0000", 2024-04-20T01:48:21.516 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.516 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.516 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.516 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.516 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711046+0000", 2024-04-20T01:48:21.517 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:21.517 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.517 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.517 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.517 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711066+0000", 2024-04-20T01:48:21.517 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0421999999999999e-05 2024-04-20T01:48:21.517 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.517 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.517 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.517 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711082+0000", 2024-04-20T01:48:21.517 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5231999999999999e-05 2024-04-20T01:48:21.517 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.517 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.517 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.518 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711091+0000", 2024-04-20T01:48:21.518 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.1679999999999993e-06 2024-04-20T01:48:21.518 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.518 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.518 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.518 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.712543+0000", 2024-04-20T01:48:21.518 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001452545 2024-04-20T01:48:21.518 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.518 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.518 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.518 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.712549+0000", 2024-04-20T01:48:21.518 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.7450000000000001e-06 2024-04-20T01:48:21.518 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.519 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.519 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.519 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720419+0000", 2024-04-20T01:48:21.519 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0078696630000000007 2024-04-20T01:48:21.519 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.519 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.519 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:21.519 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720436+0000", 2024-04-20T01:48:21.519 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7869000000000001e-05 2024-04-20T01:48:21.519 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.519 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.519 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.519 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.519 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.519 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 48/30 ECSubWrite(tid=86, reqid=client.4643.0:400, at_version=48'72, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:21.520 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.713378+0000", 2024-04-20T01:48:21.520 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 31.806014291, 2024-04-20T01:48:21.520 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0070552389999999996, 2024-04-20T01:48:21.520 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.520 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.520 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.520 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.520 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.520 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713378+0000", 2024-04-20T01:48:21.520 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.520 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.520 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.520 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.520 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713378+0000", 2024-04-20T01:48:21.521 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.521 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.521 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.521 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.521 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713378+0000", 2024-04-20T01:48:21.521 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:21.521 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.521 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.521 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.521 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713397+0000", 2024-04-20T01:48:21.521 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9165999999999999e-05 2024-04-20T01:48:21.521 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.521 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.522 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.522 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713408+0000", 2024-04-20T01:48:21.522 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0854e-05 2024-04-20T01:48:21.522 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.522 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.522 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.522 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713414+0000", 2024-04-20T01:48:21.522 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.1850000000000001e-06 2024-04-20T01:48:21.522 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.522 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.522 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.522 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713458+0000", 2024-04-20T01:48:21.522 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.4162999999999997e-05 2024-04-20T01:48:21.523 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.523 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.523 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.523 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713469+0000", 2024-04-20T01:48:21.523 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0789e-05 2024-04-20T01:48:21.523 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.523 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.523 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.523 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720404+0000", 2024-04-20T01:48:21.523 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0069350109999999996 2024-04-20T01:48:21.523 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.523 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.523 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:21.523 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720433+0000", 2024-04-20T01:48:21.524 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9660000000000001e-05 2024-04-20T01:48:21.524 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.524 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.524 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.524 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.524 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.524 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 48/30 ECSubWrite(tid=87, reqid=client.4643.0:401, at_version=48'73, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:21.524 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.713964+0000", 2024-04-20T01:48:21.524 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 31.80542853, 2024-04-20T01:48:21.524 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0065393960000000003, 2024-04-20T01:48:21.524 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.524 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.524 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.525 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.525 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.525 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713964+0000", 2024-04-20T01:48:21.525 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.525 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.525 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.525 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.525 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713964+0000", 2024-04-20T01:48:21.525 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.525 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.525 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.525 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.525 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713963+0000", 2024-04-20T01:48:21.526 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:21.526 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.526 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.526 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.526 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713977+0000", 2024-04-20T01:48:21.526 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3828999999999999e-05 2024-04-20T01:48:21.526 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.526 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.526 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.526 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713988+0000", 2024-04-20T01:48:21.526 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0728e-05 2024-04-20T01:48:21.526 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.526 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.527 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.527 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.713994+0000", 2024-04-20T01:48:21.527 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.2949999999999999e-06 2024-04-20T01:48:21.527 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.527 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.527 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.527 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715264+0000", 2024-04-20T01:48:21.527 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00126958 2024-04-20T01:48:21.527 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.527 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.527 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.527 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.715273+0000", 2024-04-20T01:48:21.527 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2720000000000003e-06 2024-04-20T01:48:21.528 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.528 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.528 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.528 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720484+0000", 2024-04-20T01:48:21.528 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0052106820000000003 2024-04-20T01:48:21.528 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.528 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.528 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:21.528 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720503+0000", 2024-04-20T01:48:21.528 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9706000000000001e-05 2024-04-20T01:48:21.528 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.528 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.528 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.528 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.529 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.529 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 48/30 ECSubWrite(tid=121, reqid=osd.4.0:121, at_version=48'53, trim_to=0'0, roll_forward_to=47'49))", 2024-04-20T01:48:21.529 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.718319+0000", 2024-04-20T01:48:21.529 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 31.801073818999999, 2024-04-20T01:48:21.529 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0075908039999999996, 2024-04-20T01:48:21.529 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.529 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.529 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.529 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.529 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.529 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718319+0000", 2024-04-20T01:48:21.529 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.529 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.529 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.529 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.530 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718319+0000", 2024-04-20T01:48:21.530 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.530 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.530 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.530 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.530 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718318+0000", 2024-04-20T01:48:21.530 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:21.530 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.530 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.530 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.530 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718351+0000", 2024-04-20T01:48:21.530 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.3179000000000002e-05 2024-04-20T01:48:21.530 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.530 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.531 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.531 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718362+0000", 2024-04-20T01:48:21.531 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0689999999999999e-05 2024-04-20T01:48:21.531 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.531 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.531 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.531 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718370+0000", 2024-04-20T01:48:21.531 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4610000000000008e-06 2024-04-20T01:48:21.531 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.531 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.531 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.531 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718703+0000", 2024-04-20T01:48:21.531 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00033314100000000001 2024-04-20T01:48:21.531 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.531 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.532 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.532 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718713+0000", 2024-04-20T01:48:21.532 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.5459999999999997e-06 2024-04-20T01:48:21.532 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.532 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.532 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.532 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.725887+0000", 2024-04-20T01:48:21.532 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0071743909999999996 2024-04-20T01:48:21.532 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.532 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.532 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:21.532 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.725909+0000", 2024-04-20T01:48:21.532 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1965999999999999e-05 2024-04-20T01:48:21.532 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.533 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.533 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.533 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.533 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.533 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 63/30 ECSubWrite(tid=164, reqid=client.4643.0:565, at_version=63'81, trim_to=0'0, roll_forward_to=62'79))", 2024-04-20T01:48:21.533 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:04.757828+0000", 2024-04-20T01:48:21.533 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 16.761564453999998, 2024-04-20T01:48:21.533 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0067220630000000003, 2024-04-20T01:48:21.533 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.533 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.533 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.533 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.533 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.533 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.757828+0000", 2024-04-20T01:48:21.533 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.533 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.534 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.534 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.534 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.757828+0000", 2024-04-20T01:48:21.534 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.534 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.534 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.534 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.534 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.757827+0000", 2024-04-20T01:48:21.534 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:21.534 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.534 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.534 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.534 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.758278+0000", 2024-04-20T01:48:21.534 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00045044699999999998 2024-04-20T01:48:21.535 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.535 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.535 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.535 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.758291+0000", 2024-04-20T01:48:21.535 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3220999999999999e-05 2024-04-20T01:48:21.535 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.535 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.535 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.535 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.758303+0000", 2024-04-20T01:48:21.535 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1909000000000001e-05 2024-04-20T01:48:21.535 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.535 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.535 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.535 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.758360+0000", 2024-04-20T01:48:21.535 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.7015999999999998e-05 2024-04-20T01:48:21.536 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.536 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.536 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.536 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.758371+0000", 2024-04-20T01:48:21.536 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0841e-05 2024-04-20T01:48:21.536 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.536 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.536 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.536 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764529+0000", 2024-04-20T01:48:21.536 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0061582030000000001 2024-04-20T01:48:21.536 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.536 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.536 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:21.536 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764550+0000", 2024-04-20T01:48:21.536 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0897999999999999e-05 2024-04-20T01:48:21.537 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.537 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.537 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.537 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.537 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.537 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 63/30 ECSubWrite(tid=165, reqid=client.4643.0:566, at_version=63'82, trim_to=0'0, roll_forward_to=62'79))", 2024-04-20T01:48:21.537 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:04.760061+0000", 2024-04-20T01:48:21.537 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 16.759330948999999, 2024-04-20T01:48:21.537 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0073565369999999998, 2024-04-20T01:48:21.537 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.537 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.537 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.538 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.538 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.538 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.760061+0000", 2024-04-20T01:48:21.538 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.538 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.538 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.538 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.538 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.760061+0000", 2024-04-20T01:48:21.538 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.538 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.538 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.538 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.538 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.760061+0000", 2024-04-20T01:48:21.539 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:21.539 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.539 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.539 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.539 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.760383+0000", 2024-04-20T01:48:21.539 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00032234400000000002 2024-04-20T01:48:21.539 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.539 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.539 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.539 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.760397+0000", 2024-04-20T01:48:21.539 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4302e-05 2024-04-20T01:48:21.539 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.539 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.539 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.539 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.760407+0000", 2024-04-20T01:48:21.540 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4029999999999996e-06 2024-04-20T01:48:21.540 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.540 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.540 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.540 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.760449+0000", 2024-04-20T01:48:21.540 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2517e-05 2024-04-20T01:48:21.540 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.540 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.540 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.540 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.760460+0000", 2024-04-20T01:48:21.540 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0730999999999999e-05 2024-04-20T01:48:21.540 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.540 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.540 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.540 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.767386+0000", 2024-04-20T01:48:21.541 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0069258799999999997 2024-04-20T01:48:21.541 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.542 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.542 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:21.542 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.767418+0000", 2024-04-20T01:48:21.542 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.2082000000000002e-05 2024-04-20T01:48:21.543 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.543 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.543 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.543 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.543 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.543 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 63/30 ECSubWrite(tid=166, reqid=client.4643.0:567, at_version=63'83, trim_to=0'0, roll_forward_to=62'79))", 2024-04-20T01:48:21.543 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:04.764484+0000", 2024-04-20T01:48:21.543 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 16.754908378, 2024-04-20T01:48:21.543 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0083177819999999993, 2024-04-20T01:48:21.543 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.543 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.543 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.544 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.544 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.544 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764484+0000", 2024-04-20T01:48:21.544 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.544 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.544 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.544 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.544 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764484+0000", 2024-04-20T01:48:21.544 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.544 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.545 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.545 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.545 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764483+0000", 2024-04-20T01:48:21.545 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:21.545 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.545 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.545 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.545 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764821+0000", 2024-04-20T01:48:21.545 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00033812600000000001 2024-04-20T01:48:21.545 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.545 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.546 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.546 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764833+0000", 2024-04-20T01:48:21.546 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1722000000000001e-05 2024-04-20T01:48:21.546 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.546 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.546 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.546 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764843+0000", 2024-04-20T01:48:21.546 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.431e-06 2024-04-20T01:48:21.546 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.547 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.547 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.547 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764896+0000", 2024-04-20T01:48:21.547 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.2988000000000002e-05 2024-04-20T01:48:21.547 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.547 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.547 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.547 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764905+0000", 2024-04-20T01:48:21.547 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.1079999999999999e-06 2024-04-20T01:48:21.547 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.547 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.547 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.548 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.772779+0000", 2024-04-20T01:48:21.548 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0078742910000000003 2024-04-20T01:48:21.548 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.548 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.548 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:21.548 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.772802+0000", 2024-04-20T01:48:21.548 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2869000000000001e-05 2024-04-20T01:48:21.548 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.548 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.548 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.549 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.549 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.549 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.es2 68/30 ECSubWrite(tid=74, reqid=client.4643.0:609, at_version=68'56, trim_to=0'0, roll_forward_to=68'55))", 2024-04-20T01:48:21.549 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:09.802706+0000", 2024-04-20T01:48:21.549 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 11.716686085999999, 2024-04-20T01:48:21.549 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0065556440000000002, 2024-04-20T01:48:21.549 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.549 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.549 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.549 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.550 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.550 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.802706+0000", 2024-04-20T01:48:21.550 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.550 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.550 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.550 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.550 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.802706+0000", 2024-04-20T01:48:21.550 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.550 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.550 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.551 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.551 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.802706+0000", 2024-04-20T01:48:21.551 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:21.551 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.551 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.551 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.551 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.803924+0000", 2024-04-20T01:48:21.551 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001218434 2024-04-20T01:48:21.551 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.551 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.552 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.552 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.803938+0000", 2024-04-20T01:48:21.552 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3713000000000001e-05 2024-04-20T01:48:21.552 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.552 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.552 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.552 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.803948+0000", 2024-04-20T01:48:21.553 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0251000000000001e-05 2024-04-20T01:48:21.553 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.553 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.553 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.553 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.804062+0000", 2024-04-20T01:48:21.553 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000113967 2024-04-20T01:48:21.553 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.553 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.553 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.553 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.804074+0000", 2024-04-20T01:48:21.553 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2058999999999999e-05 2024-04-20T01:48:21.554 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.554 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.554 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.554 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.809238+0000", 2024-04-20T01:48:21.554 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0051638070000000003 2024-04-20T01:48:21.554 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.554 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.554 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:21.554 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.809262+0000", 2024-04-20T01:48:21.554 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4224999999999999e-05 2024-04-20T01:48:21.555 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.555 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.555 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.555 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.555 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.555 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 68/30 ECSubWrite(tid=232, reqid=client.4643.0:715, at_version=68'123, trim_to=0'0, roll_forward_to=68'121))", 2024-04-20T01:48:21.555 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:10.127605+0000", 2024-04-20T01:48:21.555 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 11.391787520999999, 2024-04-20T01:48:21.555 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0065499709999999999, 2024-04-20T01:48:21.555 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.555 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.556 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.556 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.556 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.556 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.127605+0000", 2024-04-20T01:48:21.556 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.556 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.556 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.556 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.556 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.127605+0000", 2024-04-20T01:48:21.556 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.557 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.557 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.557 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.557 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.127604+0000", 2024-04-20T01:48:21.557 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:21.557 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.557 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.557 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.557 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.128067+0000", 2024-04-20T01:48:21.557 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00046319200000000002 2024-04-20T01:48:21.557 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.557 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.557 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.557 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.128078+0000", 2024-04-20T01:48:21.558 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0521e-05 2024-04-20T01:48:21.558 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.558 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.558 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.558 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.128084+0000", 2024-04-20T01:48:21.558 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.5379999999999999e-06 2024-04-20T01:48:21.558 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.558 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.558 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.558 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.128125+0000", 2024-04-20T01:48:21.558 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.0797999999999998e-05 2024-04-20T01:48:21.559 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.559 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.559 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.559 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.128133+0000", 2024-04-20T01:48:21.559 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.5680000000000002e-06 2024-04-20T01:48:21.559 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.559 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.559 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.559 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.134131+0000", 2024-04-20T01:48:21.559 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0059987030000000002 2024-04-20T01:48:21.559 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.559 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.559 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:21.559 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.134155+0000", 2024-04-20T01:48:21.560 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3396000000000001e-05 2024-04-20T01:48:21.560 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.560 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.560 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.560 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.560 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.560 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 75/57 ECSubWrite(tid=188, reqid=osd.8.0:188, at_version=75'132, trim_to=0'0, roll_forward_to=75'128))", 2024-04-20T01:48:21.560 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.730586+0000", 2024-04-20T01:48:21.560 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 4.788806031, 2024-04-20T01:48:21.560 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010377753999999999, 2024-04-20T01:48:21.560 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.560 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.560 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.560 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.561 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.561 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730586+0000", 2024-04-20T01:48:21.561 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.561 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.561 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.561 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.561 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730586+0000", 2024-04-20T01:48:21.561 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.561 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.561 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.561 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.561 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730586+0000", 2024-04-20T01:48:21.561 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:21.562 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.562 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.562 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.562 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730610+0000", 2024-04-20T01:48:21.562 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4423e-05 2024-04-20T01:48:21.562 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.562 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.562 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.562 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730626+0000", 2024-04-20T01:48:21.562 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5549999999999999e-05 2024-04-20T01:48:21.562 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.563 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.563 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.563 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730635+0000", 2024-04-20T01:48:21.563 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7589999999999992e-06 2024-04-20T01:48:21.563 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.563 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.563 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.563 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730677+0000", 2024-04-20T01:48:21.563 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.1668999999999997e-05 2024-04-20T01:48:21.563 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.563 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.563 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.563 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730685+0000", 2024-04-20T01:48:21.564 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.1009999999999992e-06 2024-04-20T01:48:21.564 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.564 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.564 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.564 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740942+0000", 2024-04-20T01:48:21.564 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010257252 2024-04-20T01:48:21.564 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.564 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.564 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:21.564 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740964+0000", 2024-04-20T01:48:21.564 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1838999999999999e-05 2024-04-20T01:48:21.564 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.564 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.564 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.565 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.565 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.565 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 75/30 ECSubWrite(tid=256, reqid=client.4643.0:783, at_version=75'137, trim_to=0'0, roll_forward_to=73'131))", 2024-04-20T01:48:21.565 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.730626+0000", 2024-04-20T01:48:21.565 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 4.7887667599999997, 2024-04-20T01:48:21.565 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.014031138, 2024-04-20T01:48:21.565 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.565 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.565 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.565 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.565 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.565 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730626+0000", 2024-04-20T01:48:21.565 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.565 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.565 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.566 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.566 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730626+0000", 2024-04-20T01:48:21.566 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.566 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.566 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.566 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.566 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730625+0000", 2024-04-20T01:48:21.566 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:21.566 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.566 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.566 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.566 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730868+0000", 2024-04-20T01:48:21.566 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000242778 2024-04-20T01:48:21.566 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.567 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.567 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.567 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730887+0000", 2024-04-20T01:48:21.567 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8913999999999999e-05 2024-04-20T01:48:21.567 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.567 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.567 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.567 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730903+0000", 2024-04-20T01:48:21.567 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6841999999999999e-05 2024-04-20T01:48:21.567 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.567 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.567 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.567 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730953+0000", 2024-04-20T01:48:21.567 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.9836999999999997e-05 2024-04-20T01:48:21.568 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.568 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.568 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.568 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730964+0000", 2024-04-20T01:48:21.568 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0209e-05 2024-04-20T01:48:21.568 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.568 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.568 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.568 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744636+0000", 2024-04-20T01:48:21.568 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013672253000000001 2024-04-20T01:48:21.568 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.568 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.568 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:21.568 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744657+0000", 2024-04-20T01:48:21.568 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1016999999999999e-05 2024-04-20T01:48:21.569 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.569 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.569 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.569 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.569 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.569 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s3 75/30 ECSubWrite(tid=55, reqid=client.4643.0:781, at_version=75'41, trim_to=0'0, roll_forward_to=66'39))", 2024-04-20T01:48:21.569 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.734920+0000", 2024-04-20T01:48:21.569 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 4.7844727730000001, 2024-04-20T01:48:21.569 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.017734534, 2024-04-20T01:48:21.569 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.569 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.569 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.569 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.569 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.570 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.734920+0000", 2024-04-20T01:48:21.570 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.570 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.570 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.570 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.570 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.734920+0000", 2024-04-20T01:48:21.570 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.570 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.570 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.570 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.570 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.734919+0000", 2024-04-20T01:48:21.570 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:48:21.570 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.570 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.571 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.571 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735179+0000", 2024-04-20T01:48:21.571 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00025959699999999997 2024-04-20T01:48:21.571 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.571 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.571 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.571 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735191+0000", 2024-04-20T01:48:21.571 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1606e-05 2024-04-20T01:48:21.571 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.571 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.571 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.571 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735198+0000", 2024-04-20T01:48:21.571 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.2840000000000001e-06 2024-04-20T01:48:21.571 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.572 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.572 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.572 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735251+0000", 2024-04-20T01:48:21.572 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3513000000000001e-05 2024-04-20T01:48:21.572 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.572 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.572 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.572 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735262+0000", 2024-04-20T01:48:21.572 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0556e-05 2024-04-20T01:48:21.572 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.572 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.572 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.572 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.752626+0000", 2024-04-20T01:48:21.572 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.017363983 2024-04-20T01:48:21.572 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.752654+0000", 2024-04-20T01:48:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8135000000000001e-05 2024-04-20T01:48:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.es2 75/30 ECSubWrite(tid=102, reqid=osd.11.0:101, at_version=0'0, trim_to=0'0, roll_forward_to=75'78))", 2024-04-20T01:48:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.737314+0000", 2024-04-20T01:48:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 4.7820779939999998, 2024-04-20T01:48:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0066859959999999996, 2024-04-20T01:48:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737314+0000", 2024-04-20T01:48:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737314+0000", 2024-04-20T01:48:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737313+0000", 2024-04-20T01:48:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737327+0000", 2024-04-20T01:48:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3199e-05 2024-04-20T01:48:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737343+0000", 2024-04-20T01:48:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6779999999999999e-05 2024-04-20T01:48:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737353+0000", 2024-04-20T01:48:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.9569999999999997e-06 2024-04-20T01:48:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737402+0000", 2024-04-20T01:48:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.8312000000000001e-05 2024-04-20T01:48:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737412+0000", 2024-04-20T01:48:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0461e-05 2024-04-20T01:48:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.743964+0000", 2024-04-20T01:48:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0065519760000000002 2024-04-20T01:48:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744000+0000", 2024-04-20T01:48:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.6328999999999997e-05 2024-04-20T01:48:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 75/30 ECSubWrite(tid=355, reqid=osd.13.0:352, at_version=0'0, trim_to=0'0, roll_forward_to=75'123))", 2024-04-20T01:48:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.738213+0000", 2024-04-20T01:48:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 4.7811795569999997, 2024-04-20T01:48:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0065918410000000002, 2024-04-20T01:48:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738213+0000", 2024-04-20T01:48:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738213+0000", 2024-04-20T01:48:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738212+0000", 2024-04-20T01:48:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.580 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.580 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.580 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738225+0000", 2024-04-20T01:48:21.580 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3156000000000001e-05 2024-04-20T01:48:21.580 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.580 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.580 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.581 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738242+0000", 2024-04-20T01:48:21.581 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6585000000000001e-05 2024-04-20T01:48:21.581 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.581 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.581 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.581 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738253+0000", 2024-04-20T01:48:21.581 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1609e-05 2024-04-20T01:48:21.581 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739209+0000", 2024-04-20T01:48:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00095566499999999997 2024-04-20T01:48:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739222+0000", 2024-04-20T01:48:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3158e-05 2024-04-20T01:48:21.583 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.583 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.583 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744786+0000", 2024-04-20T01:48:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0055634550000000001 2024-04-20T01:48:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744805+0000", 2024-04-20T01:48:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9151999999999999e-05 2024-04-20T01:48:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 75/30 ECSubWrite(tid=356, reqid=osd.13.0:356, at_version=75'125, trim_to=0'0, roll_forward_to=75'123))", 2024-04-20T01:48:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.739126+0000", 2024-04-20T01:48:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 4.7802666, 2024-04-20T01:48:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0079893540000000006, 2024-04-20T01:48:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739126+0000", 2024-04-20T01:48:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739126+0000", 2024-04-20T01:48:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739125+0000", 2024-04-20T01:48:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739148+0000", 2024-04-20T01:48:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3448e-05 2024-04-20T01:48:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739162+0000", 2024-04-20T01:48:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3492000000000001e-05 2024-04-20T01:48:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739171+0000", 2024-04-20T01:48:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3260000000000002e-06 2024-04-20T01:48:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739908+0000", 2024-04-20T01:48:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000736482 2024-04-20T01:48:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739920+0000", 2024-04-20T01:48:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1963999999999999e-05 2024-04-20T01:48:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747088+0000", 2024-04-20T01:48:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0071685769999999998 2024-04-20T01:48:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747115+0000", 2024-04-20T01:48:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6826e-05 2024-04-20T01:48:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 75/30 ECSubWrite(tid=357, reqid=osd.13.0:357, at_version=75'127, trim_to=0'0, roll_forward_to=75'123))", 2024-04-20T01:48:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.739696+0000", 2024-04-20T01:48:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 4.7796964119999998, 2024-04-20T01:48:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010085257, 2024-04-20T01:48:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739696+0000", 2024-04-20T01:48:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739696+0000", 2024-04-20T01:48:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739695+0000", 2024-04-20T01:48:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739724+0000", 2024-04-20T01:48:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8401000000000001e-05 2024-04-20T01:48:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739736+0000", 2024-04-20T01:48:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2340999999999999e-05 2024-04-20T01:48:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739745+0000", 2024-04-20T01:48:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.5109999999999992e-06 2024-04-20T01:48:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.741433+0000", 2024-04-20T01:48:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0016870329999999999 2024-04-20T01:48:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.741442+0000", 2024-04-20T01:48:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7200000000000001e-06 2024-04-20T01:48:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.749753+0000", 2024-04-20T01:48:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0083103659999999996 2024-04-20T01:48:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.749781+0000", 2024-04-20T01:48:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8665000000000001e-05 2024-04-20T01:48:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 75/30 ECSubWrite(tid=260, reqid=osd.4.0:260, at_version=75'139, trim_to=0'0, roll_forward_to=75'135))", 2024-04-20T01:48:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.742436+0000", 2024-04-20T01:48:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 4.7769563919999998, 2024-04-20T01:48:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.007991382, 2024-04-20T01:48:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742436+0000", 2024-04-20T01:48:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742436+0000", 2024-04-20T01:48:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742435+0000", 2024-04-20T01:48:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742465+0000", 2024-04-20T01:48:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.955e-05 2024-04-20T01:48:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742484+0000", 2024-04-20T01:48:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9009999999999999e-05 2024-04-20T01:48:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742493+0000", 2024-04-20T01:48:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4550000000000001e-06 2024-04-20T01:48:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742599+0000", 2024-04-20T01:48:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00010645999999999999 2024-04-20T01:48:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742606+0000", 2024-04-20T01:48:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.6080000000000001e-06 2024-04-20T01:48:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.750398+0000", 2024-04-20T01:48:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.007792312 2024-04-20T01:48:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.750427+0000", 2024-04-20T01:48:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9019000000000001e-05 2024-04-20T01:48:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 75/30 ECSubWrite(tid=261, reqid=osd.4.0:261, at_version=75'141, trim_to=0'0, roll_forward_to=75'135))", 2024-04-20T01:48:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.743991+0000", 2024-04-20T01:48:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 4.7754013369999999, 2024-04-20T01:48:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011465479000000001, 2024-04-20T01:48:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.743991+0000", 2024-04-20T01:48:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.743991+0000", 2024-04-20T01:48:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.743990+0000", 2024-04-20T01:48:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744013+0000", 2024-04-20T01:48:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2344000000000002e-05 2024-04-20T01:48:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744023+0000", 2024-04-20T01:48:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0637e-05 2024-04-20T01:48:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744032+0000", 2024-04-20T01:48:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.1729999999999995e-06 2024-04-20T01:48:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744690+0000", 2024-04-20T01:48:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00065875600000000003 2024-04-20T01:48:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744700+0000", 2024-04-20T01:48:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3710000000000006e-06 2024-04-20T01:48:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.755437+0000", 2024-04-20T01:48:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010737412 2024-04-20T01:48:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.755457+0000", 2024-04-20T01:48:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9465000000000001e-05 2024-04-20T01:48:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.601 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:21.603 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-20T01:48:21.604 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:21.618+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:21.704 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:21.718+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:21.714 INFO:tasks.rados.rados.0.smithi165.stdout:345: done (2 left) 2024-04-20T01:48:21.714 INFO:tasks.rados.rados.0.smithi165.stdout:346: rollback oid 40 current snap is 25 2024-04-20T01:48:21.714 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 40 to 20 2024-04-20T01:48:21.714 INFO:tasks.rados.rados.0.smithi165.stdout:347: read oid 13 snap -1 2024-04-20T01:48:21.714 INFO:tasks.rados.rados.0.smithi165.stdout:347: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-20T01:48:21.714 INFO:tasks.rados.rados.0.smithi165.stdout:348: snap_create 2024-04-20T01:48:21.715 INFO:tasks.rados.rados.0.smithi165.stdout:343: finishing write tid 1 to smithi16579500-49 2024-04-20T01:48:21.715 INFO:tasks.rados.rados.0.smithi165.stdout:343: finishing write tid 2 to smithi16579500-49 2024-04-20T01:48:21.715 INFO:tasks.rados.rados.0.smithi165.stdout:343: finishing write tid 3 to smithi16579500-49 2024-04-20T01:48:21.715 INFO:tasks.rados.rados.0.smithi165.stdout:343: oid 49 updating version 0 to 102 2024-04-20T01:48:21.715 INFO:tasks.rados.rados.0.smithi165.stdout:343: oid 49 updating version 102 to 103 2024-04-20T01:48:21.715 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 49 v 103 (ObjNum 148 snap 25 seq_num 148) dirty exists 2024-04-20T01:48:21.715 INFO:tasks.rados.rados.0.smithi165.stdout:343: left oid 49 (ObjNum 148 snap 25 seq_num 148) 2024-04-20T01:48:21.715 INFO:tasks.rados.rados.0.smithi165.stdout:344: finishing copy_from racing read to smithi16579500-39 2024-04-20T01:48:21.716 INFO:tasks.rados.rados.0.smithi165.stdout:344: finishing copy_from to smithi16579500-39 2024-04-20T01:48:21.716 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 39 v 24 (ObjNum 105 snap 16 seq_num 105) dirty exists 2024-04-20T01:48:21.725 INFO:tasks.rados.rados.0.smithi165.stdout:347: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-20T01:48:21.740 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:21.741 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-04-20T01:48:21.741 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.741 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s4 80/30 ECSubWrite(tid=70, reqid=osd.3.0:69, at_version=0'0, trim_to=0'0, roll_forward_to=80'51))", 2024-04-20T01:48:21.741 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:21.752888+0000", 2024-04-20T01:48:21.741 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.0028918099999999999, 2024-04-20T01:48:21.741 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0029094680000000001, 2024-04-20T01:48:21.741 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.741 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:21.741 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.741 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.741 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.741 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.752888+0000", 2024-04-20T01:48:21.742 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.742 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.742 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.742 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.742 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.752888+0000", 2024-04-20T01:48:21.742 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.742 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.742 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.742 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.742 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.752888+0000", 2024-04-20T01:48:21.742 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:21.742 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.742 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.743 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.743 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.752894+0000", 2024-04-20T01:48:21.743 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.3080000000000004e-06 2024-04-20T01:48:21.743 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.743 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.743 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.743 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.752899+0000", 2024-04-20T01:48:21.743 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.1109999999999999e-06 2024-04-20T01:48:21.743 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.743 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.743 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.743 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.752904+0000", 2024-04-20T01:48:21.743 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.0839999999999999e-06 2024-04-20T01:48:21.743 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.744 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.744 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:21.744 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.752985+0000", 2024-04-20T01:48:21.744 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.0683e-05 2024-04-20T01:48:21.744 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.744 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.744 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:21.744 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.752995+0000", 2024-04-20T01:48:21.744 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7429999999999997e-06 2024-04-20T01:48:21.744 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.744 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.744 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.744 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.744 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.744 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s5 80/30 ECSubWrite(tid=311, reqid=osd.4.0:309, at_version=0'0, trim_to=0'0, roll_forward_to=80'32))", 2024-04-20T01:48:21.745 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:21.755805+0000", 2024-04-20T01:48:21.745 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 4294967295.9999747, 2024-04-20T01:48:21.745 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.2739999999999999e-05, 2024-04-20T01:48:21.745 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:21.745 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "queued for pg", 2024-04-20T01:48:21.745 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:21.745 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.745 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:21.745 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.755805+0000", 2024-04-20T01:48:21.745 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.745 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.745 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.745 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:21.745 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.755805+0000", 2024-04-20T01:48:21.746 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:21.746 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.746 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.746 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:21.746 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.755804+0000", 2024-04-20T01:48:21.746 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:21.746 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.746 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.746 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:21.746 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.755815+0000", 2024-04-20T01:48:21.746 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0681e-05 2024-04-20T01:48:21.746 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.746 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.746 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:21.747 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.755828+0000", 2024-04-20T01:48:21.747 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3196999999999999e-05 2024-04-20T01:48:21.747 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:21.747 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:21.747 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:21.747 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.755837+0000", 2024-04-20T01:48:21.747 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.9309999999999997e-06 2024-04-20T01:48:21.747 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.747 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:21.747 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.747 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:21.747 INFO:teuthology.orchestra.run.smithi121.stdout: ], 2024-04-20T01:48:21.747 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 2 2024-04-20T01:48:21.747 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:21.748 INFO:tasks.rados.rados.0.smithi165.stdout:346: finishing rollback tid 0 to smithi16579500-40 2024-04-20T01:48:21.748 INFO:tasks.rados.rados.0.smithi165.stdout:346: finishing rollback tid 1 to smithi16579500-40 2024-04-20T01:48:21.748 INFO:tasks.rados.rados.0.smithi165.stdout:346: finishing rollback tid 2 to smithi16579500-40 2024-04-20T01:48:21.748 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 40 v 24 (ObjNum 102 snap 16 seq_num 102) dirty exists 2024-04-20T01:48:21.752 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-20T01:48:21.805 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:21.819+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:21.906 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:21.921+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:21.921 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:21.921 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:21.921 INFO:teuthology.orchestra.run.smithi121.stdout: "complaint_time": 30, 2024-04-20T01:48:21.921 INFO:teuthology.orchestra.run.smithi121.stdout: "num_blocked_ops": 0 2024-04-20T01:48:21.921 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:21.931 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-20T01:48:22.007 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:22.021+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:22.061 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:22.061 INFO:teuthology.orchestra.run.smithi121.stdout: "size": 20, 2024-04-20T01:48:22.061 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 600, 2024-04-20T01:48:22.061 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-04-20T01:48:22.061 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.061 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:287 3.8s0 3:12b7be30:::smithi16579500-11:head [read 0~180224,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:22.061 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.806248+0000", 2024-04-20T01:48:22.061 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 38.268234063000001, 2024-04-20T01:48:22.061 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0084388380000000006, 2024-04-20T01:48:22.061 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.061 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:22.061 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:22.061 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:22.062 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:22.062 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 287 2024-04-20T01:48:22.062 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.062 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.062 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.062 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.062 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806248+0000", 2024-04-20T01:48:22.062 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.062 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.062 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.062 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.062 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806248+0000", 2024-04-20T01:48:22.062 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.062 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.063 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.063 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.063 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806247+0000", 2024-04-20T01:48:22.063 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:22.063 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.063 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.063 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.063 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806251+0000", 2024-04-20T01:48:22.063 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7019999999999999e-06 2024-04-20T01:48:22.063 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.063 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.063 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.063 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806261+0000", 2024-04-20T01:48:22.063 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7839999999999998e-06 2024-04-20T01:48:22.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806265+0000", 2024-04-20T01:48:22.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.3599999999999998e-06 2024-04-20T01:48:22.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806370+0000", 2024-04-20T01:48:22.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00010491 2024-04-20T01:48:22.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:22.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.806375+0000", 2024-04-20T01:48:22.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.8110000000000002e-06 2024-04-20T01:48:22.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.810963+0000", 2024-04-20T01:48:22.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0045881790000000004 2024-04-20T01:48:22.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:22.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.811042+0000", 2024-04-20T01:48:22.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.8571000000000004e-05 2024-04-20T01:48:22.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.814686+0000", 2024-04-20T01:48:22.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0036447900000000002 2024-04-20T01:48:22.065 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.066 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.066 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.066 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.066 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:307 3.8s0 3:16aac79f:::smithi16579500-14:head [copy-from ver 2] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:22.066 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.823600+0000", 2024-04-20T01:48:22.066 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 38.250881286999999, 2024-04-20T01:48:22.066 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.016644029000000001, 2024-04-20T01:48:22.066 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.066 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:22.066 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:22.066 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:22.066 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:22.066 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 307 2024-04-20T01:48:22.067 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.067 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.067 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.067 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.067 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823600+0000", 2024-04-20T01:48:22.067 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.067 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.067 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.067 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.067 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823600+0000", 2024-04-20T01:48:22.067 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.067 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.067 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.067 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.067 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823598+0000", 2024-04-20T01:48:22.067 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:22.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.068 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.068 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.068 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823606+0000", 2024-04-20T01:48:22.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.2629999999999997e-06 2024-04-20T01:48:22.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.068 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.068 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.068 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823621+0000", 2024-04-20T01:48:22.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5373999999999999e-05 2024-04-20T01:48:22.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.068 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.068 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.068 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823635+0000", 2024-04-20T01:48:22.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3888e-05 2024-04-20T01:48:22.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.069 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823683+0000", 2024-04-20T01:48:22.069 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.8455000000000003e-05 2024-04-20T01:48:22.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.069 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:22.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823792+0000", 2024-04-20T01:48:22.069 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000108156 2024-04-20T01:48:22.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.069 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.830048+0000", 2024-04-20T01:48:22.069 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0062565729999999996 2024-04-20T01:48:22.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.831286+0000", 2024-04-20T01:48:22.070 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0012377829999999999 2024-04-20T01:48:22.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:22.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.840196+0000", 2024-04-20T01:48:22.070 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0089103129999999996 2024-04-20T01:48:22.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.840244+0000", 2024-04-20T01:48:22.070 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.8154e-05 2024-04-20T01:48:22.070 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.071 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.071 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.071 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.071 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.071 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:311 3.8s0 3:1d169b1d:::smithi16579500-9:head [append 0~376832] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:22.071 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.920919+0000", 2024-04-20T01:48:22.071 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 38.153562207999997, 2024-04-20T01:48:22.071 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.009305476, 2024-04-20T01:48:22.071 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.071 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:22.071 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:22.071 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:22.071 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:22.071 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 311 2024-04-20T01:48:22.072 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.072 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.072 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.072 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.072 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.920919+0000", 2024-04-20T01:48:22.072 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.072 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.072 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.072 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.072 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.920919+0000", 2024-04-20T01:48:22.072 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.072 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.072 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.072 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.072 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.920918+0000", 2024-04-20T01:48:22.072 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:22.072 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.073 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.073 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921528+0000", 2024-04-20T01:48:22.073 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00060985100000000001 2024-04-20T01:48:22.073 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.073 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.073 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921543+0000", 2024-04-20T01:48:22.073 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4302e-05 2024-04-20T01:48:22.073 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.073 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.073 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921557+0000", 2024-04-20T01:48:22.073 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4416999999999999e-05 2024-04-20T01:48:22.073 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.074 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921979+0000", 2024-04-20T01:48:22.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00042189099999999998 2024-04-20T01:48:22.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:22.074 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.922074+0000", 2024-04-20T01:48:22.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4798999999999998e-05 2024-04-20T01:48:22.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.074 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.922634+0000", 2024-04-20T01:48:22.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00056005300000000003 2024-04-20T01:48:22.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.075 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923907+0000", 2024-04-20T01:48:22.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0012733239999999999 2024-04-20T01:48:22.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.075 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:22.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930149+0000", 2024-04-20T01:48:22.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00624202 2024-04-20T01:48:22.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.075 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930225+0000", 2024-04-20T01:48:22.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.5860999999999998e-05 2024-04-20T01:48:22.076 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.076 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.076 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.076 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.076 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.076 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:312 3.8s0 3:1d169b1d:::smithi16579500-9:head [setxattr _header (55) in=7b] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:22.076 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.921568+0000", 2024-04-20T01:48:22.076 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 38.152913763999997, 2024-04-20T01:48:22.076 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0088119310000000003, 2024-04-20T01:48:22.076 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.076 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:22.076 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:22.076 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:22.076 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:22.076 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 312 2024-04-20T01:48:22.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.077 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.077 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.077 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921568+0000", 2024-04-20T01:48:22.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.077 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.077 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921568+0000", 2024-04-20T01:48:22.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.077 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.078 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921567+0000", 2024-04-20T01:48:22.078 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:22.078 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.078 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.078 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.078 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921571+0000", 2024-04-20T01:48:22.078 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.6389999999999998e-06 2024-04-20T01:48:22.078 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.078 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.078 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.078 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921577+0000", 2024-04-20T01:48:22.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.7189999999999998e-06 2024-04-20T01:48:22.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.079 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921580+0000", 2024-04-20T01:48:22.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.5810000000000001e-06 2024-04-20T01:48:22.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.079 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923124+0000", 2024-04-20T01:48:22.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0015439360000000001 2024-04-20T01:48:22.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.079 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:22.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923191+0000", 2024-04-20T01:48:22.080 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.6507000000000005e-05 2024-04-20T01:48:22.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.080 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.080 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923488+0000", 2024-04-20T01:48:22.080 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00029770699999999997 2024-04-20T01:48:22.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.080 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.080 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.924427+0000", 2024-04-20T01:48:22.080 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000938225 2024-04-20T01:48:22.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.080 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:22.080 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930309+0000", 2024-04-20T01:48:22.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0058824990000000002 2024-04-20T01:48:22.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.081 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.081 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930380+0000", 2024-04-20T01:48:22.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0779999999999997e-05 2024-04-20T01:48:22.081 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.081 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.081 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.081 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:313 3.8s0 3:1d169b1d:::smithi16579500-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:22.081 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.921593+0000", 2024-04-20T01:48:22.081 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 38.152888629000003, 2024-04-20T01:48:22.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0095502929999999996, 2024-04-20T01:48:22.082 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.082 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:22.082 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:22.082 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:22.082 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:22.082 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 313 2024-04-20T01:48:22.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.082 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.082 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.082 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.082 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921593+0000", 2024-04-20T01:48:22.082 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.082 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.083 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921593+0000", 2024-04-20T01:48:22.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.083 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921593+0000", 2024-04-20T01:48:22.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:22.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.083 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921596+0000", 2024-04-20T01:48:22.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.089e-06 2024-04-20T01:48:22.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.084 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921601+0000", 2024-04-20T01:48:22.084 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.8799999999999999e-06 2024-04-20T01:48:22.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.084 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921604+0000", 2024-04-20T01:48:22.084 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.36e-06 2024-04-20T01:48:22.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.084 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923831+0000", 2024-04-20T01:48:22.084 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0022266479999999999 2024-04-20T01:48:22.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:22.085 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923898+0000", 2024-04-20T01:48:22.085 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.7391000000000006e-05 2024-04-20T01:48:22.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.085 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.085 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.085 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930413+0000", 2024-04-20T01:48:22.085 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.006514382 2024-04-20T01:48:22.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.085 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.085 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:22.085 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930468+0000", 2024-04-20T01:48:22.085 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.5923e-05 2024-04-20T01:48:22.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.085 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.085 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.086 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.931143+0000", 2024-04-20T01:48:22.086 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00067489999999999998 2024-04-20T01:48:22.086 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.086 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.086 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.086 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.086 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.086 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:314 3.8s0 3:1746945e:::smithi16579500-24:head [copy-from ver 4] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:22.086 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.921607+0000", 2024-04-20T01:48:22.086 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 38.152875033999997, 2024-04-20T01:48:22.086 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013830053, 2024-04-20T01:48:22.086 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.086 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:22.086 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:22.087 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:22.087 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:22.087 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 314 2024-04-20T01:48:22.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.087 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.087 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.087 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.087 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921607+0000", 2024-04-20T01:48:22.087 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.087 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.087 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.087 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921607+0000", 2024-04-20T01:48:22.087 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.088 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.088 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.088 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.088 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921606+0000", 2024-04-20T01:48:22.088 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:22.088 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.088 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.088 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.088 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921609+0000", 2024-04-20T01:48:22.088 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0999999999999998e-06 2024-04-20T01:48:22.088 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.088 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.088 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.088 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921612+0000", 2024-04-20T01:48:22.088 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.4089999999999999e-06 2024-04-20T01:48:22.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.089 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.089 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.089 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921614+0000", 2024-04-20T01:48:22.089 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5610000000000001e-06 2024-04-20T01:48:22.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.089 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.089 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.089 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923919+0000", 2024-04-20T01:48:22.089 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0023049809999999998 2024-04-20T01:48:22.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.089 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.089 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:22.089 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923973+0000", 2024-04-20T01:48:22.089 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3532999999999997e-05 2024-04-20T01:48:22.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.090 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.090 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.090 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.928709+0000", 2024-04-20T01:48:22.090 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0047359979999999999 2024-04-20T01:48:22.090 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.090 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.090 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.090 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930393+0000", 2024-04-20T01:48:22.090 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0016838459999999999 2024-04-20T01:48:22.090 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.090 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.090 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:22.090 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.935394+0000", 2024-04-20T01:48:22.090 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0050015779999999996 2024-04-20T01:48:22.091 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.091 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.091 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.091 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.935437+0000", 2024-04-20T01:48:22.091 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2289000000000001e-05 2024-04-20T01:48:22.091 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.091 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.091 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.091 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.091 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.091 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:336 3.8s0 3:1746945e:::smithi16579500-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:22.091 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.973134+0000", 2024-04-20T01:48:22.091 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 38.101348066, 2024-04-20T01:48:22.091 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010536856000000001, 2024-04-20T01:48:22.091 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.092 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:22.092 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:22.092 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:22.092 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:22.092 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 336 2024-04-20T01:48:22.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.092 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.092 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.092 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.092 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973134+0000", 2024-04-20T01:48:22.092 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.092 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.092 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.093 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973134+0000", 2024-04-20T01:48:22.093 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.093 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.093 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.093 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.093 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973133+0000", 2024-04-20T01:48:22.093 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:22.093 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.093 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.093 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.093 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973135+0000", 2024-04-20T01:48:22.093 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5269999999999999e-06 2024-04-20T01:48:22.093 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.093 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.093 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.093 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973139+0000", 2024-04-20T01:48:22.094 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.3749999999999996e-06 2024-04-20T01:48:22.094 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.094 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.094 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.094 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973142+0000", 2024-04-20T01:48:22.094 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7800000000000001e-06 2024-04-20T01:48:22.094 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.094 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.094 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.094 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.978050+0000", 2024-04-20T01:48:22.094 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0049085049999999996 2024-04-20T01:48:22.094 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.094 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.094 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:22.095 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.978106+0000", 2024-04-20T01:48:22.095 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.5648e-05 2024-04-20T01:48:22.095 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.095 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.095 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.095 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.980649+0000", 2024-04-20T01:48:22.095 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0025426630000000001 2024-04-20T01:48:22.095 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.095 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.095 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:22.095 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.980701+0000", 2024-04-20T01:48:22.095 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.1907999999999998e-05 2024-04-20T01:48:22.095 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.095 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.095 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.096 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.983670+0000", 2024-04-20T01:48:22.096 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0029697590000000002 2024-04-20T01:48:22.096 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.096 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.096 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.096 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.096 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.096 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s4 53/30 ECSubWrite(tid=34, reqid=client.4643.0:445, at_version=53'23, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:22.096 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:54.722907+0000", 2024-04-20T01:48:22.096 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 27.351574752000001, 2024-04-20T01:48:22.096 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0084068279999999999, 2024-04-20T01:48:22.096 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.096 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.096 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.097 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.097 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.097 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.722907+0000", 2024-04-20T01:48:22.097 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.097 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.097 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.097 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.097 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.722907+0000", 2024-04-20T01:48:22.097 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.097 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.097 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.097 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.097 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.722896+0000", 2024-04-20T01:48:22.097 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999895 2024-04-20T01:48:22.097 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.098 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.098 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.098 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723127+0000", 2024-04-20T01:48:22.098 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00023036199999999999 2024-04-20T01:48:22.098 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.098 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.098 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.098 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723141+0000", 2024-04-20T01:48:22.098 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4174e-05 2024-04-20T01:48:22.098 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.098 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.098 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.098 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723150+0000", 2024-04-20T01:48:22.098 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3090000000000001e-06 2024-04-20T01:48:22.099 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.099 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.099 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.099 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723203+0000", 2024-04-20T01:48:22.099 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.2431e-05 2024-04-20T01:48:22.099 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.099 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.099 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.099 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.723212+0000", 2024-04-20T01:48:22.099 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3239999999999992e-06 2024-04-20T01:48:22.099 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.099 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.099 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.099 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.731289+0000", 2024-04-20T01:48:22.100 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0080767059999999995 2024-04-20T01:48:22.100 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.100 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.100 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.100 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:54.731314+0000", 2024-04-20T01:48:22.100 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5088000000000001e-05 2024-04-20T01:48:22.100 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.100 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.100 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.100 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.100 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.100 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:487 3.8s0 3:16aac79f:::smithi16579500-14:head [append 0~425984] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:22.100 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:59.760614+0000", 2024-04-20T01:48:22.100 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 22.31386805, 2024-04-20T01:48:22.101 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010465621, 2024-04-20T01:48:22.101 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.101 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:22.101 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:22.101 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:22.101 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:22.101 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 487 2024-04-20T01:48:22.101 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.101 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.101 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.101 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.101 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.760614+0000", 2024-04-20T01:48:22.101 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.101 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.102 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.102 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.102 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.760614+0000", 2024-04-20T01:48:22.102 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.102 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.102 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.102 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.102 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.760611+0000", 2024-04-20T01:48:22.102 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999971 2024-04-20T01:48:22.102 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.102 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.102 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.102 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.761151+0000", 2024-04-20T01:48:22.102 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00054006400000000004 2024-04-20T01:48:22.103 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.103 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.103 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.103 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.761173+0000", 2024-04-20T01:48:22.103 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1926999999999999e-05 2024-04-20T01:48:22.103 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.103 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.103 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.103 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.761202+0000", 2024-04-20T01:48:22.103 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9286e-05 2024-04-20T01:48:22.103 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.103 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.103 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.103 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.761276+0000", 2024-04-20T01:48:22.103 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.4319000000000005e-05 2024-04-20T01:48:22.103 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.104 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.104 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op must wait for map", 2024-04-20T01:48:22.104 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.761289+0000", 2024-04-20T01:48:22.104 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.239e-05 2024-04-20T01:48:22.104 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.104 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.104 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.104 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.763247+0000", 2024-04-20T01:48:22.104 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0019584989999999998 2024-04-20T01:48:22.104 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.104 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.104 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:22.104 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.763331+0000", 2024-04-20T01:48:22.104 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.3320999999999997e-05 2024-04-20T01:48:22.105 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.105 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.105 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.105 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.764297+0000", 2024-04-20T01:48:22.105 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00096590700000000001 2024-04-20T01:48:22.105 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.105 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.105 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.105 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768005+0000", 2024-04-20T01:48:22.105 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0037084940000000001 2024-04-20T01:48:22.105 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.105 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.105 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:22.106 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.770983+0000", 2024-04-20T01:48:22.106 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0029780129999999998 2024-04-20T01:48:22.106 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.106 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.106 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.106 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.771079+0000", 2024-04-20T01:48:22.106 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6250999999999997e-05 2024-04-20T01:48:22.106 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.106 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.106 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.106 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.106 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.106 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:489 3.8s0 3:16aac79f:::smithi16579500-14:head [append 0~376832] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:22.106 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:59.767481+0000", 2024-04-20T01:48:22.107 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 22.307000379000002, 2024-04-20T01:48:22.107 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011576091, 2024-04-20T01:48:22.107 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.107 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:22.107 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:22.107 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:22.107 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:22.107 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 489 2024-04-20T01:48:22.107 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.107 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.107 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.107 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.107 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.767481+0000", 2024-04-20T01:48:22.107 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.108 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.108 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.108 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.108 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.767481+0000", 2024-04-20T01:48:22.108 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.108 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.108 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.108 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.108 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.767480+0000", 2024-04-20T01:48:22.108 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:22.108 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.108 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.108 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.108 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768144+0000", 2024-04-20T01:48:22.108 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00066418499999999997 2024-04-20T01:48:22.109 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.109 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.109 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.109 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768157+0000", 2024-04-20T01:48:22.109 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2281e-05 2024-04-20T01:48:22.109 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.109 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.109 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.109 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768166+0000", 2024-04-20T01:48:22.109 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2629999999999992e-06 2024-04-20T01:48:22.109 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.109 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.109 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.109 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768215+0000", 2024-04-20T01:48:22.110 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.9292000000000002e-05 2024-04-20T01:48:22.110 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.110 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.110 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:22.110 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768322+0000", 2024-04-20T01:48:22.110 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000106461 2024-04-20T01:48:22.110 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.110 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.110 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.110 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.769186+0000", 2024-04-20T01:48:22.110 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00086420900000000001 2024-04-20T01:48:22.110 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.110 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.110 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.111 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.771140+0000", 2024-04-20T01:48:22.111 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0019545890000000001 2024-04-20T01:48:22.111 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.111 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.111 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:22.111 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779011+0000", 2024-04-20T01:48:22.111 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0078710150000000003 2024-04-20T01:48:22.111 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.111 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.111 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.111 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779057+0000", 2024-04-20T01:48:22.111 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5918000000000003e-05 2024-04-20T01:48:22.111 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.111 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.111 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.112 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.112 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.112 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:490 3.8s0 3:16aac79f:::smithi16579500-14:head [setxattr _header (57) in=7b] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:22.112 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:59.768179+0000", 2024-04-20T01:48:22.112 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 22.306302486, 2024-04-20T01:48:22.112 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011076167, 2024-04-20T01:48:22.112 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.112 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:22.112 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:22.112 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:22.112 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:22.112 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 490 2024-04-20T01:48:22.112 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.112 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.112 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.113 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.113 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768179+0000", 2024-04-20T01:48:22.113 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.113 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.113 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.113 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.113 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768179+0000", 2024-04-20T01:48:22.113 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.113 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.113 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.113 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.113 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768178+0000", 2024-04-20T01:48:22.113 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:22.113 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.113 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.114 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.114 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768183+0000", 2024-04-20T01:48:22.114 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.288e-06 2024-04-20T01:48:22.114 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.114 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.114 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.114 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768191+0000", 2024-04-20T01:48:22.114 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.5850000000000002e-06 2024-04-20T01:48:22.114 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.114 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.114 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.114 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768197+0000", 2024-04-20T01:48:22.114 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.9270000000000002e-06 2024-04-20T01:48:22.114 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.115 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.115 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.115 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.769809+0000", 2024-04-20T01:48:22.115 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001611913 2024-04-20T01:48:22.115 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.115 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.115 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:22.115 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.769930+0000", 2024-04-20T01:48:22.115 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000121304 2024-04-20T01:48:22.115 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.115 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.115 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.115 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.770462+0000", 2024-04-20T01:48:22.115 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00053196700000000005 2024-04-20T01:48:22.116 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.116 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.116 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.116 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.771310+0000", 2024-04-20T01:48:22.116 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00084813200000000003 2024-04-20T01:48:22.116 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.116 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.116 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:22.116 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779156+0000", 2024-04-20T01:48:22.116 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0078461580000000006 2024-04-20T01:48:22.116 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.116 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.116 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.116 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779255+0000", 2024-04-20T01:48:22.117 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.8944999999999995e-05 2024-04-20T01:48:22.117 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.117 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.117 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.117 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.117 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.117 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:491 3.8s0 3:16aac79f:::smithi16579500-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:22.117 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:59.768204+0000", 2024-04-20T01:48:22.117 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 22.306277396999999, 2024-04-20T01:48:22.117 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.012786562, 2024-04-20T01:48:22.117 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.117 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:22.117 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:22.117 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:22.118 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:22.118 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 491 2024-04-20T01:48:22.118 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.118 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.118 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.118 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.118 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768204+0000", 2024-04-20T01:48:22.118 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.118 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.118 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.118 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.118 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768204+0000", 2024-04-20T01:48:22.118 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.118 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.118 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.119 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.119 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768204+0000", 2024-04-20T01:48:22.119 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:22.119 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.119 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.119 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.119 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768207+0000", 2024-04-20T01:48:22.119 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.4180000000000001e-06 2024-04-20T01:48:22.119 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.119 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.119 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.119 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768215+0000", 2024-04-20T01:48:22.119 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.6669999999999996e-06 2024-04-20T01:48:22.119 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.119 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.120 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.120 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768220+0000", 2024-04-20T01:48:22.120 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3149999999999999e-06 2024-04-20T01:48:22.120 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.120 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.120 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.120 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.771203+0000", 2024-04-20T01:48:22.120 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.002982434 2024-04-20T01:48:22.120 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.120 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.120 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:22.120 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.771299+0000", 2024-04-20T01:48:22.120 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6111000000000002e-05 2024-04-20T01:48:22.120 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.120 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.121 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.121 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779629+0000", 2024-04-20T01:48:22.121 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0083302529999999993 2024-04-20T01:48:22.121 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.121 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.121 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:22.121 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779693+0000", 2024-04-20T01:48:22.121 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.4424999999999994e-05 2024-04-20T01:48:22.121 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.121 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.121 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.121 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.780991+0000", 2024-04-20T01:48:22.121 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0012974390000000001 2024-04-20T01:48:22.121 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.121 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.122 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.122 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.122 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.122 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s4 75/30 ECSubWrite(tid=55, reqid=client.4643.0:781, at_version=75'41, trim_to=0'0, roll_forward_to=66'39))", 2024-04-20T01:48:22.122 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.735057+0000", 2024-04-20T01:48:22.122 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.3394246140000003, 2024-04-20T01:48:22.122 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.012883867, 2024-04-20T01:48:22.122 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.122 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.122 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.122 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.122 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.122 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735057+0000", 2024-04-20T01:48:22.122 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.123 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.123 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.123 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.123 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735057+0000", 2024-04-20T01:48:22.123 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.123 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.123 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.123 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.123 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735056+0000", 2024-04-20T01:48:22.123 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:22.123 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.123 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.123 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.123 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735367+0000", 2024-04-20T01:48:22.123 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00031039199999999999 2024-04-20T01:48:22.124 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.124 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.124 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.124 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735378+0000", 2024-04-20T01:48:22.124 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1389000000000001e-05 2024-04-20T01:48:22.124 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.124 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.124 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.124 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735385+0000", 2024-04-20T01:48:22.124 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.9759999999999998e-06 2024-04-20T01:48:22.124 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.124 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.124 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.124 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735429+0000", 2024-04-20T01:48:22.125 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.3699000000000003e-05 2024-04-20T01:48:22.125 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.125 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.125 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.125 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735437+0000", 2024-04-20T01:48:22.125 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.7440000000000007e-06 2024-04-20T01:48:22.125 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.125 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.125 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.125 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747914+0000", 2024-04-20T01:48:22.125 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.012477162 2024-04-20T01:48:22.125 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.125 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.125 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.125 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747941+0000", 2024-04-20T01:48:22.125 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7107e-05 2024-04-20T01:48:22.125 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.126 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.126 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.126 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.126 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.126 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.1s3 75/30 ECSubWrite(tid=257, reqid=osd.4.0:253, at_version=0'0, trim_to=0'0, roll_forward_to=75'37))", 2024-04-20T01:48:22.126 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.737287+0000", 2024-04-20T01:48:22.126 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.3371942299999997, 2024-04-20T01:48:22.126 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00931126, 2024-04-20T01:48:22.126 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.126 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.126 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.126 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.126 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.126 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737287+0000", 2024-04-20T01:48:22.127 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.127 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.127 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.127 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.127 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737287+0000", 2024-04-20T01:48:22.127 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.127 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.127 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.127 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.127 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737287+0000", 2024-04-20T01:48:22.127 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:22.127 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.127 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.127 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.127 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737307+0000", 2024-04-20T01:48:22.128 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0547000000000002e-05 2024-04-20T01:48:22.128 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.128 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.128 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.128 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737320+0000", 2024-04-20T01:48:22.128 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3112000000000001e-05 2024-04-20T01:48:22.128 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.128 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.128 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.128 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737339+0000", 2024-04-20T01:48:22.128 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8289999999999999e-05 2024-04-20T01:48:22.128 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.128 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.128 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.129 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737420+0000", 2024-04-20T01:48:22.129 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.106e-05 2024-04-20T01:48:22.129 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.129 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.129 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.129 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737431+0000", 2024-04-20T01:48:22.129 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1289e-05 2024-04-20T01:48:22.129 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.129 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.129 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.129 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.746568+0000", 2024-04-20T01:48:22.129 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0091374909999999993 2024-04-20T01:48:22.129 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.129 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.129 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.130 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.746599+0000", 2024-04-20T01:48:22.130 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0335999999999999e-05 2024-04-20T01:48:22.130 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.130 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.130 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.130 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.130 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.130 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.cs6 75/30 ECSubWrite(tid=114, reqid=osd.14.0:113, at_version=0'0, trim_to=0'0, roll_forward_to=75'93))", 2024-04-20T01:48:22.130 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.738324+0000", 2024-04-20T01:48:22.130 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.3361572009999998, 2024-04-20T01:48:22.130 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0095662660000000004, 2024-04-20T01:48:22.130 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.130 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.130 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.130 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.131 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.131 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738324+0000", 2024-04-20T01:48:22.131 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.131 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.131 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.131 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.131 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738324+0000", 2024-04-20T01:48:22.131 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.131 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.131 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.131 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.131 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738324+0000", 2024-04-20T01:48:22.131 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:22.131 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.132 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.132 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.132 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738337+0000", 2024-04-20T01:48:22.132 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3596e-05 2024-04-20T01:48:22.132 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.132 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.132 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.132 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738349+0000", 2024-04-20T01:48:22.132 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1857e-05 2024-04-20T01:48:22.132 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.132 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.132 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.132 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738358+0000", 2024-04-20T01:48:22.132 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4060000000000005e-06 2024-04-20T01:48:22.132 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.132 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.133 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.133 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738425+0000", 2024-04-20T01:48:22.133 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.7916000000000005e-05 2024-04-20T01:48:22.133 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.133 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.133 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.133 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738442+0000", 2024-04-20T01:48:22.133 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6872999999999999e-05 2024-04-20T01:48:22.133 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.133 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.133 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.133 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747865+0000", 2024-04-20T01:48:22.133 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0094223810000000005 2024-04-20T01:48:22.133 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.134 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.134 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.134 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747891+0000", 2024-04-20T01:48:22.134 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6061999999999999e-05 2024-04-20T01:48:22.134 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.134 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.134 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.134 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.134 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.134 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 75/30 ECSubWriteReply(tid=102, last_complete=75'82, committed=1, applied=1))", 2024-04-20T01:48:22.134 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.738410+0000", 2024-04-20T01:48:22.134 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.3360712210000001, 2024-04-20T01:48:22.134 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0088662340000000006, 2024-04-20T01:48:22.134 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.134 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.135 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.135 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.135 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.135 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738410+0000", 2024-04-20T01:48:22.135 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.135 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.135 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.135 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.135 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738410+0000", 2024-04-20T01:48:22.135 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.135 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.135 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.135 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.135 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738410+0000", 2024-04-20T01:48:22.135 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:22.136 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.136 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.136 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.136 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738415+0000", 2024-04-20T01:48:22.136 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.592e-06 2024-04-20T01:48:22.136 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.136 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.136 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.136 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738428+0000", 2024-04-20T01:48:22.136 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2286000000000001e-05 2024-04-20T01:48:22.136 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.136 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.136 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.136 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738462+0000", 2024-04-20T01:48:22.136 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.4060999999999998e-05 2024-04-20T01:48:22.137 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.137 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.137 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.137 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747263+0000", 2024-04-20T01:48:22.137 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0088014679999999998 2024-04-20T01:48:22.137 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.137 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.137 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.137 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747277+0000", 2024-04-20T01:48:22.137 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3614999999999999e-05 2024-04-20T01:48:22.137 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.137 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.137 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.137 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.137 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.138 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s5 75/30 ECSubWrite(tid=258, reqid=osd.4.0:258, at_version=75'21, trim_to=0'0, roll_forward_to=68'19))", 2024-04-20T01:48:22.138 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.738907+0000", 2024-04-20T01:48:22.138 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.3355744129999998, 2024-04-20T01:48:22.138 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010750793, 2024-04-20T01:48:22.138 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.138 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.138 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.138 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.138 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.138 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738907+0000", 2024-04-20T01:48:22.138 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.138 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.138 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.138 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.138 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738907+0000", 2024-04-20T01:48:22.139 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.139 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.139 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.139 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.139 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738907+0000", 2024-04-20T01:48:22.139 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:22.139 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.139 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.139 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.139 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738926+0000", 2024-04-20T01:48:22.139 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9032e-05 2024-04-20T01:48:22.139 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.139 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.139 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.139 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738938+0000", 2024-04-20T01:48:22.140 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2802999999999999e-05 2024-04-20T01:48:22.140 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.140 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.140 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.140 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738948+0000", 2024-04-20T01:48:22.140 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.0650000000000005e-06 2024-04-20T01:48:22.140 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.140 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.140 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.140 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739017+0000", 2024-04-20T01:48:22.140 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.9621000000000003e-05 2024-04-20T01:48:22.140 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.140 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.140 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.140 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739028+0000", 2024-04-20T01:48:22.141 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0957e-05 2024-04-20T01:48:22.141 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.141 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.141 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.141 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.749625+0000", 2024-04-20T01:48:22.141 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010597031999999999 2024-04-20T01:48:22.141 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.141 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.141 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.141 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.749658+0000", 2024-04-20T01:48:22.141 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.2947999999999997e-05 2024-04-20T01:48:22.141 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.141 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.141 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.141 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.142 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.142 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.cs6 75/30 ECSubWrite(tid=115, reqid=osd.14.0:115, at_version=75'95, trim_to=0'0, roll_forward_to=75'93))", 2024-04-20T01:48:22.142 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.739080+0000", 2024-04-20T01:48:22.142 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.3354021700000001, 2024-04-20T01:48:22.142 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011132998, 2024-04-20T01:48:22.142 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.142 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.142 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.142 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.142 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.142 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739080+0000", 2024-04-20T01:48:22.142 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.142 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.142 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.142 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.143 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739080+0000", 2024-04-20T01:48:22.143 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.143 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.143 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.143 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.143 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739079+0000", 2024-04-20T01:48:22.143 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:22.143 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.143 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.143 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.143 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739096+0000", 2024-04-20T01:48:22.143 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7300999999999999e-05 2024-04-20T01:48:22.143 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.143 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.144 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.144 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739123+0000", 2024-04-20T01:48:22.144 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7257999999999999e-05 2024-04-20T01:48:22.144 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.144 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.144 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.144 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739132+0000", 2024-04-20T01:48:22.144 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8859999999999993e-06 2024-04-20T01:48:22.144 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.144 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.144 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.144 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739598+0000", 2024-04-20T01:48:22.144 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00046589800000000002 2024-04-20T01:48:22.144 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.144 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.145 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.145 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739609+0000", 2024-04-20T01:48:22.145 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0737000000000001e-05 2024-04-20T01:48:22.145 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.145 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.145 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.145 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.750185+0000", 2024-04-20T01:48:22.145 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010576370999999999 2024-04-20T01:48:22.145 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.145 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.145 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.145 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.750213+0000", 2024-04-20T01:48:22.145 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7304000000000002e-05 2024-04-20T01:48:22.145 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.146 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.146 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.146 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.146 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.146 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s5 75/30 ECSubWrite(tid=259, reqid=osd.4.0:259, at_version=75'23, trim_to=0'0, roll_forward_to=68'19))", 2024-04-20T01:48:22.146 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.740102+0000", 2024-04-20T01:48:22.146 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.334379931, 2024-04-20T01:48:22.146 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011279176, 2024-04-20T01:48:22.146 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.146 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.146 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.146 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.146 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.146 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740102+0000", 2024-04-20T01:48:22.147 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.147 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.147 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.147 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.147 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740102+0000", 2024-04-20T01:48:22.147 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.147 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.147 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.147 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.147 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740101+0000", 2024-04-20T01:48:22.147 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:22.147 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.147 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.147 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.147 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740123+0000", 2024-04-20T01:48:22.148 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1708e-05 2024-04-20T01:48:22.148 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.148 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.148 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.148 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740135+0000", 2024-04-20T01:48:22.148 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1776999999999999e-05 2024-04-20T01:48:22.148 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.148 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.148 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.148 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740142+0000", 2024-04-20T01:48:22.148 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.956e-06 2024-04-20T01:48:22.148 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.148 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.148 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.148 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742932+0000", 2024-04-20T01:48:22.149 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0027900440000000002 2024-04-20T01:48:22.149 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.149 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.149 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.149 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742942+0000", 2024-04-20T01:48:22.149 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0491e-05 2024-04-20T01:48:22.149 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.149 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.149 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.149 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.751351+0000", 2024-04-20T01:48:22.149 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0084087229999999999 2024-04-20T01:48:22.149 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.149 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.149 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.150 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.751381+0000", 2024-04-20T01:48:22.150 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0173999999999999e-05 2024-04-20T01:48:22.150 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.150 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.150 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.150 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.150 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.150 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s5 80/30 ECSubWrite(tid=309, reqid=osd.4.0:309, at_version=80'32, trim_to=0'0, roll_forward_to=79'28))", 2024-04-20T01:48:22.150 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:21.742879+0000", 2024-04-20T01:48:22.150 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.33160315499999998, 2024-04-20T01:48:22.150 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0092419359999999992, 2024-04-20T01:48:22.150 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.150 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.150 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.150 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.151 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.151 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.742879+0000", 2024-04-20T01:48:22.151 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.151 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.151 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.151 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.151 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.742879+0000", 2024-04-20T01:48:22.151 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.151 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.151 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.151 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.151 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.742878+0000", 2024-04-20T01:48:22.151 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:22.152 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.152 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.152 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.152 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.742903+0000", 2024-04-20T01:48:22.152 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5403000000000001e-05 2024-04-20T01:48:22.152 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.152 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.152 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.152 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.742913+0000", 2024-04-20T01:48:22.152 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6390000000000004e-06 2024-04-20T01:48:22.152 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.152 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.152 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.152 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.742919+0000", 2024-04-20T01:48:22.152 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.1770000000000003e-06 2024-04-20T01:48:22.153 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.153 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.153 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.153 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744284+0000", 2024-04-20T01:48:22.153 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0013651189999999999 2024-04-20T01:48:22.153 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.153 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.153 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.153 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744295+0000", 2024-04-20T01:48:22.153 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0796e-05 2024-04-20T01:48:22.153 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.153 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.153 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.153 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.752095+0000", 2024-04-20T01:48:22.154 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0077997589999999999 2024-04-20T01:48:22.154 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.154 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.154 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.154 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.752120+0000", 2024-04-20T01:48:22.154 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5584000000000001e-05 2024-04-20T01:48:22.154 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.154 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.154 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.154 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.154 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.154 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:22.156 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-20T01:48:22.156 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:22.170+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:22.257 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:48:22.271+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:48:22.284 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:22.284 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:22.284 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 0 2024-04-20T01:48:22.284 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:22.294 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-20T01:48:22.358 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:48:22.372+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:48:22.422 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:22.422 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:22.422 INFO:teuthology.orchestra.run.smithi121.stdout: "complaint_time": 30, 2024-04-20T01:48:22.422 INFO:teuthology.orchestra.run.smithi121.stdout: "num_blocked_ops": 0 2024-04-20T01:48:22.422 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:22.432 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-20T01:48:22.458 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:22.473+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:22.559 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:22.574+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:22.560 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:22.560 INFO:teuthology.orchestra.run.smithi121.stdout: "size": 20, 2024-04-20T01:48:22.561 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 600, 2024-04-20T01:48:22.561 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-04-20T01:48:22.561 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.561 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.es1 47/30 ECSubWrite(tid=40, reqid=osd.11.0:39, at_version=0'0, trim_to=0'0, roll_forward_to=47'23))", 2024-04-20T01:48:22.561 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.721119+0000", 2024-04-20T01:48:22.561 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 33.853217002000001, 2024-04-20T01:48:22.561 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0069945759999999997, 2024-04-20T01:48:22.561 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.561 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.561 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.561 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.561 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.561 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721119+0000", 2024-04-20T01:48:22.561 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.562 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.562 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.562 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.562 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721119+0000", 2024-04-20T01:48:22.562 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.562 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.562 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.562 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.562 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721119+0000", 2024-04-20T01:48:22.562 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:22.562 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.562 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.562 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.562 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721128+0000", 2024-04-20T01:48:22.563 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.0620000000000007e-06 2024-04-20T01:48:22.563 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.563 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.563 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.563 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721138+0000", 2024-04-20T01:48:22.563 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0771000000000001e-05 2024-04-20T01:48:22.563 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.563 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.563 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.563 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721146+0000", 2024-04-20T01:48:22.563 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.3370000000000002e-06 2024-04-20T01:48:22.563 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.563 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.563 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.563 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721183+0000", 2024-04-20T01:48:22.564 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7283000000000002e-05 2024-04-20T01:48:22.564 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.564 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.564 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.564 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721194+0000", 2024-04-20T01:48:22.564 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1093e-05 2024-04-20T01:48:22.564 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.564 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.564 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.564 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.728091+0000", 2024-04-20T01:48:22.564 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0068972499999999997 2024-04-20T01:48:22.564 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.564 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.564 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.565 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.728114+0000", 2024-04-20T01:48:22.565 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2410000000000001e-05 2024-04-20T01:48:22.565 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.565 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.565 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.565 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.565 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.565 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.6s6 68/30 ECSubWrite(tid=20, reqid=client.4643.0:643, at_version=68'13, trim_to=0'0, roll_forward_to=68'10))", 2024-04-20T01:48:22.565 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:09.878445+0000", 2024-04-20T01:48:22.565 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 12.695891245, 2024-04-20T01:48:22.565 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00751251, 2024-04-20T01:48:22.565 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.565 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.565 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.565 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.565 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.566 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.878445+0000", 2024-04-20T01:48:22.566 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.566 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.566 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.566 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.566 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.878445+0000", 2024-04-20T01:48:22.566 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.566 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.566 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.566 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.566 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.878443+0000", 2024-04-20T01:48:22.566 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:22.566 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.566 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.567 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.567 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.878670+0000", 2024-04-20T01:48:22.567 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00022695199999999999 2024-04-20T01:48:22.567 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.567 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.567 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.567 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.878680+0000", 2024-04-20T01:48:22.567 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.217e-06 2024-04-20T01:48:22.567 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.567 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.567 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.567 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.878688+0000", 2024-04-20T01:48:22.567 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.6000000000000007e-06 2024-04-20T01:48:22.567 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.567 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.568 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.568 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.878733+0000", 2024-04-20T01:48:22.568 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.4490000000000003e-05 2024-04-20T01:48:22.568 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.568 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.568 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.568 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.878743+0000", 2024-04-20T01:48:22.568 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0294e-05 2024-04-20T01:48:22.568 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.568 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.568 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.568 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.885926+0000", 2024-04-20T01:48:22.568 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0071831669999999999 2024-04-20T01:48:22.568 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.568 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.569 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.569 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.885957+0000", 2024-04-20T01:48:22.569 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.1411999999999997e-05 2024-04-20T01:48:22.569 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.569 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.569 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.569 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.569 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.569 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.6s6 68/30 ECSubWrite(tid=22, reqid=client.4643.0:645, at_version=68'15, trim_to=0'0, roll_forward_to=68'10))", 2024-04-20T01:48:22.569 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:09.880701+0000", 2024-04-20T01:48:22.569 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 12.693634817, 2024-04-20T01:48:22.569 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0082028380000000005, 2024-04-20T01:48:22.569 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.569 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.569 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.570 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.570 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.570 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.880701+0000", 2024-04-20T01:48:22.570 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.570 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.570 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.570 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.570 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.880701+0000", 2024-04-20T01:48:22.570 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.570 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.570 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.570 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.570 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.880701+0000", 2024-04-20T01:48:22.570 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:22.570 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.571 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.571 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.571 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.880714+0000", 2024-04-20T01:48:22.571 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3468999999999999e-05 2024-04-20T01:48:22.571 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.571 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.571 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.571 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.880726+0000", 2024-04-20T01:48:22.571 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1727e-05 2024-04-20T01:48:22.571 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.571 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.571 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.571 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.880732+0000", 2024-04-20T01:48:22.571 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.3829999999999997e-06 2024-04-20T01:48:22.571 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.572 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.572 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.572 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.880785+0000", 2024-04-20T01:48:22.572 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.2932000000000001e-05 2024-04-20T01:48:22.572 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.572 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.572 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.572 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.880807+0000", 2024-04-20T01:48:22.572 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.143e-05 2024-04-20T01:48:22.572 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.572 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.572 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.572 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.888877+0000", 2024-04-20T01:48:22.572 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0080705440000000007 2024-04-20T01:48:22.572 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.573 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.573 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.573 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.888904+0000", 2024-04-20T01:48:22.573 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7033999999999999e-05 2024-04-20T01:48:22.573 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.573 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.573 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.573 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.573 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.573 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 68/30 ECSubWrite(tid=216, reqid=client.4643.0:684, at_version=68'115, trim_to=0'0, roll_forward_to=68'113))", 2024-04-20T01:48:22.573 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:10.004517+0000", 2024-04-20T01:48:22.573 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 12.569819150000001, 2024-04-20T01:48:22.573 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0078424040000000007, 2024-04-20T01:48:22.573 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.573 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.574 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.574 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.574 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.574 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.004517+0000", 2024-04-20T01:48:22.574 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.574 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.574 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.574 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.574 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.004517+0000", 2024-04-20T01:48:22.574 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.574 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.574 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.574 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.574 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.004516+0000", 2024-04-20T01:48:22.575 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:22.575 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.575 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.575 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.575 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.004540+0000", 2024-04-20T01:48:22.575 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3144999999999999e-05 2024-04-20T01:48:22.575 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.575 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.575 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.575 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.004551+0000", 2024-04-20T01:48:22.575 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1516e-05 2024-04-20T01:48:22.575 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.575 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.575 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.575 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.004557+0000", 2024-04-20T01:48:22.575 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.8080000000000001e-06 2024-04-20T01:48:22.576 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.576 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.576 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.576 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.004601+0000", 2024-04-20T01:48:22.576 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.3911e-05 2024-04-20T01:48:22.576 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.576 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.576 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.576 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.004611+0000", 2024-04-20T01:48:22.576 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0404000000000001e-05 2024-04-20T01:48:22.576 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.576 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.576 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.576 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.012340+0000", 2024-04-20T01:48:22.577 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0077283819999999998 2024-04-20T01:48:22.577 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.577 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.577 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.577 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.012359+0000", 2024-04-20T01:48:22.577 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9862999999999999e-05 2024-04-20T01:48:22.577 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.577 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.577 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.577 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.577 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.577 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.es1 70/30 ECSubWrite(tid=82, reqid=client.4643.0:728, at_version=70'61, trim_to=0'0, roll_forward_to=68'58))", 2024-04-20T01:48:22.577 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:11.821013+0000", 2024-04-20T01:48:22.577 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 10.753322847, 2024-04-20T01:48:22.577 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0073864279999999996, 2024-04-20T01:48:22.578 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.578 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.578 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.578 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.578 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.578 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.821013+0000", 2024-04-20T01:48:22.578 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.578 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.578 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.578 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.578 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.821013+0000", 2024-04-20T01:48:22.578 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.578 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.578 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.579 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.579 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.821013+0000", 2024-04-20T01:48:22.579 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:22.579 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.579 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.579 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.579 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.821466+0000", 2024-04-20T01:48:22.579 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00045382300000000001 2024-04-20T01:48:22.579 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.579 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.579 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.579 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.821482+0000", 2024-04-20T01:48:22.579 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5341000000000001e-05 2024-04-20T01:48:22.579 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.579 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.580 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.580 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.821489+0000", 2024-04-20T01:48:22.580 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.4819999999999997e-06 2024-04-20T01:48:22.580 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.580 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.580 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.580 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.821535+0000", 2024-04-20T01:48:22.580 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5583e-05 2024-04-20T01:48:22.580 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.580 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.580 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.580 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.821542+0000", 2024-04-20T01:48:22.580 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.3830000000000003e-06 2024-04-20T01:48:22.580 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.580 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.581 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.581 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.828362+0000", 2024-04-20T01:48:22.581 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0068202860000000001 2024-04-20T01:48:22.581 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.581 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.581 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.581 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.828400+0000", 2024-04-20T01:48:22.581 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7379999999999998e-05 2024-04-20T01:48:22.581 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.581 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.581 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.581 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.581 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.581 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.es1 70/30 ECSubWrite(tid=83, reqid=client.4643.0:729, at_version=70'62, trim_to=0'0, roll_forward_to=68'58))", 2024-04-20T01:48:22.581 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:11.823811+0000", 2024-04-20T01:48:22.581 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 10.750524812, 2024-04-20T01:48:22.582 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.007216582, 2024-04-20T01:48:22.582 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.582 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.582 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.582 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.582 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.582 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.823811+0000", 2024-04-20T01:48:22.582 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.582 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.582 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.582 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.582 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.823811+0000", 2024-04-20T01:48:22.582 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.582 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.583 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.583 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.583 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.823811+0000", 2024-04-20T01:48:22.583 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:22.583 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.583 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.583 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.583 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.824011+0000", 2024-04-20T01:48:22.583 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000200216 2024-04-20T01:48:22.583 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.583 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.583 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.583 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.824026+0000", 2024-04-20T01:48:22.583 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4861999999999999e-05 2024-04-20T01:48:22.583 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.583 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.584 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.584 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.824034+0000", 2024-04-20T01:48:22.584 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.6549999999999993e-06 2024-04-20T01:48:22.584 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.584 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.584 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.584 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.824091+0000", 2024-04-20T01:48:22.584 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6532e-05 2024-04-20T01:48:22.584 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.584 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.584 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.584 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.824098+0000", 2024-04-20T01:48:22.584 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0389999999999999e-06 2024-04-20T01:48:22.584 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.584 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.585 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.585 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.831007+0000", 2024-04-20T01:48:22.585 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0069089640000000001 2024-04-20T01:48:22.585 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.585 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.585 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.585 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.831028+0000", 2024-04-20T01:48:22.585 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0982e-05 2024-04-20T01:48:22.585 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.585 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.585 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.585 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.585 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.585 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.es1 70/30 ECSubWrite(tid=84, reqid=client.4643.0:730, at_version=70'63, trim_to=0'0, roll_forward_to=68'58))", 2024-04-20T01:48:22.585 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:11.826634+0000", 2024-04-20T01:48:22.586 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 10.747702361, 2024-04-20T01:48:22.586 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0073249450000000002, 2024-04-20T01:48:22.586 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.586 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.586 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.586 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.586 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.586 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.826634+0000", 2024-04-20T01:48:22.586 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.586 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.586 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.586 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.586 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.826634+0000", 2024-04-20T01:48:22.586 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.586 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.587 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.587 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.587 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.826633+0000", 2024-04-20T01:48:22.587 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:22.587 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.587 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.587 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.587 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.826728+0000", 2024-04-20T01:48:22.587 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.5006999999999993e-05 2024-04-20T01:48:22.587 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.587 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.587 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.587 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.826742+0000", 2024-04-20T01:48:22.587 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.361e-05 2024-04-20T01:48:22.588 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.588 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.588 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.588 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.826751+0000", 2024-04-20T01:48:22.588 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7529999999999996e-06 2024-04-20T01:48:22.588 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.588 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.588 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.588 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.826793+0000", 2024-04-20T01:48:22.588 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.1112999999999998e-05 2024-04-20T01:48:22.588 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.588 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.588 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.588 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.826800+0000", 2024-04-20T01:48:22.589 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.4039999999999998e-06 2024-04-20T01:48:22.589 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.589 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.589 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.589 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.833928+0000", 2024-04-20T01:48:22.589 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0071284549999999997 2024-04-20T01:48:22.589 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.589 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.589 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.589 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.833959+0000", 2024-04-20T01:48:22.589 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0426e-05 2024-04-20T01:48:22.589 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.589 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.589 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.590 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.590 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.590 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.1s2 75/30 ECSubWrite(tid=253, reqid=osd.4.0:253, at_version=75'37, trim_to=0'0, roll_forward_to=73'35))", 2024-04-20T01:48:22.590 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.725657+0000", 2024-04-20T01:48:22.590 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.8486787209999997, 2024-04-20T01:48:22.590 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0071635270000000003, 2024-04-20T01:48:22.590 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.590 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.590 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.590 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.590 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.590 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.725657+0000", 2024-04-20T01:48:22.590 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.590 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.591 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.591 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.591 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.725657+0000", 2024-04-20T01:48:22.591 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.591 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.591 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.591 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.591 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.725657+0000", 2024-04-20T01:48:22.591 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:48:22.591 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.591 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.591 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.591 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.725677+0000", 2024-04-20T01:48:22.591 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0139000000000001e-05 2024-04-20T01:48:22.592 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.592 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.592 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.592 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.725690+0000", 2024-04-20T01:48:22.592 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2459e-05 2024-04-20T01:48:22.592 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.592 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.592 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.592 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.725699+0000", 2024-04-20T01:48:22.592 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8529999999999998e-06 2024-04-20T01:48:22.592 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.592 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.592 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.592 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.725775+0000", 2024-04-20T01:48:22.592 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.6748000000000004e-05 2024-04-20T01:48:22.593 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.593 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.593 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.593 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.725786+0000", 2024-04-20T01:48:22.593 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0428000000000001e-05 2024-04-20T01:48:22.593 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.593 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.593 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.593 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.732794+0000", 2024-04-20T01:48:22.593 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0070079160000000003 2024-04-20T01:48:22.593 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.593 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.593 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.593 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.732821+0000", 2024-04-20T01:48:22.593 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7195e-05 2024-04-20T01:48:22.594 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.594 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.594 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.594 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.594 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.594 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 75/30 ECSubWrite(tid=254, reqid=osd.4.0:254, at_version=75'133, trim_to=0'0, roll_forward_to=73'131))", 2024-04-20T01:48:22.594 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.727466+0000", 2024-04-20T01:48:22.594 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.8468701039999997, 2024-04-20T01:48:22.594 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.012011501000000001, 2024-04-20T01:48:22.594 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.594 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.594 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.594 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.594 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.595 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.727466+0000", 2024-04-20T01:48:22.595 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.595 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.595 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.595 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.595 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.727466+0000", 2024-04-20T01:48:22.595 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.595 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.595 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.595 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.595 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.727465+0000", 2024-04-20T01:48:22.595 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:22.595 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.595 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.595 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.596 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.727485+0000", 2024-04-20T01:48:22.596 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0041999999999999e-05 2024-04-20T01:48:22.596 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.596 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.596 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.596 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.727495+0000", 2024-04-20T01:48:22.596 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0127e-05 2024-04-20T01:48:22.596 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.596 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.596 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.596 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.727502+0000", 2024-04-20T01:48:22.596 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.6019999999999996e-06 2024-04-20T01:48:22.596 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.596 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.596 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.597 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.727540+0000", 2024-04-20T01:48:22.597 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7867999999999997e-05 2024-04-20T01:48:22.597 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.597 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.597 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.597 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.727550+0000", 2024-04-20T01:48:22.597 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6129999999999993e-06 2024-04-20T01:48:22.597 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.597 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.597 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.597 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739449+0000", 2024-04-20T01:48:22.597 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011899442 2024-04-20T01:48:22.597 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.597 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.597 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.598 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739478+0000", 2024-04-20T01:48:22.598 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8643e-05 2024-04-20T01:48:22.598 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.598 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.598 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.598 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.598 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.598 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 75/30 ECSubWrite(tid=255, reqid=osd.4.0:255, at_version=75'135, trim_to=0'0, roll_forward_to=73'131))", 2024-04-20T01:48:22.598 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.729295+0000", 2024-04-20T01:48:22.598 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.8450415119999999, 2024-04-20T01:48:22.598 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.012091924, 2024-04-20T01:48:22.598 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.598 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.598 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.599 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.599 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.599 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.729295+0000", 2024-04-20T01:48:22.599 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.599 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.599 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.599 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.599 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.729295+0000", 2024-04-20T01:48:22.599 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.599 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.599 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.599 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.599 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.729294+0000", 2024-04-20T01:48:22.599 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:22.599 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.600 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.600 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.600 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.729312+0000", 2024-04-20T01:48:22.600 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8488e-05 2024-04-20T01:48:22.600 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.600 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.600 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.600 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.729325+0000", 2024-04-20T01:48:22.600 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2143999999999999e-05 2024-04-20T01:48:22.600 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.600 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.600 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.600 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.729334+0000", 2024-04-20T01:48:22.600 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3149999999999998e-06 2024-04-20T01:48:22.601 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.601 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.601 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.601 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730067+0000", 2024-04-20T01:48:22.601 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00073259599999999996 2024-04-20T01:48:22.601 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.601 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.601 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.601 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730078+0000", 2024-04-20T01:48:22.601 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1635e-05 2024-04-20T01:48:22.601 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.601 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.601 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.601 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.741357+0000", 2024-04-20T01:48:22.601 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011278923999999999 2024-04-20T01:48:22.602 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.602 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.602 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.602 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.741387+0000", 2024-04-20T01:48:22.602 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9530999999999999e-05 2024-04-20T01:48:22.602 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.602 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.602 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.602 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.602 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.602 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 75/30 ECSubWrite(tid=256, reqid=client.4643.0:783, at_version=75'137, trim_to=0'0, roll_forward_to=73'131))", 2024-04-20T01:48:22.602 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.730687+0000", 2024-04-20T01:48:22.602 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.84364887, 2024-04-20T01:48:22.602 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.019480198000000001, 2024-04-20T01:48:22.603 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.603 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.603 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.603 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.603 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.603 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730687+0000", 2024-04-20T01:48:22.603 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.603 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.603 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.603 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.604 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730687+0000", 2024-04-20T01:48:22.604 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.604 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.604 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.604 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.604 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730687+0000", 2024-04-20T01:48:22.604 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:22.604 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.604 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.604 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.604 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730847+0000", 2024-04-20T01:48:22.604 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00016085900000000001 2024-04-20T01:48:22.604 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.604 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.605 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.605 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730863+0000", 2024-04-20T01:48:22.605 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5324000000000001e-05 2024-04-20T01:48:22.605 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.605 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.605 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.605 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730873+0000", 2024-04-20T01:48:22.605 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0059e-05 2024-04-20T01:48:22.605 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.605 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.605 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.605 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.733352+0000", 2024-04-20T01:48:22.605 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0024793559999999998 2024-04-20T01:48:22.605 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.606 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.606 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.606 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.733363+0000", 2024-04-20T01:48:22.606 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0456e-05 2024-04-20T01:48:22.606 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.606 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.606 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.606 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.750144+0000", 2024-04-20T01:48:22.606 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.016780970999999999 2024-04-20T01:48:22.606 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.606 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.606 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.606 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.750168+0000", 2024-04-20T01:48:22.606 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3966999999999999e-05 2024-04-20T01:48:22.607 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.607 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.607 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.607 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.607 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.607 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.es1 75/30 ECSubWrite(tid=102, reqid=osd.11.0:101, at_version=0'0, trim_to=0'0, roll_forward_to=75'78))", 2024-04-20T01:48:22.607 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.738230+0000", 2024-04-20T01:48:22.607 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.8361064100000002, 2024-04-20T01:48:22.607 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0089712109999999998, 2024-04-20T01:48:22.607 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.607 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.607 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.607 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.607 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.608 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738230+0000", 2024-04-20T01:48:22.608 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.608 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.608 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.608 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.608 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738230+0000", 2024-04-20T01:48:22.608 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.608 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.608 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.608 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.608 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738229+0000", 2024-04-20T01:48:22.608 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:22.608 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.608 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.608 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.609 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738241+0000", 2024-04-20T01:48:22.609 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2243e-05 2024-04-20T01:48:22.609 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.609 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.609 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.609 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738255+0000", 2024-04-20T01:48:22.609 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4003e-05 2024-04-20T01:48:22.609 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.609 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.609 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.609 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738263+0000", 2024-04-20T01:48:22.609 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.2330000000000006e-06 2024-04-20T01:48:22.609 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.609 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.609 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.610 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738309+0000", 2024-04-20T01:48:22.610 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.6035999999999998e-05 2024-04-20T01:48:22.610 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.610 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.610 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.610 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738321+0000", 2024-04-20T01:48:22.610 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1874e-05 2024-04-20T01:48:22.610 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.610 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.610 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.610 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747172+0000", 2024-04-20T01:48:22.610 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0088509569999999996 2024-04-20T01:48:22.610 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.610 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.611 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.611 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747201+0000", 2024-04-20T01:48:22.611 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8683e-05 2024-04-20T01:48:22.611 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.611 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.611 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.611 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.611 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.611 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s2 75/30 ECSubWrite(tid=258, reqid=osd.4.0:258, at_version=75'21, trim_to=0'0, roll_forward_to=68'19))", 2024-04-20T01:48:22.611 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.738933+0000", 2024-04-20T01:48:22.611 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.8354030090000002, 2024-04-20T01:48:22.611 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0088958019999999995, 2024-04-20T01:48:22.611 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.611 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.611 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.612 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.612 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.612 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738933+0000", 2024-04-20T01:48:22.612 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.612 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.612 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.612 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.612 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738933+0000", 2024-04-20T01:48:22.612 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.612 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.612 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.612 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.612 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738932+0000", 2024-04-20T01:48:22.612 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:22.612 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.613 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.613 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.613 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738957+0000", 2024-04-20T01:48:22.613 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4559e-05 2024-04-20T01:48:22.613 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.613 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.613 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.613 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738970+0000", 2024-04-20T01:48:22.613 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.349e-05 2024-04-20T01:48:22.613 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.613 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.613 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.613 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738980+0000", 2024-04-20T01:48:22.613 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3149999999999998e-06 2024-04-20T01:48:22.613 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.613 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.614 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.614 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739052+0000", 2024-04-20T01:48:22.614 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.2657999999999995e-05 2024-04-20T01:48:22.614 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.614 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.614 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.614 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739062+0000", 2024-04-20T01:48:22.614 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7569999999999999e-06 2024-04-20T01:48:22.614 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.614 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.614 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.614 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747798+0000", 2024-04-20T01:48:22.614 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0087363609999999998 2024-04-20T01:48:22.614 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.615 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.615 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.615 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747829+0000", 2024-04-20T01:48:22.615 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0534000000000003e-05 2024-04-20T01:48:22.615 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.615 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.615 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.615 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.615 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.615 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s2 75/30 ECSubWrite(tid=259, reqid=osd.4.0:259, at_version=75'23, trim_to=0'0, roll_forward_to=68'19))", 2024-04-20T01:48:22.615 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.740125+0000", 2024-04-20T01:48:22.615 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.8342115349999997, 2024-04-20T01:48:22.615 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0094825380000000004, 2024-04-20T01:48:22.615 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.615 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.616 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.616 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.616 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.616 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740125+0000", 2024-04-20T01:48:22.616 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.616 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.616 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.616 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.616 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740125+0000", 2024-04-20T01:48:22.616 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.616 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.616 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.616 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.616 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740124+0000", 2024-04-20T01:48:22.616 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:22.617 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.617 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.617 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.617 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740152+0000", 2024-04-20T01:48:22.617 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7498e-05 2024-04-20T01:48:22.617 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.617 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.617 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.617 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740167+0000", 2024-04-20T01:48:22.617 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5787e-05 2024-04-20T01:48:22.617 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.617 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.617 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.617 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740177+0000", 2024-04-20T01:48:22.617 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4730000000000006e-06 2024-04-20T01:48:22.617 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.618 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.618 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.618 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740234+0000", 2024-04-20T01:48:22.618 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6827999999999999e-05 2024-04-20T01:48:22.618 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.618 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.618 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.618 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740241+0000", 2024-04-20T01:48:22.618 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.3810000000000001e-06 2024-04-20T01:48:22.618 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.618 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.618 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.618 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.749580+0000", 2024-04-20T01:48:22.618 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0093387439999999995 2024-04-20T01:48:22.619 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.619 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.619 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.619 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.749607+0000", 2024-04-20T01:48:22.619 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7492999999999999e-05 2024-04-20T01:48:22.619 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.619 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.619 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.619 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.619 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.619 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 75/30 ECSubWrite(tid=260, reqid=osd.4.0:260, at_version=75'139, trim_to=0'0, roll_forward_to=75'135))", 2024-04-20T01:48:22.619 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.742474+0000", 2024-04-20T01:48:22.619 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.8318620130000003, 2024-04-20T01:48:22.619 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0080654139999999999, 2024-04-20T01:48:22.620 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.620 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.620 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.620 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.620 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.620 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742474+0000", 2024-04-20T01:48:22.620 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.620 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.620 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.620 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.620 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742474+0000", 2024-04-20T01:48:22.620 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.620 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.620 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.620 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.621 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742473+0000", 2024-04-20T01:48:22.621 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:22.621 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.621 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.621 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.621 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742507+0000", 2024-04-20T01:48:22.621 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.3868000000000002e-05 2024-04-20T01:48:22.621 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.621 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.621 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.621 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742523+0000", 2024-04-20T01:48:22.621 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6126000000000001e-05 2024-04-20T01:48:22.621 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.621 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.621 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.621 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742534+0000", 2024-04-20T01:48:22.622 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.049e-05 2024-04-20T01:48:22.622 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.622 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.622 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.622 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742659+0000", 2024-04-20T01:48:22.622 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00012551000000000001 2024-04-20T01:48:22.622 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.622 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.622 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.622 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742671+0000", 2024-04-20T01:48:22.622 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1731e-05 2024-04-20T01:48:22.622 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.622 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.622 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.622 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.750515+0000", 2024-04-20T01:48:22.623 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0078435850000000001 2024-04-20T01:48:22.623 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.623 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.623 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.623 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.750540+0000", 2024-04-20T01:48:22.623 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4887000000000001e-05 2024-04-20T01:48:22.623 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.623 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.623 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.623 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.623 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.623 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 75/30 ECSubWrite(tid=261, reqid=osd.4.0:261, at_version=75'141, trim_to=0'0, roll_forward_to=75'135))", 2024-04-20T01:48:22.623 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.744213+0000", 2024-04-20T01:48:22.623 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.8301229599999997, 2024-04-20T01:48:22.623 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0088062550000000007, 2024-04-20T01:48:22.624 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.624 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.624 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.624 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.624 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.624 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744213+0000", 2024-04-20T01:48:22.624 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.624 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.624 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.624 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.624 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744213+0000", 2024-04-20T01:48:22.624 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.624 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.624 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.624 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.625 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744212+0000", 2024-04-20T01:48:22.625 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:22.625 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.625 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.625 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.625 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744236+0000", 2024-04-20T01:48:22.625 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3189000000000001e-05 2024-04-20T01:48:22.625 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.625 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.625 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.625 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744260+0000", 2024-04-20T01:48:22.625 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3845e-05 2024-04-20T01:48:22.625 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.625 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.625 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.625 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744269+0000", 2024-04-20T01:48:22.626 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2690000000000005e-06 2024-04-20T01:48:22.626 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.626 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.626 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.626 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747687+0000", 2024-04-20T01:48:22.626 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0034180909999999998 2024-04-20T01:48:22.626 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.626 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.626 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.626 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747694+0000", 2024-04-20T01:48:22.626 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.8580000000000002e-06 2024-04-20T01:48:22.626 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.626 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.626 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.626 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.752999+0000", 2024-04-20T01:48:22.627 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0053053809999999996 2024-04-20T01:48:22.627 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.627 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.627 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.627 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.753020+0000", 2024-04-20T01:48:22.627 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0383000000000001e-05 2024-04-20T01:48:22.627 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.627 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.627 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.627 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.627 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.627 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.1s2 75/30 ECSubWrite(tid=264, reqid=client.4643.0:793, at_version=75'40, trim_to=0'0, roll_forward_to=75'37))", 2024-04-20T01:48:22.627 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.756570+0000", 2024-04-20T01:48:22.627 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.8177658069999998, 2024-04-20T01:48:22.628 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0075168049999999997, 2024-04-20T01:48:22.628 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.628 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.628 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.628 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.628 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.628 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.756570+0000", 2024-04-20T01:48:22.628 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.628 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.628 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.628 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.628 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.756570+0000", 2024-04-20T01:48:22.628 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.628 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.628 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.629 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.629 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.756570+0000", 2024-04-20T01:48:22.629 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:22.629 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.629 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.629 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.629 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.756590+0000", 2024-04-20T01:48:22.629 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9802000000000001e-05 2024-04-20T01:48:22.629 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.629 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.629 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.629 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.756609+0000", 2024-04-20T01:48:22.629 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8992e-05 2024-04-20T01:48:22.629 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.629 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.630 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.630 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.756618+0000", 2024-04-20T01:48:22.630 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7769999999999997e-06 2024-04-20T01:48:22.630 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.630 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.630 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.630 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.756657+0000", 2024-04-20T01:48:22.630 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.8383999999999997e-05 2024-04-20T01:48:22.630 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.630 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.630 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.630 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.756663+0000", 2024-04-20T01:48:22.630 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.0800000000000002e-06 2024-04-20T01:48:22.630 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.631 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.631 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.631 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.764054+0000", 2024-04-20T01:48:22.631 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0073911519999999998 2024-04-20T01:48:22.631 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.631 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.631 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.631 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.764087+0000", 2024-04-20T01:48:22.631 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.3312000000000002e-05 2024-04-20T01:48:22.631 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.631 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.631 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.631 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.631 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.631 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 75/30 ECSubWrite(tid=268, reqid=osd.4.0:268, at_version=75'145, trim_to=0'0, roll_forward_to=75'141))", 2024-04-20T01:48:22.632 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.759435+0000", 2024-04-20T01:48:22.632 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.8149007179999996, 2024-04-20T01:48:22.632 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0072970989999999996, 2024-04-20T01:48:22.632 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.632 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.632 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.632 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.632 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.632 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.759435+0000", 2024-04-20T01:48:22.632 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.632 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.632 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.632 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.632 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.759435+0000", 2024-04-20T01:48:22.632 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.632 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.633 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.633 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.633 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.759435+0000", 2024-04-20T01:48:22.633 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:22.633 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.633 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.633 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.633 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.759456+0000", 2024-04-20T01:48:22.633 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0958000000000002e-05 2024-04-20T01:48:22.633 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.633 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.633 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.633 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.759470+0000", 2024-04-20T01:48:22.633 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4589e-05 2024-04-20T01:48:22.633 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.634 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.634 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.634 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.759479+0000", 2024-04-20T01:48:22.634 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.0999999999999993e-06 2024-04-20T01:48:22.634 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.634 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.634 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.634 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.759526+0000", 2024-04-20T01:48:22.634 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.7120999999999998e-05 2024-04-20T01:48:22.634 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.634 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.634 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.634 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.759536+0000", 2024-04-20T01:48:22.634 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6010000000000001e-06 2024-04-20T01:48:22.634 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.635 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.635 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.635 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.766700+0000", 2024-04-20T01:48:22.635 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0071636 2024-04-20T01:48:22.635 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.635 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.635 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.635 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.766733+0000", 2024-04-20T01:48:22.635 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.2928000000000001e-05 2024-04-20T01:48:22.635 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.635 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.635 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.635 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.635 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.635 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.1s2 75/30 ECSubWrite(tid=269, reqid=client.4643.0:794, at_version=0'0, trim_to=0'0, roll_forward_to=75'41))", 2024-04-20T01:48:22.636 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.764978+0000", 2024-04-20T01:48:22.636 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.8093584050000002, 2024-04-20T01:48:22.636 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.006913336, 2024-04-20T01:48:22.636 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.636 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.636 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.636 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.636 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.636 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.764978+0000", 2024-04-20T01:48:22.636 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.636 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.636 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.636 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.636 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.764978+0000", 2024-04-20T01:48:22.636 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.637 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.637 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.637 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.637 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.764977+0000", 2024-04-20T01:48:22.637 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:22.637 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.637 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.637 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.637 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.764990+0000", 2024-04-20T01:48:22.637 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.311e-05 2024-04-20T01:48:22.637 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.637 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.637 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.637 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.765005+0000", 2024-04-20T01:48:22.637 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5440000000000001e-05 2024-04-20T01:48:22.638 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.638 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.638 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.638 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.765015+0000", 2024-04-20T01:48:22.638 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.8160000000000005e-06 2024-04-20T01:48:22.638 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.638 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.638 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.638 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.765110+0000", 2024-04-20T01:48:22.638 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.5353000000000007e-05 2024-04-20T01:48:22.638 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.638 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.638 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.638 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.765122+0000", 2024-04-20T01:48:22.638 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1895e-05 2024-04-20T01:48:22.639 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.639 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.639 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.639 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.771857+0000", 2024-04-20T01:48:22.639 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0067349949999999997 2024-04-20T01:48:22.639 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.639 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.639 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.639 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.771891+0000", 2024-04-20T01:48:22.639 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.3791999999999998e-05 2024-04-20T01:48:22.639 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.639 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.639 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.639 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.639 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.640 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 80/30 ECSubWrite(tid=310, reqid=osd.4.0:303, at_version=0'0, trim_to=0'0, roll_forward_to=80'162))", 2024-04-20T01:48:22.640 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:21.744639+0000", 2024-04-20T01:48:22.640 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 0.82969681299999998, 2024-04-20T01:48:22.640 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0072014840000000002, 2024-04-20T01:48:22.640 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:22.640 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:22.640 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:22.640 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.640 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:22.640 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744639+0000", 2024-04-20T01:48:22.640 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.640 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.640 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.640 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:22.641 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744639+0000", 2024-04-20T01:48:22.641 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:22.641 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.641 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.641 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:22.641 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744639+0000", 2024-04-20T01:48:22.641 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:22.641 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.641 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.641 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:22.641 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744649+0000", 2024-04-20T01:48:22.641 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7189999999999996e-06 2024-04-20T01:48:22.641 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.641 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.642 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:22.642 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744661+0000", 2024-04-20T01:48:22.642 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2013e-05 2024-04-20T01:48:22.642 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.642 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.642 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:22.642 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744668+0000", 2024-04-20T01:48:22.642 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.9209999999999992e-06 2024-04-20T01:48:22.642 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.642 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.642 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:22.642 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744725+0000", 2024-04-20T01:48:22.642 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6833000000000003e-05 2024-04-20T01:48:22.642 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.643 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.643 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:22.643 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744738+0000", 2024-04-20T01:48:22.643 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2343e-05 2024-04-20T01:48:22.643 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.643 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.643 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:22.643 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.751817+0000", 2024-04-20T01:48:22.643 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0070796779999999998 2024-04-20T01:48:22.643 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:22.643 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:22.643 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:22.643 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.751841+0000", 2024-04-20T01:48:22.643 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.357e-05 2024-04-20T01:48:22.643 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.644 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.644 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.644 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:22.644 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:22.644 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:22.645 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-20T01:48:22.660 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:22.674+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:22.698 INFO:tasks.rados.rados.0.smithi165.stdout:343: done (4 left) 2024-04-20T01:48:22.698 INFO:tasks.rados.rados.0.smithi165.stdout:344: done (3 left) 2024-04-20T01:48:22.698 INFO:tasks.rados.rados.0.smithi165.stdout:346: done (2 left) 2024-04-20T01:48:22.698 INFO:tasks.rados.rados.0.smithi165.stdout:347: done (1 left) 2024-04-20T01:48:22.698 INFO:tasks.rados.rados.0.smithi165.stdout:348: done (0 left) 2024-04-20T01:48:22.698 INFO:tasks.rados.rados.0.smithi165.stdout:349: rollback oid 25 current snap is 26 2024-04-20T01:48:22.698 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 25 to 25 2024-04-20T01:48:22.698 INFO:tasks.rados.rados.0.smithi165.stdout:350: read oid 49 snap -1 2024-04-20T01:48:22.698 INFO:tasks.rados.rados.0.smithi165.stdout:350: expect (ObjNum 148 snap 25 seq_num 148) 2024-04-20T01:48:22.698 INFO:tasks.rados.rados.0.smithi165.stdout:351: snap_remove snap 23 2024-04-20T01:48:22.728 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-20T01:48:22.761 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:22.775+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:22.776 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:22.776 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:22.776 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 0 2024-04-20T01:48:22.776 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:22.786 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-20T01:48:22.861 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:22.875+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:22.915 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:22.915 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:22.915 INFO:teuthology.orchestra.run.smithi121.stdout: "complaint_time": 30, 2024-04-20T01:48:22.915 INFO:teuthology.orchestra.run.smithi121.stdout: "num_blocked_ops": 0 2024-04-20T01:48:22.915 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:22.925 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-20T01:48:22.969 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:22.984+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:23.055 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:23.055 INFO:teuthology.orchestra.run.smithi121.stdout: "size": 20, 2024-04-20T01:48:23.055 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 600, 2024-04-20T01:48:23.055 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-04-20T01:48:23.055 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.055 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-20T01:48:23.055 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:17.004323+0000", 2024-04-20T01:48:23.055 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 66.063692381999999, 2024-04-20T01:48:23.055 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.29931750600000001, 2024-04-20T01:48:23.055 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:23.055 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:23.055 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:23.056 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:48:23.056 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:48:23.056 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 1 2024-04-20T01:48:23.056 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.056 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:23.056 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.056 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:23.056 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004323+0000", 2024-04-20T01:48:23.056 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.056 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.056 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.056 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:23.056 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004323+0000", 2024-04-20T01:48:23.056 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.056 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.057 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.057 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:23.057 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004321+0000", 2024-04-20T01:48:23.057 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:23.057 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.057 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.057 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:23.057 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004337+0000", 2024-04-20T01:48:23.057 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5866999999999999e-05 2024-04-20T01:48:23.057 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.057 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.057 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:23.057 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004349+0000", 2024-04-20T01:48:23.057 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1802e-05 2024-04-20T01:48:23.058 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.058 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.058 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:23.058 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004360+0000", 2024-04-20T01:48:23.058 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1348999999999999e-05 2024-04-20T01:48:23.058 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.058 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.058 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.058 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004411+0000", 2024-04-20T01:48:23.058 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.0385e-05 2024-04-20T01:48:23.058 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.058 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.058 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for peered", 2024-04-20T01:48:23.058 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004419+0000", 2024-04-20T01:48:23.059 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4430000000000003e-06 2024-04-20T01:48:23.059 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.059 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.059 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.059 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.302330+0000", 2024-04-20T01:48:23.059 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.29791090199999998 2024-04-20T01:48:23.059 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.059 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.059 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:23.059 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.302488+0000", 2024-04-20T01:48:23.059 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00015830799999999999 2024-04-20T01:48:23.059 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.059 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.059 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:48:23.059 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.302650+0000", 2024-04-20T01:48:23.060 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00016196900000000001 2024-04-20T01:48:23.060 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.060 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.060 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:48:23.060 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.303204+0000", 2024-04-20T01:48:23.060 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00055385000000000005 2024-04-20T01:48:23.060 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.060 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.060 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:48:23.060 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.303574+0000", 2024-04-20T01:48:23.060 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00037030800000000002 2024-04-20T01:48:23.060 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.060 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.060 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:23.060 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.303614+0000", 2024-04-20T01:48:23.061 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.9161e-05 2024-04-20T01:48:23.061 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.061 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.061 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:23.061 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.303640+0000", 2024-04-20T01:48:23.061 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6752999999999999e-05 2024-04-20T01:48:23.061 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.061 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.061 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.061 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.061 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.061 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:18 3.3s0 3:cb777d78:::smithi16579500-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:23.061 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.756072+0000", 2024-04-20T01:48:23.061 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 48.311942975000001, 2024-04-20T01:48:23.061 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013705281, 2024-04-20T01:48:23.062 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:23.062 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:23.062 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:23.062 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:23.062 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:23.062 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 18 2024-04-20T01:48:23.062 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.062 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:23.062 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.062 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:23.062 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756072+0000", 2024-04-20T01:48:23.062 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.062 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.062 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.062 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:23.063 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756072+0000", 2024-04-20T01:48:23.063 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.063 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.063 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.063 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:23.063 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756072+0000", 2024-04-20T01:48:23.063 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:23.063 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.063 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:23.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756074+0000", 2024-04-20T01:48:23.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.148e-06 2024-04-20T01:48:23.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:23.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756079+0000", 2024-04-20T01:48:23.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.4790000000000004e-06 2024-04-20T01:48:23.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:23.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756083+0000", 2024-04-20T01:48:23.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.4910000000000001e-06 2024-04-20T01:48:23.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.758130+0000", 2024-04-20T01:48:23.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00204725 2024-04-20T01:48:23.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:23.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.758200+0000", 2024-04-20T01:48:23.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0246000000000005e-05 2024-04-20T01:48:23.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766738+0000", 2024-04-20T01:48:23.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0085377939999999996 2024-04-20T01:48:23.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.066 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.066 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:23.066 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766828+0000", 2024-04-20T01:48:23.066 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.9677000000000002e-05 2024-04-20T01:48:23.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.066 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.066 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:23.066 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.769777+0000", 2024-04-20T01:48:23.066 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0029495369999999999 2024-04-20T01:48:23.066 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.066 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.066 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.067 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.067 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:48 3.2s0 3:4efae9ea:::smithi16579500-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:23.067 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.768007+0000", 2024-04-20T01:48:23.067 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 48.300007831000002, 2024-04-20T01:48:23.067 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010953058, 2024-04-20T01:48:23.067 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:23.067 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:23.067 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:23.067 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:23.067 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:23.067 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 48 2024-04-20T01:48:23.067 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.067 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:23.067 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.067 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:23.068 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768007+0000", 2024-04-20T01:48:23.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.068 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.068 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:23.068 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768007+0000", 2024-04-20T01:48:23.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.068 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.068 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:23.068 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768007+0000", 2024-04-20T01:48:23.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:23.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.068 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:23.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768011+0000", 2024-04-20T01:48:23.069 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2899999999999996e-06 2024-04-20T01:48:23.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.069 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:23.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768018+0000", 2024-04-20T01:48:23.069 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0060000000000003e-06 2024-04-20T01:48:23.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.069 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:23.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768025+0000", 2024-04-20T01:48:23.069 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.8390000000000001e-06 2024-04-20T01:48:23.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.069 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.772765+0000", 2024-04-20T01:48:23.070 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0047401409999999998 2024-04-20T01:48:23.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:23.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.772861+0000", 2024-04-20T01:48:23.070 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6513000000000002e-05 2024-04-20T01:48:23.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.776375+0000", 2024-04-20T01:48:23.070 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0035133679999999998 2024-04-20T01:48:23.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.071 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.071 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:23.071 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.776459+0000", 2024-04-20T01:48:23.071 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4368000000000003e-05 2024-04-20T01:48:23.071 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.071 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.071 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:23.071 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.778960+0000", 2024-04-20T01:48:23.071 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0025012910000000001 2024-04-20T01:48:23.071 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.071 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.071 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.071 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.071 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.071 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:67 3.3s0 3:c86e2dd3:::smithi16579500-23:head [append 0~376832,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:23.072 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.783364+0000", 2024-04-20T01:48:23.072 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 48.284651040999996, 2024-04-20T01:48:23.072 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013527099000000001, 2024-04-20T01:48:23.072 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:23.072 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:23.072 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:23.072 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:23.072 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:23.072 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 67 2024-04-20T01:48:23.072 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.073 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:23.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.073 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:23.073 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783364+0000", 2024-04-20T01:48:23.073 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.073 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.073 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:23.073 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783364+0000", 2024-04-20T01:48:23.073 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.073 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.073 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:23.073 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783355+0000", 2024-04-20T01:48:23.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999909 2024-04-20T01:48:23.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:23.074 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783704+0000", 2024-04-20T01:48:23.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00034900399999999998 2024-04-20T01:48:23.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:23.074 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783720+0000", 2024-04-20T01:48:23.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5464e-05 2024-04-20T01:48:23.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:23.074 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783727+0000", 2024-04-20T01:48:23.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.6199999999999999e-06 2024-04-20T01:48:23.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.075 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783773+0000", 2024-04-20T01:48:23.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5237000000000001e-05 2024-04-20T01:48:23.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.075 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:23.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783881+0000", 2024-04-20T01:48:23.076 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00010799200000000001 2024-04-20T01:48:23.076 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.076 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.076 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:23.076 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.784468+0000", 2024-04-20T01:48:23.076 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00058749400000000004 2024-04-20T01:48:23.076 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.076 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.076 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:23.076 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.785849+0000", 2024-04-20T01:48:23.076 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0013810420000000001 2024-04-20T01:48:23.076 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.076 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.076 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:23.076 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.796778+0000", 2024-04-20T01:48:23.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010928955000000001 2024-04-20T01:48:23.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.077 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:23.077 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.796891+0000", 2024-04-20T01:48:23.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00011315399999999999 2024-04-20T01:48:23.077 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.077 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.077 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.077 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:68 3.3s0 3:c86e2dd3:::smithi16579500-23:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:23.077 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.783739+0000", 2024-04-20T01:48:23.077 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 48.284275766999997, 2024-04-20T01:48:23.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013359042, 2024-04-20T01:48:23.077 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:23.078 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:23.078 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:23.078 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:23.078 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:23.078 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 68 2024-04-20T01:48:23.078 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.078 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:23.078 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.078 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:23.078 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783739+0000", 2024-04-20T01:48:23.078 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.078 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.078 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.078 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:23.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783739+0000", 2024-04-20T01:48:23.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.079 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:23.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783739+0000", 2024-04-20T01:48:23.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:23.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.079 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:23.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783744+0000", 2024-04-20T01:48:23.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.7690000000000004e-06 2024-04-20T01:48:23.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.079 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:23.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783750+0000", 2024-04-20T01:48:23.080 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.0719999999999996e-06 2024-04-20T01:48:23.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.080 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:23.080 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783754+0000", 2024-04-20T01:48:23.080 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.6450000000000004e-06 2024-04-20T01:48:23.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.080 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.080 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.784916+0000", 2024-04-20T01:48:23.080 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001161477 2024-04-20T01:48:23.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.080 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:23.080 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.785022+0000", 2024-04-20T01:48:23.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000106153 2024-04-20T01:48:23.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.081 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:23.081 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.785477+0000", 2024-04-20T01:48:23.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00045462699999999998 2024-04-20T01:48:23.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.081 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:23.081 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.787709+0000", 2024-04-20T01:48:23.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0022322309999999999 2024-04-20T01:48:23.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.081 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:23.082 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797021+0000", 2024-04-20T01:48:23.082 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0093120730000000006 2024-04-20T01:48:23.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.082 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.082 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:23.082 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797098+0000", 2024-04-20T01:48:23.082 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.7526000000000003e-05 2024-04-20T01:48:23.082 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.082 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.082 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.083 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:69 3.3s0 3:c86e2dd3:::smithi16579500-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:23.083 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.783758+0000", 2024-04-20T01:48:23.083 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 48.284257220000001, 2024-04-20T01:48:23.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.017903170999999999, 2024-04-20T01:48:23.083 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:23.083 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:23.083 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:23.083 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:23.083 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:23.083 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 69 2024-04-20T01:48:23.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.083 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:23.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:23.084 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783758+0000", 2024-04-20T01:48:23.084 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:23.084 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783758+0000", 2024-04-20T01:48:23.084 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:23.084 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783758+0000", 2024-04-20T01:48:23.084 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:23.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.085 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.085 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:23.085 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783759+0000", 2024-04-20T01:48:23.085 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.734e-06 2024-04-20T01:48:23.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.085 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.085 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:23.085 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783763+0000", 2024-04-20T01:48:23.085 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.5820000000000002e-06 2024-04-20T01:48:23.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.085 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.085 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:23.085 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783766+0000", 2024-04-20T01:48:23.085 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0469999999999998e-06 2024-04-20T01:48:23.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.086 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.086 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.086 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.785726+0000", 2024-04-20T01:48:23.086 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0019601079999999999 2024-04-20T01:48:23.086 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.086 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.086 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:23.086 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.785810+0000", 2024-04-20T01:48:23.086 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.3522000000000004e-05 2024-04-20T01:48:23.086 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.086 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.086 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.086 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797485+0000", 2024-04-20T01:48:23.086 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011675207 2024-04-20T01:48:23.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.087 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.087 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:23.087 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797573+0000", 2024-04-20T01:48:23.087 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8146000000000006e-05 2024-04-20T01:48:23.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.087 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.087 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:23.087 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.801661+0000", 2024-04-20T01:48:23.087 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0040880969999999997 2024-04-20T01:48:23.087 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.087 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.087 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.087 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.088 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:75 3.2s0 3:48443997:::smithi16579500-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:23.088 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.789051+0000", 2024-04-20T01:48:23.088 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 48.278963685000001, 2024-04-20T01:48:23.088 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013741625, 2024-04-20T01:48:23.088 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:23.088 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:23.088 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:23.088 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:23.088 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:23.088 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 75 2024-04-20T01:48:23.088 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.088 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:23.088 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.088 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:23.089 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789051+0000", 2024-04-20T01:48:23.089 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.089 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.089 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:23.089 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789051+0000", 2024-04-20T01:48:23.089 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.089 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.089 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:23.089 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789051+0000", 2024-04-20T01:48:23.089 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:23.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.089 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.089 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:23.090 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789054+0000", 2024-04-20T01:48:23.090 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7499999999999999e-06 2024-04-20T01:48:23.090 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.090 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.090 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:23.090 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789060+0000", 2024-04-20T01:48:23.090 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.567e-06 2024-04-20T01:48:23.090 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.090 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.090 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:23.090 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789066+0000", 2024-04-20T01:48:23.090 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.1499999999999998e-06 2024-04-20T01:48:23.090 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.090 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.091 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.091 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.790905+0000", 2024-04-20T01:48:23.091 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0018397000000000001 2024-04-20T01:48:23.091 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.091 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.091 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:23.091 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.790994+0000", 2024-04-20T01:48:23.091 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8679999999999998e-05 2024-04-20T01:48:23.091 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.091 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.091 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.091 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797798+0000", 2024-04-20T01:48:23.091 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0068040899999999996 2024-04-20T01:48:23.091 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.092 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.092 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:23.092 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797879+0000", 2024-04-20T01:48:23.092 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.1318999999999998e-05 2024-04-20T01:48:23.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.092 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.092 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:23.092 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.802793+0000", 2024-04-20T01:48:23.092 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.004913789 2024-04-20T01:48:23.092 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.092 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.092 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.092 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.092 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:84 3.3s0 3:cb8bd5ef:::smithi16579500-28:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:23.093 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.797448+0000", 2024-04-20T01:48:23.093 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 48.270567223, 2024-04-20T01:48:23.093 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.01109836, 2024-04-20T01:48:23.093 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:23.093 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:23.093 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:23.093 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:23.093 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:23.093 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 84 2024-04-20T01:48:23.093 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.093 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:23.093 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.093 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:23.093 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797448+0000", 2024-04-20T01:48:23.094 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.094 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.094 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.094 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:23.094 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797448+0000", 2024-04-20T01:48:23.094 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.094 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.094 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.094 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:23.094 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797448+0000", 2024-04-20T01:48:23.094 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:23.094 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.094 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.094 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:23.095 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797450+0000", 2024-04-20T01:48:23.095 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7999999999999999e-06 2024-04-20T01:48:23.095 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.095 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.095 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:23.095 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797458+0000", 2024-04-20T01:48:23.095 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.2590000000000003e-06 2024-04-20T01:48:23.095 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.095 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.095 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:23.095 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797464+0000", 2024-04-20T01:48:23.095 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.2979999999999997e-06 2024-04-20T01:48:23.095 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.095 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.095 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.096 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.801705+0000", 2024-04-20T01:48:23.096 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0042409819999999999 2024-04-20T01:48:23.096 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.096 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.096 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:23.096 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.801761+0000", 2024-04-20T01:48:23.096 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.5696999999999999e-05 2024-04-20T01:48:23.096 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.096 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.096 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.096 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.808533+0000", 2024-04-20T01:48:23.096 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0067724639999999997 2024-04-20T01:48:23.096 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.096 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.097 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:23.097 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.808546+0000", 2024-04-20T01:48:23.097 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3276e-05 2024-04-20T01:48:23.097 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.097 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.097 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.097 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.097 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.097 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:232 3.3s0 3:cb777d78:::smithi16579500-6:head [copy-from ver 6] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:23.097 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.787663+0000", 2024-04-20T01:48:23.097 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 44.280352618999999, 2024-04-20T01:48:23.097 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.019650832, 2024-04-20T01:48:23.097 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:23.097 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:23.098 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:23.098 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:23.098 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:23.098 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 232 2024-04-20T01:48:23.098 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.098 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:23.098 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.098 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:23.098 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787663+0000", 2024-04-20T01:48:23.098 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.098 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.098 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.098 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:23.098 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787663+0000", 2024-04-20T01:48:23.098 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.099 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.099 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.099 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:23.099 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787661+0000", 2024-04-20T01:48:23.099 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:23.099 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.099 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.099 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:23.099 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787673+0000", 2024-04-20T01:48:23.099 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.181e-05 2024-04-20T01:48:23.099 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.099 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.099 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:23.099 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787689+0000", 2024-04-20T01:48:23.099 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6731999999999998e-05 2024-04-20T01:48:23.100 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.100 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.100 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:23.100 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787706+0000", 2024-04-20T01:48:23.100 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6739e-05 2024-04-20T01:48:23.100 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.100 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.100 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.100 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787795+0000", 2024-04-20T01:48:23.100 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.9226000000000003e-05 2024-04-20T01:48:23.100 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.100 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.100 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:23.100 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787904+0000", 2024-04-20T01:48:23.101 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000108452 2024-04-20T01:48:23.101 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.101 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.101 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:23.101 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.797712+0000", 2024-04-20T01:48:23.101 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0098079599999999992 2024-04-20T01:48:23.101 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.101 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.101 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:23.101 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.799543+0000", 2024-04-20T01:48:23.101 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001831596 2024-04-20T01:48:23.101 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.101 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.101 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:23.101 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807224+0000", 2024-04-20T01:48:23.102 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0076805249999999997 2024-04-20T01:48:23.102 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.102 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.102 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:23.102 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807313+0000", 2024-04-20T01:48:23.102 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.9431999999999993e-05 2024-04-20T01:48:23.102 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.102 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.102 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.102 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.102 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.102 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:233 3.3s0 3:cb777d78:::smithi16579500-6:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:23.102 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.787721+0000", 2024-04-20T01:48:23.102 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 44.280294007000002, 2024-04-20T01:48:23.103 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.01143623, 2024-04-20T01:48:23.103 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:23.103 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:23.103 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:23.103 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:23.103 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:23.103 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 233 2024-04-20T01:48:23.103 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.103 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:23.103 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.103 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:23.103 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787721+0000", 2024-04-20T01:48:23.103 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.103 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.103 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.104 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:23.104 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787721+0000", 2024-04-20T01:48:23.104 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.104 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.104 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.104 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:23.104 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787721+0000", 2024-04-20T01:48:23.104 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:23.104 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.104 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.104 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:23.104 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787743+0000", 2024-04-20T01:48:23.104 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2540000000000001e-05 2024-04-20T01:48:23.104 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.105 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.105 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:23.105 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787753+0000", 2024-04-20T01:48:23.105 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0101e-05 2024-04-20T01:48:23.105 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.105 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.105 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:23.105 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787760+0000", 2024-04-20T01:48:23.105 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.3729999999999998e-06 2024-04-20T01:48:23.105 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.105 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.105 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.105 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.788051+0000", 2024-04-20T01:48:23.105 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00029180999999999999 2024-04-20T01:48:23.105 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.106 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.106 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:23.106 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.788092+0000", 2024-04-20T01:48:23.106 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.0309000000000003e-05 2024-04-20T01:48:23.106 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.106 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.106 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.106 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.798980+0000", 2024-04-20T01:48:23.106 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010888070999999999 2024-04-20T01:48:23.106 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.106 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.106 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:23.106 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.799065+0000", 2024-04-20T01:48:23.106 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.5241999999999995e-05 2024-04-20T01:48:23.107 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.107 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.107 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:23.107 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.799157+0000", 2024-04-20T01:48:23.107 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2388999999999999e-05 2024-04-20T01:48:23.107 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.107 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.107 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.107 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.107 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.107 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:296 3.2s0 3:44d9544b:::smithi16579500-38:head [read 0~1474560,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:23.107 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.807470+0000", 2024-04-20T01:48:23.107 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 39.260545034000003, 2024-04-20T01:48:23.107 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011565101, 2024-04-20T01:48:23.107 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:23.108 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:23.108 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:23.108 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:23.108 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:23.108 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 296 2024-04-20T01:48:23.108 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.108 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:23.108 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.108 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:23.108 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807470+0000", 2024-04-20T01:48:23.108 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.108 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.108 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.108 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:23.108 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807470+0000", 2024-04-20T01:48:23.109 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.109 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.109 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.109 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:23.109 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807470+0000", 2024-04-20T01:48:23.109 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:23.109 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.109 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.109 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:23.109 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807473+0000", 2024-04-20T01:48:23.109 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9809999999999999e-06 2024-04-20T01:48:23.109 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.109 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.110 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:23.110 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807479+0000", 2024-04-20T01:48:23.110 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.1970000000000001e-06 2024-04-20T01:48:23.110 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.110 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.110 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:23.110 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807483+0000", 2024-04-20T01:48:23.110 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.053e-06 2024-04-20T01:48:23.110 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.110 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.110 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.110 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807633+0000", 2024-04-20T01:48:23.110 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00015004999999999999 2024-04-20T01:48:23.110 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.111 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.111 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:23.111 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807643+0000", 2024-04-20T01:48:23.111 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7049999999999994e-06 2024-04-20T01:48:23.111 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.111 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.111 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.111 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.811287+0000", 2024-04-20T01:48:23.111 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0036436699999999999 2024-04-20T01:48:23.111 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.111 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.111 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:23.111 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.811372+0000", 2024-04-20T01:48:23.111 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.5251000000000001e-05 2024-04-20T01:48:23.111 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.112 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.112 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:23.112 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819035+0000", 2024-04-20T01:48:23.112 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.007663464 2024-04-20T01:48:23.112 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.112 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.112 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.112 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.112 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.112 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:299 3.2s0 3:4302a124:::smithi16579500-33:head [rollback a] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:23.112 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.808485+0000", 2024-04-20T01:48:23.112 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 39.259530263000002, 2024-04-20T01:48:23.112 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011156045, 2024-04-20T01:48:23.112 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:23.113 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:23.113 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:23.113 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:23.113 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:23.113 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 299 2024-04-20T01:48:23.113 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.113 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:23.113 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.113 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:23.113 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808485+0000", 2024-04-20T01:48:23.113 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.113 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.113 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.113 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:23.113 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808485+0000", 2024-04-20T01:48:23.114 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.114 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.114 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.114 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:23.114 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808484+0000", 2024-04-20T01:48:23.114 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:23.114 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.114 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.114 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:23.114 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808488+0000", 2024-04-20T01:48:23.114 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.3529999999999999e-06 2024-04-20T01:48:23.114 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.114 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.114 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:23.115 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808495+0000", 2024-04-20T01:48:23.115 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.4390000000000003e-06 2024-04-20T01:48:23.115 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.115 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.115 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:23.115 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808501+0000", 2024-04-20T01:48:23.115 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.4480000000000002e-06 2024-04-20T01:48:23.115 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.115 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.115 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.115 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808634+0000", 2024-04-20T01:48:23.115 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000133461 2024-04-20T01:48:23.115 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.115 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.115 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:23.116 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808650+0000", 2024-04-20T01:48:23.116 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6271999999999999e-05 2024-04-20T01:48:23.116 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.116 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.116 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.116 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.815215+0000", 2024-04-20T01:48:23.116 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0065644509999999998 2024-04-20T01:48:23.116 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.116 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.116 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:23.116 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.815328+0000", 2024-04-20T01:48:23.116 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000112754 2024-04-20T01:48:23.116 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.116 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.116 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:23.117 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.815818+0000", 2024-04-20T01:48:23.117 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00049083000000000004 2024-04-20T01:48:23.117 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.117 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.117 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:23.117 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819209+0000", 2024-04-20T01:48:23.117 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0033910139999999999 2024-04-20T01:48:23.117 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.117 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.117 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:23.117 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819596+0000", 2024-04-20T01:48:23.117 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00038603299999999998 2024-04-20T01:48:23.117 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.117 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.117 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:23.118 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819641+0000", 2024-04-20T01:48:23.118 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5442999999999998e-05 2024-04-20T01:48:23.118 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.118 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.118 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.118 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.118 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.118 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:300 3.2s0 3:4302a124:::smithi16579500-33:head [append 0~0] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:23.118 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.808516+0000", 2024-04-20T01:48:23.118 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 39.259499126000001, 2024-04-20T01:48:23.118 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.016176293000000001, 2024-04-20T01:48:23.118 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:23.118 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:23.118 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:23.119 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:23.119 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:23.119 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 300 2024-04-20T01:48:23.119 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.119 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:23.119 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.119 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:23.119 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808516+0000", 2024-04-20T01:48:23.119 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.119 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.119 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.119 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:23.119 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808516+0000", 2024-04-20T01:48:23.119 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.119 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.120 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.120 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:23.120 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808515+0000", 2024-04-20T01:48:23.120 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:23.120 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.120 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.120 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:23.120 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808521+0000", 2024-04-20T01:48:23.120 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.8919999999999997e-06 2024-04-20T01:48:23.120 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.120 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.120 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:23.120 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808528+0000", 2024-04-20T01:48:23.120 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0190000000000001e-06 2024-04-20T01:48:23.120 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.121 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.121 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:23.121 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808534+0000", 2024-04-20T01:48:23.121 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.891e-06 2024-04-20T01:48:23.121 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.121 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.121 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.121 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.809751+0000", 2024-04-20T01:48:23.121 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0012163580000000001 2024-04-20T01:48:23.121 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.121 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.121 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:23.121 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.809758+0000", 2024-04-20T01:48:23.121 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.6580000000000002e-06 2024-04-20T01:48:23.122 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.122 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.122 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.122 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819973+0000", 2024-04-20T01:48:23.122 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010214983 2024-04-20T01:48:23.122 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.122 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.122 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:23.122 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.820076+0000", 2024-04-20T01:48:23.122 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00010318 2024-04-20T01:48:23.122 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.122 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.122 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:23.122 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.820519+0000", 2024-04-20T01:48:23.123 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00044209799999999998 2024-04-20T01:48:23.123 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.123 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.123 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:23.123 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.821591+0000", 2024-04-20T01:48:23.123 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001072677 2024-04-20T01:48:23.123 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.123 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.123 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:23.123 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.824619+0000", 2024-04-20T01:48:23.123 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0030279170000000002 2024-04-20T01:48:23.123 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.123 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.123 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:23.123 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.824692+0000", 2024-04-20T01:48:23.124 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.3220999999999996e-05 2024-04-20T01:48:23.124 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.124 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.124 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.124 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.124 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.124 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:302 3.2s0 3:4cb85a84:::smithi16579500-29:head [copy-from ver 10] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:23.124 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.823551+0000", 2024-04-20T01:48:23.124 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 39.244464069999999, 2024-04-20T01:48:23.124 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.014229169999999999, 2024-04-20T01:48:23.124 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:23.124 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:23.124 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:23.124 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:23.125 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:23.125 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 302 2024-04-20T01:48:23.125 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.125 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:23.125 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.125 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:23.125 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823551+0000", 2024-04-20T01:48:23.125 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.125 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.125 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.125 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:23.125 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823551+0000", 2024-04-20T01:48:23.125 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.125 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.125 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.126 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:23.126 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823550+0000", 2024-04-20T01:48:23.126 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:23.126 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.126 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.126 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:23.126 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823556+0000", 2024-04-20T01:48:23.126 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.455e-06 2024-04-20T01:48:23.126 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.126 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.126 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:23.126 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823568+0000", 2024-04-20T01:48:23.126 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1921e-05 2024-04-20T01:48:23.126 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.127 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.127 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:23.127 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823575+0000", 2024-04-20T01:48:23.127 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.1169999999999998e-06 2024-04-20T01:48:23.127 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.127 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.127 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.127 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823617+0000", 2024-04-20T01:48:23.127 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2012999999999999e-05 2024-04-20T01:48:23.127 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.127 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.127 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:23.127 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823741+0000", 2024-04-20T01:48:23.127 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000123454 2024-04-20T01:48:23.128 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.128 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.128 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:23.128 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.829454+0000", 2024-04-20T01:48:23.128 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0057129570000000003 2024-04-20T01:48:23.128 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.128 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.128 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:23.128 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.831037+0000", 2024-04-20T01:48:23.128 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0015832559999999999 2024-04-20T01:48:23.128 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.128 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.128 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:23.128 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.837719+0000", 2024-04-20T01:48:23.128 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.006681953 2024-04-20T01:48:23.129 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.129 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.129 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:23.129 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.837780+0000", 2024-04-20T01:48:23.129 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.1561000000000002e-05 2024-04-20T01:48:23.129 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.129 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.129 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.129 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.129 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.129 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:403 3.2s0 3:44d9544b:::smithi16579500-38:head [append 0~638976] snapc d=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-04-20T01:48:23.129 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.784120+0000", 2024-04-20T01:48:23.129 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 33.283895547999997, 2024-04-20T01:48:23.129 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.014614749999999999, 2024-04-20T01:48:23.129 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:23.130 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:23.130 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:23.130 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:23.130 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:23.130 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 403 2024-04-20T01:48:23.130 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.130 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:23.130 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.130 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:23.130 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.784120+0000", 2024-04-20T01:48:23.130 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.130 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.130 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.130 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:23.131 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.784120+0000", 2024-04-20T01:48:23.131 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.131 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.131 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.131 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:23.131 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.784118+0000", 2024-04-20T01:48:23.131 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:23.131 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.131 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.131 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:23.131 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788188+0000", 2024-04-20T01:48:23.131 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0040692860000000001 2024-04-20T01:48:23.131 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.131 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.131 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:23.132 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788201+0000", 2024-04-20T01:48:23.132 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3142999999999999e-05 2024-04-20T01:48:23.132 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.132 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.132 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:23.132 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788207+0000", 2024-04-20T01:48:23.132 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.4300000000000003e-06 2024-04-20T01:48:23.132 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.132 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.132 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.132 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788261+0000", 2024-04-20T01:48:23.132 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3257000000000002e-05 2024-04-20T01:48:23.132 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.132 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.133 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:23.133 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788365+0000", 2024-04-20T01:48:23.133 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000103928 2024-04-20T01:48:23.133 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.133 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.133 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:23.133 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.789323+0000", 2024-04-20T01:48:23.133 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00095858900000000003 2024-04-20T01:48:23.133 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.133 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.133 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:23.133 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.793128+0000", 2024-04-20T01:48:23.133 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.003805307 2024-04-20T01:48:23.133 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.133 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.133 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:23.134 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.798692+0000", 2024-04-20T01:48:23.134 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0055635160000000001 2024-04-20T01:48:23.134 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.134 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.134 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:23.134 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.798734+0000", 2024-04-20T01:48:23.134 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2453999999999997e-05 2024-04-20T01:48:23.134 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.134 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.134 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.134 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.134 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.134 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:404 3.2s0 3:44d9544b:::smithi16579500-38:head [append 0~622592] snapc d=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-04-20T01:48:23.134 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.788218+0000", 2024-04-20T01:48:23.135 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 33.279797248000001, 2024-04-20T01:48:23.135 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.012422041, 2024-04-20T01:48:23.135 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:23.135 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:23.135 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:23.135 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:23.135 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:23.135 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 404 2024-04-20T01:48:23.135 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.135 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:23.135 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.135 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:23.135 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788218+0000", 2024-04-20T01:48:23.135 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.135 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.136 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.136 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:23.136 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788218+0000", 2024-04-20T01:48:23.136 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.136 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.136 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.136 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:23.136 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788217+0000", 2024-04-20T01:48:23.136 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:23.136 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.136 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.136 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:23.136 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.790736+0000", 2024-04-20T01:48:23.136 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0025187270000000001 2024-04-20T01:48:23.137 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.137 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.137 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:23.137 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.790752+0000", 2024-04-20T01:48:23.137 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5684000000000001e-05 2024-04-20T01:48:23.137 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.137 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.137 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:23.137 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.790762+0000", 2024-04-20T01:48:23.137 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0693000000000001e-05 2024-04-20T01:48:23.137 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.137 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.137 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.137 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.790813+0000", 2024-04-20T01:48:23.138 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.0235000000000003e-05 2024-04-20T01:48:23.138 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.138 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.138 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:23.138 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.790933+0000", 2024-04-20T01:48:23.138 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000120417 2024-04-20T01:48:23.138 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.138 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.138 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:23.138 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.791589+0000", 2024-04-20T01:48:23.138 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000655972 2024-04-20T01:48:23.138 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.138 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.138 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:23.139 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.793913+0000", 2024-04-20T01:48:23.139 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.002323716 2024-04-20T01:48:23.139 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.139 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.139 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:23.139 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.800601+0000", 2024-04-20T01:48:23.139 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0066881880000000003 2024-04-20T01:48:23.139 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.139 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.139 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:23.139 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.800640+0000", 2024-04-20T01:48:23.139 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.9150999999999998e-05 2024-04-20T01:48:23.139 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.139 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.140 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.140 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.140 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.140 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:523 3.2s0 3:44d9544b:::smithi16579500-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-04-20T01:48:23.140 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:01.840230+0000", 2024-04-20T01:48:23.140 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.227785042000001, 2024-04-20T01:48:23.140 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010941562, 2024-04-20T01:48:23.140 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:23.140 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:23.140 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:23.140 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:23.140 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:23.140 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 523 2024-04-20T01:48:23.140 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.140 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:23.141 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.141 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:23.141 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.840230+0000", 2024-04-20T01:48:23.141 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.141 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.141 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.141 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:23.141 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.840230+0000", 2024-04-20T01:48:23.141 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.141 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.141 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.141 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:23.141 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.840230+0000", 2024-04-20T01:48:23.141 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:23.142 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.142 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.142 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:23.142 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.840237+0000", 2024-04-20T01:48:23.142 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.2309999999999999e-06 2024-04-20T01:48:23.142 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.142 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.142 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:23.142 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.840243+0000", 2024-04-20T01:48:23.142 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.0079999999999999e-06 2024-04-20T01:48:23.142 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.142 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.142 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:23.142 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.840251+0000", 2024-04-20T01:48:23.143 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.3019999999999995e-06 2024-04-20T01:48:23.143 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.143 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.143 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.143 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.843356+0000", 2024-04-20T01:48:23.143 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0031046419999999999 2024-04-20T01:48:23.143 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.143 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.143 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:23.143 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.843447+0000", 2024-04-20T01:48:23.143 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.1217000000000004e-05 2024-04-20T01:48:23.143 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.143 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.143 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.143 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.850113+0000", 2024-04-20T01:48:23.144 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0066663060000000003 2024-04-20T01:48:23.144 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.144 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.144 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:23.144 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.850210+0000", 2024-04-20T01:48:23.144 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6875999999999999e-05 2024-04-20T01:48:23.144 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.144 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.144 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:23.144 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.851172+0000", 2024-04-20T01:48:23.144 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00096138500000000004 2024-04-20T01:48:23.144 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.144 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.144 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.144 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.145 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.145 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:574 3.2s0 3:44d9544b:::smithi16579500-38:head [append 0~655360] snapc 1f=[1f,1b,1a,b] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-20T01:48:23.145 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:04.793176+0000", 2024-04-20T01:48:23.145 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 18.27483896, 2024-04-20T01:48:23.145 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.012353861000000001, 2024-04-20T01:48:23.145 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:23.145 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:23.145 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:23.145 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:23.145 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:23.145 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 574 2024-04-20T01:48:23.145 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.145 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:23.145 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.146 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:23.146 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793176+0000", 2024-04-20T01:48:23.146 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.146 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.146 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.146 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:23.146 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793176+0000", 2024-04-20T01:48:23.146 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.146 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.146 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.146 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:23.146 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793174+0000", 2024-04-20T01:48:23.146 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:23.146 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.146 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.147 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:23.147 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793851+0000", 2024-04-20T01:48:23.147 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00067681499999999999 2024-04-20T01:48:23.147 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.147 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.147 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:23.147 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793869+0000", 2024-04-20T01:48:23.147 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7569000000000001e-05 2024-04-20T01:48:23.147 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.147 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.147 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:23.147 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793878+0000", 2024-04-20T01:48:23.147 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8659999999999995e-06 2024-04-20T01:48:23.147 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.147 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.148 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.148 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793935+0000", 2024-04-20T01:48:23.148 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6984999999999997e-05 2024-04-20T01:48:23.148 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.148 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.148 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:23.148 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.794017+0000", 2024-04-20T01:48:23.148 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.2719000000000005e-05 2024-04-20T01:48:23.148 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.148 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.148 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:23.148 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.794790+0000", 2024-04-20T01:48:23.148 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000772495 2024-04-20T01:48:23.148 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.149 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.149 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:23.149 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.796798+0000", 2024-04-20T01:48:23.149 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0020083980000000002 2024-04-20T01:48:23.149 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.149 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.149 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:23.149 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.805483+0000", 2024-04-20T01:48:23.149 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0086847739999999993 2024-04-20T01:48:23.149 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.149 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.149 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:23.149 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.805530+0000", 2024-04-20T01:48:23.149 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.7042999999999997e-05 2024-04-20T01:48:23.150 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.150 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.150 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.150 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.150 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.150 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:790 3.3s0 3:cb777d78:::smithi16579500-6:head [copy-from ver 101] snapc 2a=[2a,28] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-04-20T01:48:23.150 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.754166+0000", 2024-04-20T01:48:23.150 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 6.313849373, 2024-04-20T01:48:23.150 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.020397073000000002, 2024-04-20T01:48:23.150 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:23.150 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:23.150 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:23.150 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:23.150 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:23.151 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 790 2024-04-20T01:48:23.151 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.151 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:23.151 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.151 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:23.151 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754166+0000", 2024-04-20T01:48:23.151 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.151 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.151 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.151 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:23.151 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754166+0000", 2024-04-20T01:48:23.151 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.151 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.151 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.151 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:23.152 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754165+0000", 2024-04-20T01:48:23.152 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:23.152 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.152 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.152 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:23.152 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754171+0000", 2024-04-20T01:48:23.152 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.3489999999999997e-06 2024-04-20T01:48:23.152 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.152 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.152 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:23.152 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754183+0000", 2024-04-20T01:48:23.152 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2428e-05 2024-04-20T01:48:23.153 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.153 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.153 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:23.153 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754190+0000", 2024-04-20T01:48:23.153 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.3350000000000003e-06 2024-04-20T01:48:23.153 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.153 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.153 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.153 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754243+0000", 2024-04-20T01:48:23.153 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3245000000000003e-05 2024-04-20T01:48:23.153 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.153 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.153 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:23.153 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754343+0000", 2024-04-20T01:48:23.154 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.9770000000000002e-05 2024-04-20T01:48:23.154 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.154 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.154 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:23.154 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.766334+0000", 2024-04-20T01:48:23.154 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011991017 2024-04-20T01:48:23.154 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.154 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.154 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:23.154 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.772169+0000", 2024-04-20T01:48:23.154 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0058356240000000002 2024-04-20T01:48:23.154 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.154 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.154 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:23.154 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.774513+0000", 2024-04-20T01:48:23.155 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0023441130000000001 2024-04-20T01:48:23.155 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.155 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.155 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:23.155 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.774563+0000", 2024-04-20T01:48:23.155 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.9421e-05 2024-04-20T01:48:23.155 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.155 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.155 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.155 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.155 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.155 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:791 3.3s0 3:cb777d78:::smithi16579500-6:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e75)", 2024-04-20T01:48:23.155 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.754206+0000", 2024-04-20T01:48:23.155 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 6.3138089219999998, 2024-04-20T01:48:23.155 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.014483546999999999, 2024-04-20T01:48:23.156 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:23.156 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:23.156 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:23.156 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:23.156 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:23.156 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 791 2024-04-20T01:48:23.156 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.156 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:23.156 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.156 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:23.156 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754206+0000", 2024-04-20T01:48:23.156 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.156 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.156 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.157 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:23.157 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754206+0000", 2024-04-20T01:48:23.157 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:23.157 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.157 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.157 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:23.157 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754206+0000", 2024-04-20T01:48:23.157 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:23.157 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.157 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.157 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:23.157 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754209+0000", 2024-04-20T01:48:23.157 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.5190000000000001e-06 2024-04-20T01:48:23.157 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.157 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.158 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:23.158 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754216+0000", 2024-04-20T01:48:23.158 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.1060000000000002e-06 2024-04-20T01:48:23.158 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.158 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.158 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:23.158 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754222+0000", 2024-04-20T01:48:23.158 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.9479999999999996e-06 2024-04-20T01:48:23.158 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.158 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.158 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.158 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754512+0000", 2024-04-20T01:48:23.158 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00028918800000000001 2024-04-20T01:48:23.158 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.158 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.159 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:23.159 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754551+0000", 2024-04-20T01:48:23.159 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.9524999999999999e-05 2024-04-20T01:48:23.159 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.159 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.159 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:23.159 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.768532+0000", 2024-04-20T01:48:23.159 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013981247 2024-04-20T01:48:23.159 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.159 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.159 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:23.159 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.768614+0000", 2024-04-20T01:48:23.159 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.2078999999999998e-05 2024-04-20T01:48:23.159 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:23.159 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:23.160 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:23.160 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.768690+0000", 2024-04-20T01:48:23.160 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.5413000000000005e-05 2024-04-20T01:48:23.160 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.160 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.160 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.160 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:23.160 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:23.160 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:23.161 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-20T01:48:23.162 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:23.175+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:23.163 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-20T01:48:23.263 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:23.276+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:23.309 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:23.309 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:23.309 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:48:23.309 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:23.320 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-20T01:48:23.325 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:48:23.346 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:48:23.363 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:23.378+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:23.367 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:48:23.389 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:48:23.410 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:48:23.430 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:48:23.448 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:23.449 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:23.451 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:48:23.459 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-20T01:48:23.464 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:23.478+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:23.473 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:48:23.493 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:48:23.514 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:48:23.535 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:48:23.557 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:48:23.565 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:23.579+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:23.577 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:48:23.586 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:23.586 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:23.596 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-20T01:48:23.598 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:48:23.619 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:48:23.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:48:23.634+0000 7fdd5bfff700 0 -- 172.21.15.44:0/2080578126 >> [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] conn(0x7fdd5c21f310 msgr2=0x7fdd5c221700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-20T01:48:23.665 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:23.680+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:23.699 INFO:tasks.rados.rados.0.smithi165.stdout:351: done (2 left) 2024-04-20T01:48:23.699 INFO:tasks.rados.rados.0.smithi165.stdout:352: rollback oid 50 current snap is 26 2024-04-20T01:48:23.699 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 50 to 20 2024-04-20T01:48:23.701 INFO:tasks.rados.rados.0.smithi165.stdout:349: finishing rollback tid 1 to smithi16579500-25 2024-04-20T01:48:23.701 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 25 v 0 (ObjNum 161 snap 0 seq_num 1) dirty dne 2024-04-20T01:48:23.701 INFO:tasks.rados.rados.0.smithi165.stdout:350: expect (ObjNum 148 snap 25 seq_num 148) 2024-04-20T01:48:23.707 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is unset 2024-04-20T01:48:23.723 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-20T01:48:23.723 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-20T01:48:23.723 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-04-20T01:48:23.723 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-20T01:48:23.730 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:23.730 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:23.741 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-20T01:48:23.767 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:23.781+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:23.783 INFO:tasks.rados.rados.0.smithi165.stdout:352: finishing rollback tid 0 to smithi16579500-50 2024-04-20T01:48:23.783 INFO:tasks.rados.rados.0.smithi165.stdout:352: finishing rollback tid 1 to smithi16579500-50 2024-04-20T01:48:23.783 INFO:tasks.rados.rados.0.smithi165.stdout:352: finishing rollback tid 2 to smithi16579500-50 2024-04-20T01:48:23.783 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 50 v 115 (ObjNum 130 snap 20 seq_num 130) dirty exists 2024-04-20T01:48:23.783 INFO:tasks.rados.rados.0.smithi165.stdout:349: done (2 left) 2024-04-20T01:48:23.784 INFO:tasks.rados.rados.0.smithi165.stdout:350: done (1 left) 2024-04-20T01:48:23.784 INFO:tasks.rados.rados.0.smithi165.stdout:352: done (0 left) 2024-04-20T01:48:23.784 INFO:tasks.rados.rados.0.smithi165.stdout:353: snap_create 2024-04-20T01:48:23.842 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:48:23.844 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:48:23.844 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:48:23.844 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:48:23.844 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:48:23.844 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:48:23.844 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:48:23.844 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:48:23.844 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:48:23.844 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:48:23.844 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:48:23.845 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:48:23.845 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:48:23.845 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:48:23.845 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:48:23.845 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:48:23.845 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:48:23.868 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:23.882+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:23.868 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:23.868 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:23.878 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-20T01:48:23.969 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:23.982+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:24.045 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:24.045 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:24.055 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-20T01:48:24.099 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:24.113+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:24.135 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:48:24.136 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:24.136 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 82, 2024-04-20T01:48:24.136 INFO:teuthology.orchestra.run.smithi044.stdout: "fsid": "9c2f64c1-aa35-4337-9f5c-081b9e3a2938", 2024-04-20T01:48:24.136 INFO:teuthology.orchestra.run.smithi044.stdout: "created": "2024-04-20T01:47:02.934241+0000", 2024-04-20T01:48:24.136 INFO:teuthology.orchestra.run.smithi044.stdout: "modified": "2024-04-20T01:48:23.709271+0000", 2024-04-20T01:48:24.136 INFO:teuthology.orchestra.run.smithi044.stdout: "last_up_change": "2024-04-20T01:47:15.986754+0000", 2024-04-20T01:48:24.136 INFO:teuthology.orchestra.run.smithi044.stdout: "last_in_change": "2024-04-20T01:47:07.894719+0000", 2024-04-20T01:48:24.136 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-20T01:48:24.136 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_num": 5804032, 2024-04-20T01:48:24.136 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_set": [ 2024-04-20T01:48:24.136 INFO:teuthology.orchestra.run.smithi044.stdout: "nodeep-scrub", 2024-04-20T01:48:24.136 INFO:teuthology.orchestra.run.smithi044.stdout: "pglog_hardlimit", 2024-04-20T01:48:24.136 INFO:teuthology.orchestra.run.smithi044.stdout: "purged_snapdirs", 2024-04-20T01:48:24.137 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_deletes", 2024-04-20T01:48:24.137 INFO:teuthology.orchestra.run.smithi044.stdout: "sortbitwise" 2024-04-20T01:48:24.137 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:48:24.137 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_version": 5, 2024-04-20T01:48:24.137 INFO:teuthology.orchestra.run.smithi044.stdout: "full_ratio": 0.94999998807907104, 2024-04-20T01:48:24.137 INFO:teuthology.orchestra.run.smithi044.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-20T01:48:24.137 INFO:teuthology.orchestra.run.smithi044.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-20T01:48:24.137 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_snapshot": "", 2024-04-20T01:48:24.137 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_max": 3, 2024-04-20T01:48:24.137 INFO:teuthology.orchestra.run.smithi044.stdout: "max_osd": 16, 2024-04-20T01:48:24.137 INFO:teuthology.orchestra.run.smithi044.stdout: "require_min_compat_client": "luminous", 2024-04-20T01:48:24.137 INFO:teuthology.orchestra.run.smithi044.stdout: "min_compat_client": "jewel", 2024-04-20T01:48:24.137 INFO:teuthology.orchestra.run.smithi044.stdout: "require_osd_release": "reef", 2024-04-20T01:48:24.137 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_crimson": false, 2024-04-20T01:48:24.138 INFO:teuthology.orchestra.run.smithi044.stdout: "pools": [ 2024-04-20T01:48:24.138 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.138 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 1, 2024-04-20T01:48:24.138 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": ".mgr", 2024-04-20T01:48:24.138 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-04-20T01:47:15.195181+0000", 2024-04-20T01:48:24.138 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T01:48:24.138 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool", 2024-04-20T01:48:24.138 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 1, 2024-04-20T01:48:24.138 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 2, 2024-04-20T01:48:24.138 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 1, 2024-04-20T01:48:24.138 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 0, 2024-04-20T01:48:24.138 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-04-20T01:48:24.138 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-04-20T01:48:24.138 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-04-20T01:48:24.139 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-20T01:48:24.139 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-04-20T01:48:24.139 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-04-20T01:48:24.139 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 1, 2024-04-20T01:48:24.139 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 1, 2024-04-20T01:48:24.139 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 1, 2024-04-20T01:48:24.139 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 1, 2024-04-20T01:48:24.139 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 1, 2024-04-20T01:48:24.139 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-04-20T01:48:24.139 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-04-20T01:48:24.139 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-04-20T01:48:24.139 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-04-20T01:48:24.139 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-04-20T01:48:24.139 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-04-20T01:48:24.140 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-04-20T01:48:24.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.140 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "24", 2024-04-20T01:48:24.140 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-04-20T01:48:24.140 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-20T01:48:24.140 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-04-20T01:48:24.140 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-04-20T01:48:24.140 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-04-20T01:48:24.140 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 0, 2024-04-20T01:48:24.140 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 0, 2024-04-20T01:48:24.140 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-04-20T01:48:24.140 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-04-20T01:48:24.140 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-04-20T01:48:24.140 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-04-20T01:48:24.141 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-04-20T01:48:24.141 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-04-20T01:48:24.141 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-04-20T01:48:24.141 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-04-20T01:48:24.141 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-04-20T01:48:24.141 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-04-20T01:48:24.141 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-04-20T01:48:24.141 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-20T01:48:24.141 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-20T01:48:24.141 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-20T01:48:24.141 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-04-20T01:48:24.141 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-04-20T01:48:24.141 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "", 2024-04-20T01:48:24.141 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-04-20T01:48:24.142 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-04-20T01:48:24.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.142 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-04-20T01:48:24.142 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-04-20T01:48:24.142 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-04-20T01:48:24.142 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-04-20T01:48:24.142 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-04-20T01:48:24.142 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-04-20T01:48:24.142 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-04-20T01:48:24.142 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-04-20T01:48:24.142 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 0, 2024-04-20T01:48:24.142 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-04-20T01:48:24.142 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-04-20T01:48:24.142 INFO:teuthology.orchestra.run.smithi044.stdout: "options": { 2024-04-20T01:48:24.143 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_max": 32, 2024-04-20T01:48:24.143 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_min": 1 2024-04-20T01:48:24.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.143 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-04-20T01:48:24.143 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr": {} 2024-04-20T01:48:24.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.143 INFO:teuthology.orchestra.run.smithi044.stdout: "read_balance": { 2024-04-20T01:48:24.143 INFO:teuthology.orchestra.run.smithi044.stdout: "score_acting": 15.380000114440918, 2024-04-20T01:48:24.143 INFO:teuthology.orchestra.run.smithi044.stdout: "score_stable": 15.380000114440918, 2024-04-20T01:48:24.143 INFO:teuthology.orchestra.run.smithi044.stdout: "optimal_score": 0.12999999523162842, 2024-04-20T01:48:24.143 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_acting": 2, 2024-04-20T01:48:24.143 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_stable": 2, 2024-04-20T01:48:24.143 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity_weighted": 0.95162487030029297, 2024-04-20T01:48:24.143 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity": 0.94999998807907104, 2024-04-20T01:48:24.144 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity_weighted": 0.94999998807907104 2024-04-20T01:48:24.144 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.144 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.144 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 2, 2024-04-20T01:48:24.144 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": "rbd", 2024-04-20T01:48:24.144 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-04-20T01:47:18.448004+0000", 2024-04-20T01:48:24.144 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-04-20T01:48:24.144 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool", 2024-04-20T01:48:24.144 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 1, 2024-04-20T01:48:24.144 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 2, 2024-04-20T01:48:24.144 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 1, 2024-04-20T01:48:24.144 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 0, 2024-04-20T01:48:24.144 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-04-20T01:48:24.145 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-04-20T01:48:24.145 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-04-20T01:48:24.145 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-20T01:48:24.145 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-04-20T01:48:24.145 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-04-20T01:48:24.145 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 8, 2024-04-20T01:48:24.145 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 8, 2024-04-20T01:48:24.145 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 8, 2024-04-20T01:48:24.145 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 8, 2024-04-20T01:48:24.145 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 8, 2024-04-20T01:48:24.145 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-04-20T01:48:24.145 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-04-20T01:48:24.145 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-04-20T01:48:24.145 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-04-20T01:48:24.146 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-04-20T01:48:24.146 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-04-20T01:48:24.146 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-04-20T01:48:24.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.146 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "27", 2024-04-20T01:48:24.146 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-04-20T01:48:24.146 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-20T01:48:24.146 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-04-20T01:48:24.146 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-04-20T01:48:24.146 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-04-20T01:48:24.146 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 0, 2024-04-20T01:48:24.146 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 0, 2024-04-20T01:48:24.146 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-04-20T01:48:24.146 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-04-20T01:48:24.146 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-04-20T01:48:24.147 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-04-20T01:48:24.147 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-04-20T01:48:24.147 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-04-20T01:48:24.147 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-04-20T01:48:24.147 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-04-20T01:48:24.147 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-04-20T01:48:24.147 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-04-20T01:48:24.147 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-04-20T01:48:24.147 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-20T01:48:24.147 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-20T01:48:24.147 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-20T01:48:24.147 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-04-20T01:48:24.147 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-04-20T01:48:24.147 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "", 2024-04-20T01:48:24.147 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-04-20T01:48:24.148 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-04-20T01:48:24.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.148 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-04-20T01:48:24.148 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-04-20T01:48:24.148 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-04-20T01:48:24.148 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-04-20T01:48:24.148 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-04-20T01:48:24.148 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-04-20T01:48:24.148 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-04-20T01:48:24.148 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-04-20T01:48:24.148 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 0, 2024-04-20T01:48:24.148 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-04-20T01:48:24.148 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-04-20T01:48:24.148 INFO:teuthology.orchestra.run.smithi044.stdout: "options": {}, 2024-04-20T01:48:24.149 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-04-20T01:48:24.149 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd": {} 2024-04-20T01:48:24.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.149 INFO:teuthology.orchestra.run.smithi044.stdout: "read_balance": { 2024-04-20T01:48:24.149 INFO:teuthology.orchestra.run.smithi044.stdout: "score_acting": 2, 2024-04-20T01:48:24.149 INFO:teuthology.orchestra.run.smithi044.stdout: "score_stable": 2, 2024-04-20T01:48:24.149 INFO:teuthology.orchestra.run.smithi044.stdout: "optimal_score": 0.68999999761581421, 2024-04-20T01:48:24.149 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-20T01:48:24.149 INFO:teuthology.orchestra.run.smithi044.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-20T01:48:24.149 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity_weighted": 0.95162487030029297, 2024-04-20T01:48:24.149 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity": 0.94999998807907104, 2024-04-20T01:48:24.149 INFO:teuthology.orchestra.run.smithi044.stdout: "average_primary_affinity_weighted": 0.94999998807907104 2024-04-20T01:48:24.149 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.149 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.150 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 3, 2024-04-20T01:48:24.150 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": "unique_pool_0", 2024-04-20T01:48:24.150 INFO:teuthology.orchestra.run.smithi044.stdout: "create_time": "2024-04-20T01:47:30.734962+0000", 2024-04-20T01:48:24.150 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 8193, 2024-04-20T01:48:24.150 INFO:teuthology.orchestra.run.smithi044.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-04-20T01:48:24.150 INFO:teuthology.orchestra.run.smithi044.stdout: "type": 3, 2024-04-20T01:48:24.150 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 7, 2024-04-20T01:48:24.150 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": 5, 2024-04-20T01:48:24.150 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_rule": 1, 2024-04-20T01:48:24.150 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_count": 0, 2024-04-20T01:48:24.150 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_target": 0, 2024-04-20T01:48:24.150 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_barrier": 0, 2024-04-20T01:48:24.150 INFO:teuthology.orchestra.run.smithi044.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-20T01:48:24.150 INFO:teuthology.orchestra.run.smithi044.stdout: "object_hash": 2, 2024-04-20T01:48:24.150 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscale_mode": "off", 2024-04-20T01:48:24.151 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": 16, 2024-04-20T01:48:24.151 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num": 16, 2024-04-20T01:48:24.151 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_placement_num_target": 16, 2024-04-20T01:48:24.151 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_target": 16, 2024-04-20T01:48:24.151 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num_pending": 16, 2024-04-20T01:48:24.151 INFO:teuthology.orchestra.run.smithi044.stdout: "last_pg_merge_meta": { 2024-04-20T01:48:24.151 INFO:teuthology.orchestra.run.smithi044.stdout: "source_pgid": "0.0", 2024-04-20T01:48:24.151 INFO:teuthology.orchestra.run.smithi044.stdout: "ready_epoch": 0, 2024-04-20T01:48:24.151 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_started": 0, 2024-04-20T01:48:24.151 INFO:teuthology.orchestra.run.smithi044.stdout: "last_epoch_clean": 0, 2024-04-20T01:48:24.151 INFO:teuthology.orchestra.run.smithi044.stdout: "source_version": "0'0", 2024-04-20T01:48:24.151 INFO:teuthology.orchestra.run.smithi044.stdout: "target_version": "0'0" 2024-04-20T01:48:24.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.151 INFO:teuthology.orchestra.run.smithi044.stdout: "last_change": "82", 2024-04-20T01:48:24.152 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend": "0", 2024-04-20T01:48:24.152 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-20T01:48:24.152 INFO:teuthology.orchestra.run.smithi044.stdout: "last_force_op_resend_preluminous": "0", 2024-04-20T01:48:24.152 INFO:teuthology.orchestra.run.smithi044.stdout: "auid": 0, 2024-04-20T01:48:24.152 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_mode": "selfmanaged", 2024-04-20T01:48:24.152 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_seq": 50, 2024-04-20T01:48:24.152 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_epoch": 82, 2024-04-20T01:48:24.152 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_snaps": [], 2024-04-20T01:48:24.152 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps": "[]", 2024-04-20T01:48:24.152 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_bytes": 0, 2024-04-20T01:48:24.152 INFO:teuthology.orchestra.run.smithi044.stdout: "quota_max_objects": 0, 2024-04-20T01:48:24.152 INFO:teuthology.orchestra.run.smithi044.stdout: "tiers": [], 2024-04-20T01:48:24.152 INFO:teuthology.orchestra.run.smithi044.stdout: "tier_of": -1, 2024-04-20T01:48:24.152 INFO:teuthology.orchestra.run.smithi044.stdout: "read_tier": -1, 2024-04-20T01:48:24.152 INFO:teuthology.orchestra.run.smithi044.stdout: "write_tier": -1, 2024-04-20T01:48:24.153 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_mode": "none", 2024-04-20T01:48:24.153 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_bytes": 0, 2024-04-20T01:48:24.153 INFO:teuthology.orchestra.run.smithi044.stdout: "target_max_objects": 0, 2024-04-20T01:48:24.153 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-20T01:48:24.153 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-20T01:48:24.153 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-20T01:48:24.153 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_flush_age": 0, 2024-04-20T01:48:24.153 INFO:teuthology.orchestra.run.smithi044.stdout: "cache_min_evict_age": 0, 2024-04-20T01:48:24.153 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profile": "shecprofile", 2024-04-20T01:48:24.153 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_params": { 2024-04-20T01:48:24.153 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "none" 2024-04-20T01:48:24.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.153 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_period": 0, 2024-04-20T01:48:24.153 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_count": 0, 2024-04-20T01:48:24.153 INFO:teuthology.orchestra.run.smithi044.stdout: "use_gmt_hitset": true, 2024-04-20T01:48:24.154 INFO:teuthology.orchestra.run.smithi044.stdout: "min_read_recency_for_promote": 0, 2024-04-20T01:48:24.154 INFO:teuthology.orchestra.run.smithi044.stdout: "min_write_recency_for_promote": 0, 2024-04-20T01:48:24.154 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_grade_decay_rate": 0, 2024-04-20T01:48:24.154 INFO:teuthology.orchestra.run.smithi044.stdout: "hit_set_search_last_n": 0, 2024-04-20T01:48:24.154 INFO:teuthology.orchestra.run.smithi044.stdout: "grade_table": [], 2024-04-20T01:48:24.154 INFO:teuthology.orchestra.run.smithi044.stdout: "stripe_width": 16384, 2024-04-20T01:48:24.154 INFO:teuthology.orchestra.run.smithi044.stdout: "expected_num_objects": 0, 2024-04-20T01:48:24.154 INFO:teuthology.orchestra.run.smithi044.stdout: "fast_read": false, 2024-04-20T01:48:24.154 INFO:teuthology.orchestra.run.smithi044.stdout: "options": {}, 2024-04-20T01:48:24.154 INFO:teuthology.orchestra.run.smithi044.stdout: "application_metadata": { 2024-04-20T01:48:24.154 INFO:teuthology.orchestra.run.smithi044.stdout: "rados": {} 2024-04-20T01:48:24.154 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.154 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.154 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:48:24.155 INFO:teuthology.orchestra.run.smithi044.stdout: "osds": [ 2024-04-20T01:48:24.155 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.155 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 0, 2024-04-20T01:48:24.155 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "3903b88f-ff2b-4252-9480-92b701b1e59b", 2024-04-20T01:48:24.155 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:24.155 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:24.155 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:24.155 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:24.155 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:24.155 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:24.155 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:24.155 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:24.155 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:24.155 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:24.155 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:24.156 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.156 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.156 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6824", 2024-04-20T01:48:24.156 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:48:24.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.156 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.156 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.156 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6825", 2024-04-20T01:48:24.156 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:48:24.156 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.156 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.156 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:24.156 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.157 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.157 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6826", 2024-04-20T01:48:24.157 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:48:24.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.157 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.157 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6827", 2024-04-20T01:48:24.157 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:48:24.157 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.157 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.157 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:24.157 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.157 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.158 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.158 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6830", 2024-04-20T01:48:24.158 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:48:24.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.158 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.158 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6831", 2024-04-20T01:48:24.158 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:48:24.158 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.158 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.158 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:24.158 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.158 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.159 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.159 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6828", 2024-04-20T01:48:24.159 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:48:24.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.159 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.159 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6829", 2024-04-20T01:48:24.159 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43518 2024-04-20T01:48:24.159 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.159 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.159 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6825/43518", 2024-04-20T01:48:24.159 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6827/43518", 2024-04-20T01:48:24.159 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6831/43518", 2024-04-20T01:48:24.159 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6829/43518", 2024-04-20T01:48:24.160 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:24.160 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:24.160 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:24.160 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.160 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.160 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.160 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 1, 2024-04-20T01:48:24.160 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "c5ff0175-b4f0-40a5-86c7-7a0a351bf070", 2024-04-20T01:48:24.160 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:24.160 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:24.160 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:24.160 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:24.160 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:24.160 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:24.160 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:24.161 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 0, 2024-04-20T01:48:24.161 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:24.161 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:24.161 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:24.161 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.161 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.161 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6816", 2024-04-20T01:48:24.161 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:48:24.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.161 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.161 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6817", 2024-04-20T01:48:24.161 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:48:24.161 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.162 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.162 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:24.162 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.162 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.162 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6818", 2024-04-20T01:48:24.162 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:48:24.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.162 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.162 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.162 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6819", 2024-04-20T01:48:24.162 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:48:24.162 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.162 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.163 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:24.163 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.163 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.163 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6822", 2024-04-20T01:48:24.163 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:48:24.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.163 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.163 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.163 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6823", 2024-04-20T01:48:24.163 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:48:24.163 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.163 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.164 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:24.164 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.164 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.164 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6820", 2024-04-20T01:48:24.164 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:48:24.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.164 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.164 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.164 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6821", 2024-04-20T01:48:24.164 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74361 2024-04-20T01:48:24.164 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.164 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.165 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.121:6817/74361", 2024-04-20T01:48:24.165 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.121:6819/74361", 2024-04-20T01:48:24.165 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.121:6823/74361", 2024-04-20T01:48:24.165 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.121:6821/74361", 2024-04-20T01:48:24.165 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:24.165 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:24.165 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:24.165 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.165 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 2, 2024-04-20T01:48:24.165 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "7af325c1-36b5-420c-a2a9-44c00a002977", 2024-04-20T01:48:24.165 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:24.165 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:24.165 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:24.165 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 0.2259979248046875, 2024-04-20T01:48:24.166 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:24.166 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:24.166 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:24.166 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:24.166 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:24.166 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:24.166 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:24.166 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.166 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.166 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6809", 2024-04-20T01:48:24.166 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:48:24.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.166 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.167 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.167 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6811", 2024-04-20T01:48:24.167 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:48:24.167 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.167 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.167 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:24.167 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.167 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.167 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6814", 2024-04-20T01:48:24.167 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:48:24.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.167 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.168 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6816", 2024-04-20T01:48:24.168 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:48:24.168 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.168 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.168 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:24.168 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.168 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.168 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6822", 2024-04-20T01:48:24.168 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:48:24.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.168 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.168 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.168 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6823", 2024-04-20T01:48:24.169 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:48:24.169 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.169 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.169 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:24.169 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.169 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.169 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6819", 2024-04-20T01:48:24.169 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:48:24.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.169 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.169 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.170 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6821", 2024-04-20T01:48:24.170 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43108 2024-04-20T01:48:24.170 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.170 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.170 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.123:6811/43108", 2024-04-20T01:48:24.170 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.123:6816/43108", 2024-04-20T01:48:24.170 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.123:6823/43108", 2024-04-20T01:48:24.170 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.123:6821/43108", 2024-04-20T01:48:24.170 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:24.170 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:24.170 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:24.170 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 3, 2024-04-20T01:48:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "99334f43-83ea-4c3f-b91d-ae72c8361b8c", 2024-04-20T01:48:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 77, 2024-04-20T01:48:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:24.171 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.172 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.172 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6824", 2024-04-20T01:48:24.172 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:48:24.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.172 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.172 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6825", 2024-04-20T01:48:24.172 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:48:24.172 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.172 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.172 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:24.172 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6826", 2024-04-20T01:48:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:48:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6827", 2024-04-20T01:48:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:48:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.173 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.174 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6830", 2024-04-20T01:48:24.174 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:48:24.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.174 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.174 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6831", 2024-04-20T01:48:24.174 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:48:24.174 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.174 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.174 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:24.174 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.175 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.175 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6828", 2024-04-20T01:48:24.175 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:48:24.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.175 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.175 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6829", 2024-04-20T01:48:24.175 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74545 2024-04-20T01:48:24.175 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.175 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.175 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.165:6825/74545", 2024-04-20T01:48:24.175 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.165:6827/74545", 2024-04-20T01:48:24.175 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.165:6831/74545", 2024-04-20T01:48:24.176 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.165:6829/74545", 2024-04-20T01:48:24.176 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:24.176 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:24.176 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:24.176 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.176 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 4, 2024-04-20T01:48:24.176 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "8764d799-af67-4e6c-889f-9fdd91201cc2", 2024-04-20T01:48:24.176 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:24.176 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:24.176 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:24.176 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:24.176 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:24.177 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:24.177 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:24.177 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:24.177 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:24.177 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:24.177 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:24.177 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.177 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.177 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6809", 2024-04-20T01:48:24.177 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:48:24.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.177 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.177 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6811", 2024-04-20T01:48:24.178 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:48:24.178 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.178 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.178 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:24.178 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.178 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.178 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6813", 2024-04-20T01:48:24.178 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:48:24.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.178 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.178 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6815", 2024-04-20T01:48:24.179 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:48:24.179 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.179 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.179 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:24.179 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.179 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.179 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6821", 2024-04-20T01:48:24.179 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:48:24.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.179 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.179 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6823", 2024-04-20T01:48:24.179 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:48:24.180 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.180 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.180 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:24.180 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.180 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.180 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6817", 2024-04-20T01:48:24.180 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:48:24.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.180 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.180 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6819", 2024-04-20T01:48:24.180 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43517 2024-04-20T01:48:24.180 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.181 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.181 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6811/43517", 2024-04-20T01:48:24.181 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6815/43517", 2024-04-20T01:48:24.181 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6823/43517", 2024-04-20T01:48:24.181 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6819/43517", 2024-04-20T01:48:24.181 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:24.181 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:24.181 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:24.181 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.181 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 5, 2024-04-20T01:48:24.181 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "2556b2fa-4dcc-479c-a379-6f9fe96e9c77", 2024-04-20T01:48:24.181 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:24.182 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:24.182 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:24.182 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:24.182 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:24.182 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:24.182 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:24.182 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:24.182 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:24.182 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:24.182 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:24.182 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.182 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.182 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6800", 2024-04-20T01:48:24.183 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:48:24.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.183 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.183 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6801", 2024-04-20T01:48:24.183 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:48:24.183 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.183 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.183 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:24.183 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.183 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.183 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6802", 2024-04-20T01:48:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:48:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6803", 2024-04-20T01:48:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:48:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6806", 2024-04-20T01:48:24.184 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:48:24.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.185 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.185 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6807", 2024-04-20T01:48:24.185 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:48:24.185 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.185 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.185 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:24.185 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.185 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.185 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6804", 2024-04-20T01:48:24.185 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:48:24.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.186 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.186 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6805", 2024-04-20T01:48:24.186 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74354 2024-04-20T01:48:24.186 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.186 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.186 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.121:6801/74354", 2024-04-20T01:48:24.186 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.121:6803/74354", 2024-04-20T01:48:24.186 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.121:6807/74354", 2024-04-20T01:48:24.186 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.121:6805/74354", 2024-04-20T01:48:24.186 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:24.186 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:24.186 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:24.187 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.187 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 6, 2024-04-20T01:48:24.187 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "baeaae3a-fff4-4ccc-a08a-ff32db4531ba", 2024-04-20T01:48:24.187 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:24.187 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:24.187 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:24.187 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:24.187 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:24.187 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:24.187 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:24.187 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:24.187 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:24.187 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:24.188 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:24.188 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.188 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.188 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6800", 2024-04-20T01:48:24.188 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:48:24.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.188 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.188 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6801", 2024-04-20T01:48:24.188 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:48:24.188 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.188 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.189 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:24.189 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.189 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.189 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6802", 2024-04-20T01:48:24.189 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:48:24.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.189 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.189 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6803", 2024-04-20T01:48:24.189 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:48:24.189 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.189 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.190 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:24.190 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.190 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.190 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6806", 2024-04-20T01:48:24.190 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:48:24.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.190 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.190 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6807", 2024-04-20T01:48:24.190 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:48:24.190 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.190 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.191 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:24.191 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.191 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.191 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6804", 2024-04-20T01:48:24.191 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:48:24.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.191 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.191 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6805", 2024-04-20T01:48:24.191 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43107 2024-04-20T01:48:24.191 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.191 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.191 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.123:6801/43107", 2024-04-20T01:48:24.192 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.123:6803/43107", 2024-04-20T01:48:24.192 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.123:6807/43107", 2024-04-20T01:48:24.192 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.123:6805/43107", 2024-04-20T01:48:24.192 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:24.192 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:24.192 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:24.192 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.192 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 7, 2024-04-20T01:48:24.192 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "0019f005-3e62-4d9d-b334-f1d347a484ce", 2024-04-20T01:48:24.192 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:24.192 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:24.192 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:24.192 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:24.193 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:24.193 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:24.193 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:24.193 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:24.193 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:24.193 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:24.193 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:24.193 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.193 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.193 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6800", 2024-04-20T01:48:24.193 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:48:24.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.193 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.194 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6801", 2024-04-20T01:48:24.194 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:48:24.194 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.194 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.194 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:24.194 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.194 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.194 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6802", 2024-04-20T01:48:24.194 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:48:24.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.195 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.195 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6803", 2024-04-20T01:48:24.195 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:48:24.195 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.195 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.195 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:24.195 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.195 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.195 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6806", 2024-04-20T01:48:24.195 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:48:24.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.195 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.196 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6807", 2024-04-20T01:48:24.196 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:48:24.196 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.196 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.196 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:24.196 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.196 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.196 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6804", 2024-04-20T01:48:24.196 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:48:24.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.196 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.196 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6805", 2024-04-20T01:48:24.197 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74543 2024-04-20T01:48:24.197 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.197 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.197 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.165:6801/74543", 2024-04-20T01:48:24.197 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.165:6803/74543", 2024-04-20T01:48:24.197 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.165:6807/74543", 2024-04-20T01:48:24.197 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.165:6805/74543", 2024-04-20T01:48:24.197 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:24.197 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:24.197 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:24.198 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.198 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 8, 2024-04-20T01:48:24.198 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "6f5200a3-8980-4241-a6fc-ef604fa612b2", 2024-04-20T01:48:24.198 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:24.198 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:24.198 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:24.198 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:24.198 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:24.198 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:24.198 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:24.198 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 57, 2024-04-20T01:48:24.198 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:24.199 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:24.199 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:24.199 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.199 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.199 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6808", 2024-04-20T01:48:24.199 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:48:24.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.199 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.199 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6810", 2024-04-20T01:48:24.199 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:48:24.199 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.199 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.200 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:24.200 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.200 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.200 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6812", 2024-04-20T01:48:24.200 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:48:24.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.200 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.200 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6814", 2024-04-20T01:48:24.200 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:48:24.200 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.200 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.201 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:24.201 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.201 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.201 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6820", 2024-04-20T01:48:24.201 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:48:24.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.201 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.201 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6822", 2024-04-20T01:48:24.201 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:48:24.201 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.201 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.201 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:24.202 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.202 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.202 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6816", 2024-04-20T01:48:24.202 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:48:24.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.202 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.202 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6818", 2024-04-20T01:48:24.202 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43515 2024-04-20T01:48:24.202 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.202 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.202 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6810/43515", 2024-04-20T01:48:24.202 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6814/43515", 2024-04-20T01:48:24.203 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6822/43515", 2024-04-20T01:48:24.203 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6818/43515", 2024-04-20T01:48:24.203 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:24.203 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:24.203 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:24.203 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.203 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 9, 2024-04-20T01:48:24.203 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "29cf9aaa-3a99-4605-92f4-64f1f6a1728d", 2024-04-20T01:48:24.203 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:24.203 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:24.203 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:24.203 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:24.203 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:24.204 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:24.204 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:24.204 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 0, 2024-04-20T01:48:24.204 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:24.204 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:24.204 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:24.204 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.204 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.204 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6824", 2024-04-20T01:48:24.204 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:48:24.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.204 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.205 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6825", 2024-04-20T01:48:24.205 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:48:24.205 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.205 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.205 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:24.205 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.205 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.205 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6826", 2024-04-20T01:48:24.205 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:48:24.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.205 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.206 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6827", 2024-04-20T01:48:24.206 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:48:24.206 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.206 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.206 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:24.206 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.206 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.206 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6830", 2024-04-20T01:48:24.206 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:48:24.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.206 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.207 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6831", 2024-04-20T01:48:24.207 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:48:24.207 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.207 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.207 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:24.207 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.207 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.207 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6828", 2024-04-20T01:48:24.207 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:48:24.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.207 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.207 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6829", 2024-04-20T01:48:24.208 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74379 2024-04-20T01:48:24.208 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.208 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.208 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.121:6825/74379", 2024-04-20T01:48:24.208 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.121:6827/74379", 2024-04-20T01:48:24.208 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.121:6831/74379", 2024-04-20T01:48:24.208 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.121:6829/74379", 2024-04-20T01:48:24.208 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:24.208 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:24.208 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:24.208 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.208 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 10, 2024-04-20T01:48:24.209 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "705906c4-1514-4754-ac40-45226a837180", 2024-04-20T01:48:24.209 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:24.209 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:24.209 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:24.209 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:24.209 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:24.209 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:24.209 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:24.209 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:24.209 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:24.209 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:24.209 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:24.209 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.209 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.210 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6808", 2024-04-20T01:48:24.210 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:48:24.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.210 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.210 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6810", 2024-04-20T01:48:24.210 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:48:24.210 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.210 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.210 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:24.210 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.210 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.211 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6812", 2024-04-20T01:48:24.211 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:48:24.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.211 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.211 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6813", 2024-04-20T01:48:24.211 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:48:24.211 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.211 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.211 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:24.211 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.211 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.211 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6818", 2024-04-20T01:48:24.212 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:48:24.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.212 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.212 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6820", 2024-04-20T01:48:24.212 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:48:24.212 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.212 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.212 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:24.212 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.212 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.212 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6815", 2024-04-20T01:48:24.212 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:48:24.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.213 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.213 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6817", 2024-04-20T01:48:24.213 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43109 2024-04-20T01:48:24.213 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.213 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.213 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.123:6810/43109", 2024-04-20T01:48:24.213 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.123:6813/43109", 2024-04-20T01:48:24.213 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.123:6820/43109", 2024-04-20T01:48:24.213 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.123:6817/43109", 2024-04-20T01:48:24.213 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:24.213 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:24.213 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:24.214 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.214 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 11, 2024-04-20T01:48:24.214 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "b45f2645-72d1-4d72-89e9-2212f5aa074c", 2024-04-20T01:48:24.214 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:24.214 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:24.214 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:24.214 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:24.214 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:24.214 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:24.214 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:24.214 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:24.214 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:24.215 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:24.215 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:24.215 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.215 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.215 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6808", 2024-04-20T01:48:24.215 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:48:24.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.215 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.215 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6809", 2024-04-20T01:48:24.215 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:48:24.215 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.215 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.216 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:24.216 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.216 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.216 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6810", 2024-04-20T01:48:24.216 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:48:24.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.216 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.216 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6811", 2024-04-20T01:48:24.216 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:48:24.216 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.216 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.217 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:24.217 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.217 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.217 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6814", 2024-04-20T01:48:24.217 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:48:24.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.217 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.217 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6815", 2024-04-20T01:48:24.217 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:48:24.217 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.217 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.218 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:24.218 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.218 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.218 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6812", 2024-04-20T01:48:24.218 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:48:24.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.218 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.218 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6813", 2024-04-20T01:48:24.218 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74544 2024-04-20T01:48:24.218 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.218 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.219 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.165:6809/74544", 2024-04-20T01:48:24.219 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.165:6811/74544", 2024-04-20T01:48:24.219 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.165:6815/74544", 2024-04-20T01:48:24.219 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.165:6813/74544", 2024-04-20T01:48:24.219 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:24.219 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:24.219 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:24.219 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.219 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 12, 2024-04-20T01:48:24.219 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "3ea410b9-a368-467b-a179-2e3aba562d63", 2024-04-20T01:48:24.219 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:24.219 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:24.220 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:24.220 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:24.220 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:24.220 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:24.220 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:24.220 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 0, 2024-04-20T01:48:24.220 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:24.220 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:24.220 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:24.220 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.220 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.220 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6800", 2024-04-20T01:48:24.220 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:48:24.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.221 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.221 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6801", 2024-04-20T01:48:24.221 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:48:24.221 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.221 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.221 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:24.221 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.221 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.221 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6802", 2024-04-20T01:48:24.221 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:48:24.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.222 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.222 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6803", 2024-04-20T01:48:24.222 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:48:24.222 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.222 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.222 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:24.222 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.222 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.222 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6806", 2024-04-20T01:48:24.223 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:48:24.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.223 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.223 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6807", 2024-04-20T01:48:24.223 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:48:24.223 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.223 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.223 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:24.223 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.223 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.223 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6804", 2024-04-20T01:48:24.224 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:48:24.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.224 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.224 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6805", 2024-04-20T01:48:24.224 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43516 2024-04-20T01:48:24.224 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.224 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.224 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.44:6801/43516", 2024-04-20T01:48:24.224 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.44:6803/43516", 2024-04-20T01:48:24.224 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.44:6807/43516", 2024-04-20T01:48:24.224 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.44:6805/43516", 2024-04-20T01:48:24.224 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:24.224 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:24.225 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:24.225 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.225 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 13, 2024-04-20T01:48:24.225 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "15a5154b-e7cb-4b57-8468-dcb048d93f63", 2024-04-20T01:48:24.225 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:24.225 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:24.225 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:24.225 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:24.225 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:24.225 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:24.225 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 22, 2024-04-20T01:48:24.225 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:24.225 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:24.226 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:24.226 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:24.226 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.226 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.226 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6808", 2024-04-20T01:48:24.226 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:48:24.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.226 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.226 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6809", 2024-04-20T01:48:24.226 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:48:24.226 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.226 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.227 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:24.227 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.227 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.227 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6810", 2024-04-20T01:48:24.227 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:48:24.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.227 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.227 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6811", 2024-04-20T01:48:24.227 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:48:24.227 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.227 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.228 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:24.228 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.228 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.228 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.228 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6814", 2024-04-20T01:48:24.228 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:48:24.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.228 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.228 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.228 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6815", 2024-04-20T01:48:24.228 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:48:24.228 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.228 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.229 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:24.229 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.229 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.229 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6812", 2024-04-20T01:48:24.229 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:48:24.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.229 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.229 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.121:6813", 2024-04-20T01:48:24.229 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74355 2024-04-20T01:48:24.229 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.229 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.230 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.121:6809/74355", 2024-04-20T01:48:24.230 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.121:6811/74355", 2024-04-20T01:48:24.230 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.121:6815/74355", 2024-04-20T01:48:24.230 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.121:6813/74355", 2024-04-20T01:48:24.230 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:24.230 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:24.230 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:24.230 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.230 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 14, 2024-04-20T01:48:24.230 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "23b53da2-f848-4e76-9632-4d9effd5b430", 2024-04-20T01:48:24.230 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:24.231 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:24.231 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:24.231 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:24.231 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:24.231 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:24.231 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:24.231 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 30, 2024-04-20T01:48:24.231 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:24.231 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:24.231 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:24.231 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.231 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.231 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6824", 2024-04-20T01:48:24.231 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:48:24.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.232 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.232 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6825", 2024-04-20T01:48:24.232 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:48:24.232 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.232 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.232 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:24.232 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.232 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.232 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6826", 2024-04-20T01:48:24.232 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:48:24.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.233 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.233 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6827", 2024-04-20T01:48:24.233 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:48:24.233 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.233 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.233 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:24.233 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.233 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.233 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6830", 2024-04-20T01:48:24.233 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:48:24.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.234 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.234 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6831", 2024-04-20T01:48:24.234 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:48:24.234 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.234 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.234 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:24.234 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.234 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.234 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6828", 2024-04-20T01:48:24.234 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:48:24.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.235 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.235 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.123:6829", 2024-04-20T01:48:24.235 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 43110 2024-04-20T01:48:24.235 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.235 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.235 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.123:6825/43110", 2024-04-20T01:48:24.235 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.123:6827/43110", 2024-04-20T01:48:24.235 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.123:6831/43110", 2024-04-20T01:48:24.235 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.123:6829/43110", 2024-04-20T01:48:24.235 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:24.235 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:24.235 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:24.235 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.236 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.236 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 15, 2024-04-20T01:48:24.236 INFO:teuthology.orchestra.run.smithi044.stdout: "uuid": "546efc0f-82a3-47d2-aeec-2d849eb1faa1", 2024-04-20T01:48:24.236 INFO:teuthology.orchestra.run.smithi044.stdout: "up": 1, 2024-04-20T01:48:24.236 INFO:teuthology.orchestra.run.smithi044.stdout: "in": 1, 2024-04-20T01:48:24.236 INFO:teuthology.orchestra.run.smithi044.stdout: "weight": 1, 2024-04-20T01:48:24.236 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_affinity": 1, 2024-04-20T01:48:24.236 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_begin": 0, 2024-04-20T01:48:24.236 INFO:teuthology.orchestra.run.smithi044.stdout: "last_clean_end": 0, 2024-04-20T01:48:24.236 INFO:teuthology.orchestra.run.smithi044.stdout: "up_from": 21, 2024-04-20T01:48:24.236 INFO:teuthology.orchestra.run.smithi044.stdout: "up_thru": 0, 2024-04-20T01:48:24.236 INFO:teuthology.orchestra.run.smithi044.stdout: "down_at": 0, 2024-04-20T01:48:24.236 INFO:teuthology.orchestra.run.smithi044.stdout: "lost_at": 0, 2024-04-20T01:48:24.236 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addrs": { 2024-04-20T01:48:24.237 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.237 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.237 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6816", 2024-04-20T01:48:24.237 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:48:24.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.237 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.237 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6817", 2024-04-20T01:48:24.237 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:48:24.237 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.237 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.237 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addrs": { 2024-04-20T01:48:24.237 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.238 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.238 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6818", 2024-04-20T01:48:24.238 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:48:24.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.238 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.238 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6819", 2024-04-20T01:48:24.238 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:48:24.238 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.238 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.238 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addrs": { 2024-04-20T01:48:24.238 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.239 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.239 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6822", 2024-04-20T01:48:24.239 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:48:24.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.239 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.239 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6823", 2024-04-20T01:48:24.239 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:48:24.239 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.239 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.239 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addrs": { 2024-04-20T01:48:24.239 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-04-20T01:48:24.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.240 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-04-20T01:48:24.240 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6820", 2024-04-20T01:48:24.240 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:48:24.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.240 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-20T01:48:24.240 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.165:6821", 2024-04-20T01:48:24.240 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 74562 2024-04-20T01:48:24.240 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.240 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.240 INFO:teuthology.orchestra.run.smithi044.stdout: "public_addr": "172.21.15.165:6817/74562", 2024-04-20T01:48:24.240 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_addr": "172.21.15.165:6819/74562", 2024-04-20T01:48:24.240 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_back_addr": "172.21.15.165:6823/74562", 2024-04-20T01:48:24.240 INFO:teuthology.orchestra.run.smithi044.stdout: "heartbeat_front_addr": "172.21.15.165:6821/74562", 2024-04-20T01:48:24.241 INFO:teuthology.orchestra.run.smithi044.stdout: "state": [ 2024-04-20T01:48:24.241 INFO:teuthology.orchestra.run.smithi044.stdout: "exists", 2024-04-20T01:48:24.241 INFO:teuthology.orchestra.run.smithi044.stdout: "up" 2024-04-20T01:48:24.241 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.241 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.241 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:48:24.241 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_xinfo": [ 2024-04-20T01:48:24.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.241 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 0, 2024-04-20T01:48:24.241 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:24.241 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:24.241 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:24.241 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:24.241 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:24.242 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.124928+0000", 2024-04-20T01:48:24.242 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:24.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.242 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 1, 2024-04-20T01:48:24.242 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:24.242 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:24.242 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:24.242 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:24.242 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:24.242 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:48:19.984157+0000", 2024-04-20T01:48:24.242 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:24.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.242 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 2, 2024-04-20T01:48:24.243 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:24.243 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:24.243 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:24.243 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:24.243 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:24.243 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.008326+0000", 2024-04-20T01:48:24.243 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:24.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.243 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 3, 2024-04-20T01:48:24.243 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:24.243 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:24.243 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:24.243 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:24.243 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:24.244 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:48:15.114260+0000", 2024-04-20T01:48:24.244 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:24.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.244 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 4, 2024-04-20T01:48:24.244 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:24.244 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:24.244 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:24.244 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:24.244 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:24.244 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.117767+0000", 2024-04-20T01:48:24.244 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:24.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.244 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 5, 2024-04-20T01:48:24.245 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:24.245 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:24.245 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:24.245 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:24.245 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:24.245 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:12.999445+0000", 2024-04-20T01:48:24.245 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:24.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.245 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 6, 2024-04-20T01:48:24.245 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:24.245 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:24.245 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:24.245 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:24.245 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:24.246 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.032367+0000", 2024-04-20T01:48:24.246 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:24.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.246 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 7, 2024-04-20T01:48:24.246 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:24.246 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:24.246 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:24.246 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:24.246 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:24.246 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.058776+0000", 2024-04-20T01:48:24.246 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:24.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.247 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 8, 2024-04-20T01:48:24.247 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:24.247 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:24.247 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:24.247 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:24.247 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:24.247 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.113013+0000", 2024-04-20T01:48:24.247 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:24.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.247 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 9, 2024-04-20T01:48:24.247 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:24.247 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:24.247 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:24.247 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:24.248 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:24.248 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:48:23.704764+0000", 2024-04-20T01:48:24.248 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:24.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.248 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 10, 2024-04-20T01:48:24.248 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:24.248 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:24.248 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:24.248 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:24.248 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:24.248 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.092533+0000", 2024-04-20T01:48:24.248 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:24.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.249 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 11, 2024-04-20T01:48:24.249 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:24.249 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:24.249 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:24.249 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:24.249 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:24.249 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:48:14.370592+0000", 2024-04-20T01:48:24.249 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:24.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.249 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 12, 2024-04-20T01:48:24.249 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:24.249 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:24.249 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:24.249 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:24.250 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:24.250 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.094847+0000", 2024-04-20T01:48:24.250 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:24.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.250 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.250 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 13, 2024-04-20T01:48:24.250 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:24.250 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:24.250 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:24.250 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:24.250 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:24.250 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.059856+0000", 2024-04-20T01:48:24.250 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:24.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.251 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 14, 2024-04-20T01:48:24.251 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:24.251 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:24.251 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:24.251 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:24.251 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:24.251 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:47:13.128995+0000", 2024-04-20T01:48:24.251 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:24.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.251 INFO:teuthology.orchestra.run.smithi044.stdout: "osd": 15, 2024-04-20T01:48:24.251 INFO:teuthology.orchestra.run.smithi044.stdout: "down_stamp": "0.000000", 2024-04-20T01:48:24.251 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_probability": 0, 2024-04-20T01:48:24.252 INFO:teuthology.orchestra.run.smithi044.stdout: "laggy_interval": 0, 2024-04-20T01:48:24.252 INFO:teuthology.orchestra.run.smithi044.stdout: "features": 4540138322906710015, 2024-04-20T01:48:24.252 INFO:teuthology.orchestra.run.smithi044.stdout: "old_weight": 0, 2024-04-20T01:48:24.252 INFO:teuthology.orchestra.run.smithi044.stdout: "last_purged_snaps_scrub": "2024-04-20T01:48:17.320410+0000", 2024-04-20T01:48:24.252 INFO:teuthology.orchestra.run.smithi044.stdout: "dead_epoch": 0 2024-04-20T01:48:24.252 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.252 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:48:24.252 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap": [], 2024-04-20T01:48:24.252 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap_items": [], 2024-04-20T01:48:24.252 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_upmap_primaries": [], 2024-04-20T01:48:24.252 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_temp": [], 2024-04-20T01:48:24.252 INFO:teuthology.orchestra.run.smithi044.stdout: "primary_temp": [], 2024-04-20T01:48:24.252 INFO:teuthology.orchestra.run.smithi044.stdout: "blocklist": {}, 2024-04-20T01:48:24.252 INFO:teuthology.orchestra.run.smithi044.stdout: "range_blocklist": {}, 2024-04-20T01:48:24.252 INFO:teuthology.orchestra.run.smithi044.stdout: "erasure_code_profiles": { 2024-04-20T01:48:24.253 INFO:teuthology.orchestra.run.smithi044.stdout: "default": { 2024-04-20T01:48:24.253 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-failure-domain": "osd", 2024-04-20T01:48:24.253 INFO:teuthology.orchestra.run.smithi044.stdout: "k": "2", 2024-04-20T01:48:24.253 INFO:teuthology.orchestra.run.smithi044.stdout: "m": "1", 2024-04-20T01:48:24.253 INFO:teuthology.orchestra.run.smithi044.stdout: "plugin": "jerasure", 2024-04-20T01:48:24.253 INFO:teuthology.orchestra.run.smithi044.stdout: "technique": "reed_sol_van" 2024-04-20T01:48:24.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.253 INFO:teuthology.orchestra.run.smithi044.stdout: "shecprofile": { 2024-04-20T01:48:24.253 INFO:teuthology.orchestra.run.smithi044.stdout: "c": "2", 2024-04-20T01:48:24.253 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-device-class": "", 2024-04-20T01:48:24.253 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-failure-domain": "osd", 2024-04-20T01:48:24.253 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-root": "default", 2024-04-20T01:48:24.253 INFO:teuthology.orchestra.run.smithi044.stdout: "k": "4", 2024-04-20T01:48:24.253 INFO:teuthology.orchestra.run.smithi044.stdout: "m": "3", 2024-04-20T01:48:24.253 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "shecprofile", 2024-04-20T01:48:24.253 INFO:teuthology.orchestra.run.smithi044.stdout: "plugin": "shec", 2024-04-20T01:48:24.254 INFO:teuthology.orchestra.run.smithi044.stdout: "technique": "multiple" 2024-04-20T01:48:24.254 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.254 INFO:teuthology.orchestra.run.smithi044.stdout: "removed_snaps_queue": [ 2024-04-20T01:48:24.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.254 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 3, 2024-04-20T01:48:24.254 INFO:teuthology.orchestra.run.smithi044.stdout: "snaps": [ 2024-04-20T01:48:24.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.254 INFO:teuthology.orchestra.run.smithi044.stdout: "begin": 42, 2024-04-20T01:48:24.254 INFO:teuthology.orchestra.run.smithi044.stdout: "length": 1 2024-04-20T01:48:24.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:24.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.254 INFO:teuthology.orchestra.run.smithi044.stdout: "begin": 45, 2024-04-20T01:48:24.254 INFO:teuthology.orchestra.run.smithi044.stdout: "length": 2 2024-04-20T01:48:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:48:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: "new_removed_snaps": [ 2024-04-20T01:48:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: "pool": 3, 2024-04-20T01:48:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: "snaps": [ 2024-04-20T01:48:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: "begin": 46, 2024-04-20T01:48:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: "length": 1 2024-04-20T01:48:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:24.255 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.256 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-04-20T01:48:24.256 INFO:teuthology.orchestra.run.smithi044.stdout: "new_purged_snaps": [], 2024-04-20T01:48:24.256 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_node_flags": {}, 2024-04-20T01:48:24.256 INFO:teuthology.orchestra.run.smithi044.stdout: "device_class_flags": {}, 2024-04-20T01:48:24.256 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode": { 2024-04-20T01:48:24.256 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode_enabled": false, 2024-04-20T01:48:24.256 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_bucket_count": 0, 2024-04-20T01:48:24.256 INFO:teuthology.orchestra.run.smithi044.stdout: "degraded_stretch_mode": 0, 2024-04-20T01:48:24.256 INFO:teuthology.orchestra.run.smithi044.stdout: "recovering_stretch_mode": 0, 2024-04-20T01:48:24.256 INFO:teuthology.orchestra.run.smithi044.stdout: "stretch_mode_bucket": 0 2024-04-20T01:48:24.256 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:24.256 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:24.258 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:24.272+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:24.258 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:24.259 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:24.260 INFO:tasks.thrashosds.thrasher:j is {'epoch': 82, 'fsid': '9c2f64c1-aa35-4337-9f5c-081b9e3a2938', 'created': '2024-04-20T01:47:02.934241+0000', 'modified': '2024-04-20T01:48:23.709271+0000', 'last_up_change': '2024-04-20T01:47:15.986754+0000', 'last_in_change': '2024-04-20T01:47:07.894719+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T01:47:15.195181+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 15.380000114440918, 'score_stable': 15.380000114440918, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.951624870300293, 'average_primary_affinity': 0.949999988079071, 'average_primary_affinity_weighted': 0.949999988079071}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-04-20T01:47:18.448004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 0.6899999976158142, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 0.951624870300293, 'average_primary_affinity': 0.949999988079071, 'average_primary_affinity_weighted': 0.949999988079071}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-04-20T01:47:30.734962+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 7, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 50, 'snap_epoch': 82, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'shecprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '3903b88f-ff2b-4252-9480-92b701b1e59b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6824', 'nonce': 43518}, {'type': 'v1', 'addr': '172.21.15.44:6825', 'nonce': 43518}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6826', 'nonce': 43518}, {'type': 'v1', 'addr': '172.21.15.44:6827', 'nonce': 43518}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6830', 'nonce': 43518}, {'type': 'v1', 'addr': '172.21.15.44:6831', 'nonce': 43518}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6828', 'nonce': 43518}, {'type': 'v1', 'addr': '172.21.15.44:6829', 'nonce': 43518}]}, 'public_addr': '172.21.15.44:6825/43518', 'cluster_addr': '172.21.15.44:6827/43518', 'heartbeat_back_addr': '172.21.15.44:6831/43518', 'heartbeat_front_addr': '172.21.15.44:6829/43518', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c5ff0175-b4f0-40a5-86c7-7a0a351bf070', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6816', 'nonce': 74361}, {'type': 'v1', 'addr': '172.21.15.121:6817', 'nonce': 74361}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6818', 'nonce': 74361}, {'type': 'v1', 'addr': '172.21.15.121:6819', 'nonce': 74361}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6822', 'nonce': 74361}, {'type': 'v1', 'addr': '172.21.15.121:6823', 'nonce': 74361}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6820', 'nonce': 74361}, {'type': 'v1', 'addr': '172.21.15.121:6821', 'nonce': 74361}]}, 'public_addr': '172.21.15.121:6817/74361', 'cluster_addr': '172.21.15.121:6819/74361', 'heartbeat_back_addr': '172.21.15.121:6823/74361', 'heartbeat_front_addr': '172.21.15.121:6821/74361', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '7af325c1-36b5-420c-a2a9-44c00a002977', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.2259979248046875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6809', 'nonce': 43108}, {'type': 'v1', 'addr': '172.21.15.123:6811', 'nonce': 43108}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6814', 'nonce': 43108}, {'type': 'v1', 'addr': '172.21.15.123:6816', 'nonce': 43108}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6822', 'nonce': 43108}, {'type': 'v1', 'addr': '172.21.15.123:6823', 'nonce': 43108}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6819', 'nonce': 43108}, {'type': 'v1', 'addr': '172.21.15.123:6821', 'nonce': 43108}]}, 'public_addr': '172.21.15.123:6811/43108', 'cluster_addr': '172.21.15.123:6816/43108', 'heartbeat_back_addr': '172.21.15.123:6823/43108', 'heartbeat_front_addr': '172.21.15.123:6821/43108', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '99334f43-83ea-4c3f-b91d-ae72c8361b8c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6824', 'nonce': 74545}, {'type': 'v1', 'addr': '172.21.15.165:6825', 'nonce': 74545}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6826', 'nonce': 74545}, {'type': 'v1', 'addr': '172.21.15.165:6827', 'nonce': 74545}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6830', 'nonce': 74545}, {'type': 'v1', 'addr': '172.21.15.165:6831', 'nonce': 74545}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6828', 'nonce': 74545}, {'type': 'v1', 'addr': '172.21.15.165:6829', 'nonce': 74545}]}, 'public_addr': '172.21.15.165:6825/74545', 'cluster_addr': '172.21.15.165:6827/74545', 'heartbeat_back_addr': '172.21.15.165:6831/74545', 'heartbeat_front_addr': '172.21.15.165:6829/74545', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8764d799-af67-4e6c-889f-9fdd91201cc2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6809', 'nonce': 43517}, {'type': 'v1', 'addr': '172.21.15.44:6811', 'nonce': 43517}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6813', 'nonce': 43517}, {'type': 'v1', 'addr': '172.21.15.44:6815', 'nonce': 43517}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6821', 'nonce': 43517}, {'type': 'v1', 'addr': '172.21.15.44:6823', 'nonce': 43517}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6817', 'nonce': 43517}, {'type': 'v1', 'addr': '172.21.15.44:6819', 'nonce': 43517}]}, 'public_addr': '172.21.15.44:6811/43517', 'cluster_addr': '172.21.15.44:6815/43517', 'heartbeat_back_addr': '172.21.15.44:6823/43517', 'heartbeat_front_addr': '172.21.15.44:6819/43517', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '2556b2fa-4dcc-479c-a379-6f9fe96e9c77', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6800', 'nonce': 74354}, {'type': 'v1', 'addr': '172.21.15.121:6801', 'nonce': 74354}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6802', 'nonce': 74354}, {'type': 'v1', 'addr': '172.21.15.121:6803', 'nonce': 74354}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6806', 'nonce': 74354}, {'type': 'v1', 'addr': '172.21.15.121:6807', 'nonce': 74354}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6804', 'nonce': 74354}, {'type': 'v1', 'addr': '172.21.15.121:6805', 'nonce': 74354}]}, 'public_addr': '172.21.15.121:6801/74354', 'cluster_addr': '172.21.15.121:6803/74354', 'heartbeat_back_addr': '172.21.15.121:6807/74354', 'heartbeat_front_addr': '172.21.15.121:6805/74354', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'baeaae3a-fff4-4ccc-a08a-ff32db4531ba', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6800', 'nonce': 43107}, {'type': 'v1', 'addr': '172.21.15.123:6801', 'nonce': 43107}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6802', 'nonce': 43107}, {'type': 'v1', 'addr': '172.21.15.123:6803', 'nonce': 43107}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6806', 'nonce': 43107}, {'type': 'v1', 'addr': '172.21.15.123:6807', 'nonce': 43107}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6804', 'nonce': 43107}, {'type': 'v1', 'addr': '172.21.15.123:6805', 'nonce': 43107}]}, 'public_addr': '172.21.15.123:6801/43107', 'cluster_addr': '172.21.15.123:6803/43107', 'heartbeat_back_addr': '172.21.15.123:6807/43107', 'heartbeat_front_addr': '172.21.15.123:6805/43107', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '0019f005-3e62-4d9d-b334-f1d347a484ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6800', 'nonce': 74543}, {'type': 'v1', 'addr': '172.21.15.165:6801', 'nonce': 74543}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6802', 'nonce': 74543}, {'type': 'v1', 'addr': '172.21.15.165:6803', 'nonce': 74543}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6806', 'nonce': 74543}, {'type': 'v1', 'addr': '172.21.15.165:6807', 'nonce': 74543}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6804', 'nonce': 74543}, {'type': 'v1', 'addr': '172.21.15.165:6805', 'nonce': 74543}]}, 'public_addr': '172.21.15.165:6801/74543', 'cluster_addr': '172.21.15.165:6803/74543', 'heartbeat_back_addr': '172.21.15.165:6807/74543', 'heartbeat_front_addr': '172.21.15.165:6805/74543', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '6f5200a3-8980-4241-a6fc-ef604fa612b2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6808', 'nonce': 43515}, {'type': 'v1', 'addr': '172.21.15.44:6810', 'nonce': 43515}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6812', 'nonce': 43515}, {'type': 'v1', 'addr': '172.21.15.44:6814', 'nonce': 43515}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6820', 'nonce': 43515}, {'type': 'v1', 'addr': '172.21.15.44:6822', 'nonce': 43515}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6816', 'nonce': 43515}, {'type': 'v1', 'addr': '172.21.15.44:6818', 'nonce': 43515}]}, 'public_addr': '172.21.15.44:6810/43515', 'cluster_addr': '172.21.15.44:6814/43515', 'heartbeat_back_addr': '172.21.15.44:6822/43515', 'heartbeat_front_addr': '172.21.15.44:6818/43515', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '29cf9aaa-3a99-4605-92f4-64f1f6a1728d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6824', 'nonce': 74379}, {'type': 'v1', 'addr': '172.21.15.121:6825', 'nonce': 74379}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6826', 'nonce': 74379}, {'type': 'v1', 'addr': '172.21.15.121:6827', 'nonce': 74379}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6830', 'nonce': 74379}, {'type': 'v1', 'addr': '172.21.15.121:6831', 'nonce': 74379}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6828', 'nonce': 74379}, {'type': 'v1', 'addr': '172.21.15.121:6829', 'nonce': 74379}]}, 'public_addr': '172.21.15.121:6825/74379', 'cluster_addr': '172.21.15.121:6827/74379', 'heartbeat_back_addr': '172.21.15.121:6831/74379', 'heartbeat_front_addr': '172.21.15.121:6829/74379', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '705906c4-1514-4754-ac40-45226a837180', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6808', 'nonce': 43109}, {'type': 'v1', 'addr': '172.21.15.123:6810', 'nonce': 43109}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6812', 'nonce': 43109}, {'type': 'v1', 'addr': '172.21.15.123:6813', 'nonce': 43109}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6818', 'nonce': 43109}, {'type': 'v1', 'addr': '172.21.15.123:6820', 'nonce': 43109}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6815', 'nonce': 43109}, {'type': 'v1', 'addr': '172.21.15.123:6817', 'nonce': 43109}]}, 'public_addr': '172.21.15.123:6810/43109', 'cluster_addr': '172.21.15.123:6813/43109', 'heartbeat_back_addr': '172.21.15.123:6820/43109', 'heartbeat_front_addr': '172.21.15.123:6817/43109', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': 'b45f2645-72d1-4d72-89e9-2212f5aa074c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6808', 'nonce': 74544}, {'type': 'v1', 'addr': '172.21.15.165:6809', 'nonce': 74544}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6810', 'nonce': 74544}, {'type': 'v1', 'addr': '172.21.15.165:6811', 'nonce': 74544}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6814', 'nonce': 74544}, {'type': 'v1', 'addr': '172.21.15.165:6815', 'nonce': 74544}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6812', 'nonce': 74544}, {'type': 'v1', 'addr': '172.21.15.165:6813', 'nonce': 74544}]}, 'public_addr': '172.21.15.165:6809/74544', 'cluster_addr': '172.21.15.165:6811/74544', 'heartbeat_back_addr': '172.21.15.165:6815/74544', 'heartbeat_front_addr': '172.21.15.165:6813/74544', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': '3ea410b9-a368-467b-a179-2e3aba562d63', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6800', 'nonce': 43516}, {'type': 'v1', 'addr': '172.21.15.44:6801', 'nonce': 43516}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6802', 'nonce': 43516}, {'type': 'v1', 'addr': '172.21.15.44:6803', 'nonce': 43516}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6806', 'nonce': 43516}, {'type': 'v1', 'addr': '172.21.15.44:6807', 'nonce': 43516}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6804', 'nonce': 43516}, {'type': 'v1', 'addr': '172.21.15.44:6805', 'nonce': 43516}]}, 'public_addr': '172.21.15.44:6801/43516', 'cluster_addr': '172.21.15.44:6803/43516', 'heartbeat_back_addr': '172.21.15.44:6807/43516', 'heartbeat_front_addr': '172.21.15.44:6805/43516', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': '15a5154b-e7cb-4b57-8468-dcb048d93f63', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6808', 'nonce': 74355}, {'type': 'v1', 'addr': '172.21.15.121:6809', 'nonce': 74355}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6810', 'nonce': 74355}, {'type': 'v1', 'addr': '172.21.15.121:6811', 'nonce': 74355}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6814', 'nonce': 74355}, {'type': 'v1', 'addr': '172.21.15.121:6815', 'nonce': 74355}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6812', 'nonce': 74355}, {'type': 'v1', 'addr': '172.21.15.121:6813', 'nonce': 74355}]}, 'public_addr': '172.21.15.121:6809/74355', 'cluster_addr': '172.21.15.121:6811/74355', 'heartbeat_back_addr': '172.21.15.121:6815/74355', 'heartbeat_front_addr': '172.21.15.121:6813/74355', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '23b53da2-f848-4e76-9632-4d9effd5b430', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6824', 'nonce': 43110}, {'type': 'v1', 'addr': '172.21.15.123:6825', 'nonce': 43110}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6826', 'nonce': 43110}, {'type': 'v1', 'addr': '172.21.15.123:6827', 'nonce': 43110}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6830', 'nonce': 43110}, {'type': 'v1', 'addr': '172.21.15.123:6831', 'nonce': 43110}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6828', 'nonce': 43110}, {'type': 'v1', 'addr': '172.21.15.123:6829', 'nonce': 43110}]}, 'public_addr': '172.21.15.123:6825/43110', 'cluster_addr': '172.21.15.123:6827/43110', 'heartbeat_back_addr': '172.21.15.123:6831/43110', 'heartbeat_front_addr': '172.21.15.123:6829/43110', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': '546efc0f-82a3-47d2-aeec-2d849eb1faa1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6816', 'nonce': 74562}, {'type': 'v1', 'addr': '172.21.15.165:6817', 'nonce': 74562}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6818', 'nonce': 74562}, {'type': 'v1', 'addr': '172.21.15.165:6819', 'nonce': 74562}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6822', 'nonce': 74562}, {'type': 'v1', 'addr': '172.21.15.165:6823', 'nonce': 74562}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6820', 'nonce': 74562}, {'type': 'v1', 'addr': '172.21.15.165:6821', 'nonce': 74562}]}, 'public_addr': '172.21.15.165:6817/74562', 'cluster_addr': '172.21.15.165:6819/74562', 'heartbeat_back_addr': '172.21.15.165:6823/74562', 'heartbeat_front_addr': '172.21.15.165:6821/74562', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.124928+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:48:19.984157+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.008326+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:48:15.114260+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.117767+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:12.999445+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.032367+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.058776+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.113013+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:48:23.704764+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.092533+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:48:14.370592+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.094847+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.059856+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:47:13.128995+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-20T01:48:17.320410+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'shecprofile': {'c': '2', 'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'k': '4', 'm': '3', 'name': 'shecprofile', 'plugin': 'shec', 'technique': 'multiple'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 42, 'length': 1}, {'begin': 45, 'length': 2}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 46, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-20T01:48:24.260 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T01:48:24.260 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-20T01:48:24.358 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:24.371+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:24.426 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:24.426 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:24.435 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-20T01:48:24.459 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:24.474+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:24.560 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:24.573+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:24.563 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:24.563 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:24.574 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-20T01:48:24.661 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:48:24.675+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:48:24.673 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:48:24.676 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T01:48:24.686 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-04-20T01:48:23.712060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":124895776,"num_objects":73,"num_object_clones":26,"num_object_copies":501,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":1,"num_read":552,"num_read_kb":155320,"num_write":1137,"num_write_kb":161620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":24199168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1212,"ondisk_log_size":1212,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":678776,"kb_used_data":229992,"kb_used_omap":24,"kb_used_meta":448487,"kb_avail":1509270664,"statfs":{"total":1546188226560,"available":1545493159936,"internally_reserved":0,"allocated":235511808,"data_stored":322845900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25507,"internal_metadata":459250781},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10895360,"num_objects":-9,"num_object_clones":-9,"num_object_copies":-63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-9,"num_whiteouts":-3,"num_read":48,"num_read_kb":40115,"num_write":151,"num_write_kb":23008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.999284"},"pg_stats":[{"pgid":"3.f","version":"77'57","reported_seq":218,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-20T01:48:19.876179+0000","last_change":"2024-04-20T01:48:17.711695+0000","last_active":"2024-04-20T01:48:19.876179+0000","last_peered":"2024-04-20T01:48:19.876179+0000","last_clean":"2024-04-20T01:48:19.876179+0000","last_became_active":"2024-04-20T01:47:32.867228+0000","last_became_peered":"2024-04-20T01:47:32.867228+0000","last_unstale":"2024-04-20T01:48:19.876179+0000","last_undegraded":"2024-04-20T01:48:19.876179+0000","last_fullsized":"2024-04-20T01:48:19.876179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.328818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039342199999999999,"stat_sum":{"num_bytes":8863744,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":16058,"num_write":69,"num_write_kb":9456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,15,13,8,11,0],"acting":[2,3,15,13,8,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"2"},{"start":"25","length":"2"},{"start":"2a","length":"1"}]},{"pgid":"3.e","version":"77'86","reported_seq":253,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-20T01:48:19.721549+0000","last_change":"2024-04-20T01:48:17.720451+0000","last_active":"2024-04-20T01:48:19.721549+0000","last_peered":"2024-04-20T01:48:19.721549+0000","last_clean":"2024-04-20T01:48:19.721549+0000","last_became_active":"2024-04-20T01:47:32.867409+0000","last_became_peered":"2024-04-20T01:47:32.867409+0000","last_unstale":"2024-04-20T01:48:19.721549+0000","last_undegraded":"2024-04-20T01:48:19.721549+0000","last_fullsized":"2024-04-20T01:48:19.721549+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:42.581217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043712900000000002,"stat_sum":{"num_bytes":9338880,"num_objects":7,"num_object_clones":3,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":38,"num_read_kb":4092,"num_write":38,"num_write_kb":11808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,1,10,8,3,12],"acting":[11,9,1,10,8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"2"},{"start":"25","length":"2"},{"start":"2a","length":"1"}]},{"pgid":"3.d","version":"41'12","reported_seq":164,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-20T01:48:19.718956+0000","last_change":"2024-04-20T01:48:17.714626+0000","last_active":"2024-04-20T01:48:19.718956+0000","last_peered":"2024-04-20T01:48:19.718956+0000","last_clean":"2024-04-20T01:48:19.718956+0000","last_became_active":"2024-04-20T01:47:32.867954+0000","last_became_peered":"2024-04-20T01:47:32.867954+0000","last_unstale":"2024-04-20T01:48:19.718956+0000","last_undegraded":"2024-04-20T01:48:19.718956+0000","last_fullsized":"2024-04-20T01:48:19.718956+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.083554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051446,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,11,15,7,8,4],"acting":[6,2,11,15,7,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"2"},{"start":"25","length":"2"},{"start":"2a","length":"1"}]},{"pgid":"3.c","version":"77'109","reported_seq":263,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-20T01:48:20.217657+0000","last_change":"2024-04-20T01:48:17.732693+0000","last_active":"2024-04-20T01:48:20.217657+0000","last_peered":"2024-04-20T01:48:20.217657+0000","last_clean":"2024-04-20T01:48:20.217657+0000","last_became_active":"2024-04-20T01:47:32.867926+0000","last_became_peered":"2024-04-20T01:47:32.867926+0000","last_unstale":"2024-04-20T01:48:20.217657+0000","last_undegraded":"2024-04-20T01:48:20.217657+0000","last_fullsized":"2024-04-20T01:48:20.217657+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:54.374528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01098798,"stat_sum":{"num_bytes":18677760,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":23,"num_read_kb":8938,"num_write":80,"num_write_kb":22768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,8,6,3,7,12,5],"acting":[14,8,6,3,7,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"2"},{"start":"25","length":"2"},{"start":"2a","length":"1"}]},{"pgid":"3.b","version":"77'158","reported_seq":331,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-20T01:48:19.721958+0000","last_change":"2024-04-20T01:48:17.722994+0000","last_active":"2024-04-20T01:48:19.721958+0000","last_peered":"2024-04-20T01:48:19.721958+0000","last_clean":"2024-04-20T01:48:19.721958+0000","last_became_active":"2024-04-20T01:47:32.867309+0000","last_became_peered":"2024-04-20T01:47:32.867309+0000","last_unstale":"2024-04-20T01:48:19.721958+0000","last_undegraded":"2024-04-20T01:48:19.721958+0000","last_fullsized":"2024-04-20T01:48:19.721958+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:01.227735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057379520000000002,"stat_sum":{"num_bytes":24674304,"num_objects":10,"num_object_clones":4,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":60,"num_read_kb":37515,"num_write":183,"num_write_kb":26320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,11,10,9],"acting":[4,0,1,12,11,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"2"},{"start":"25","length":"2"},{"start":"2a","length":"1"}]},{"pgid":"3.a","version":"66'20","reported_seq":175,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-20T01:48:19.719034+0000","last_change":"2024-04-20T01:48:17.714621+0000","last_active":"2024-04-20T01:48:19.719034+0000","last_peered":"2024-04-20T01:48:19.719034+0000","last_clean":"2024-04-20T01:48:19.719034+0000","last_became_active":"2024-04-20T01:47:32.868002+0000","last_became_peered":"2024-04-20T01:47:32.868002+0000","last_unstale":"2024-04-20T01:48:19.719034+0000","last_undegraded":"2024-04-20T01:48:19.719034+0000","last_fullsized":"2024-04-20T01:48:19.719034+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:35.716198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063538600000000005,"stat_sum":{"num_bytes":1720320,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":1972,"num_write":10,"num_write_kb":3376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,1,14,3,15,2],"acting":[6,10,1,14,3,15,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"2"},{"start":"25","length":"2"},{"start":"2a","length":"1"}]},{"pgid":"3.9","version":"75'27","reported_seq":185,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-20T01:48:19.718950+0000","last_change":"2024-04-20T01:48:17.715320+0000","last_active":"2024-04-20T01:48:19.718950+0000","last_peered":"2024-04-20T01:48:19.718950+0000","last_clean":"2024-04-20T01:48:19.718950+0000","last_became_active":"2024-04-20T01:47:32.866706+0000","last_became_peered":"2024-04-20T01:47:32.866706+0000","last_unstale":"2024-04-20T01:48:19.718950+0000","last_undegraded":"2024-04-20T01:48:19.718950+0000","last_fullsized":"2024-04-20T01:48:19.718950+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:42.776206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063910400000000004,"stat_sum":{"num_bytes":638976,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":3124,"num_write":47,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0,11,14,8,3],"acting":[7,1,0,11,14,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"2"},{"start":"25","length":"2"},{"start":"2a","length":"1"}]},{"pgid":"3.8","version":"77'91","reported_seq":256,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-20T01:48:20.394099+0000","last_change":"2024-04-20T01:48:17.714493+0000","last_active":"2024-04-20T01:48:20.394099+0000","last_peered":"2024-04-20T01:48:20.394099+0000","last_clean":"2024-04-20T01:48:20.394099+0000","last_became_active":"2024-04-20T01:47:32.718069+0000","last_became_peered":"2024-04-20T01:47:32.718069+0000","last_unstale":"2024-04-20T01:48:20.394099+0000","last_undegraded":"2024-04-20T01:48:20.394099+0000","last_fullsized":"2024-04-20T01:48:20.394099+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:58.489885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000486806,"stat_sum":{"num_bytes":7454720,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":20031,"num_write":43,"num_write_kb":12976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,11,3,14,2,7],"acting":[5,0,11,3,14,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"2"},{"start":"25","length":"2"},{"start":"2a","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-20T01:48:19.721991+0000","last_change":"2024-04-20T01:47:20.071988+0000","last_active":"2024-04-20T01:48:19.721991+0000","last_peered":"2024-04-20T01:48:19.721991+0000","last_clean":"2024-04-20T01:48:19.721991+0000","last_became_active":"2024-04-20T01:47:20.071615+0000","last_became_peered":"2024-04-20T01:47:20.071615+0000","last_unstale":"2024-04-20T01:48:19.721991+0000","last_undegraded":"2024-04-20T01:48:19.721991+0000","last_fullsized":"2024-04-20T01:48:19.721991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:40.106777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"77'33","reported_seq":191,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-20T01:48:20.731755+0000","last_change":"2024-04-20T01:48:17.725603+0000","last_active":"2024-04-20T01:48:20.731755+0000","last_peered":"2024-04-20T01:48:20.731755+0000","last_clean":"2024-04-20T01:48:20.731755+0000","last_became_active":"2024-04-20T01:47:32.867180+0000","last_became_peered":"2024-04-20T01:47:32.867180+0000","last_unstale":"2024-04-20T01:48:20.731755+0000","last_undegraded":"2024-04-20T01:48:20.731755+0000","last_fullsized":"2024-04-20T01:48:20.731755+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:43.358783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063442100000000003,"stat_sum":{"num_bytes":4440064,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":7302,"num_write":18,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,4,14,12,0,11],"acting":[10,8,4,14,12,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"2"},{"start":"25","length":"2"},{"start":"2a","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-20T01:48:19.876052+0000","last_change":"2024-04-20T01:47:20.075613+0000","last_active":"2024-04-20T01:48:19.876052+0000","last_peered":"2024-04-20T01:48:19.876052+0000","last_clean":"2024-04-20T01:48:19.876052+0000","last_became_active":"2024-04-20T01:47:20.075314+0000","last_became_peered":"2024-04-20T01:47:20.075314+0000","last_unstale":"2024-04-20T01:48:19.876052+0000","last_undegraded":"2024-04-20T01:48:19.876052+0000","last_fullsized":"2024-04-20T01:48:19.876052+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:35.956173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,12],"acting":[2,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"80'39","reported_seq":203,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-20T01:48:21.735845+0000","last_change":"2024-04-20T01:48:21.736606+0000","last_active":"2024-04-20T01:48:21.735845+0000","last_peered":"2024-04-20T01:48:21.735845+0000","last_clean":"2024-04-20T01:48:21.735845+0000","last_became_active":"2024-04-20T01:47:32.719136+0000","last_became_peered":"2024-04-20T01:47:32.719136+0000","last_unstale":"2024-04-20T01:48:21.735845+0000","last_undegraded":"2024-04-20T01:48:21.735845+0000","last_fullsized":"2024-04-20T01:48:21.735845+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:46.940575+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010085729,"stat_sum":{"num_bytes":2244608,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":3173,"num_write":59,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,4,14,11,5,10],"acting":[13,2,4,14,11,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"2"},{"start":"25","length":"2"},{"start":"2a","length":"1"},{"start":"2d","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":62,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-20T01:48:20.722949+0000","last_change":"2024-04-20T01:47:20.039221+0000","last_active":"2024-04-20T01:48:20.722949+0000","last_peered":"2024-04-20T01:48:20.722949+0000","last_clean":"2024-04-20T01:48:20.722949+0000","last_became_active":"2024-04-20T01:47:20.038972+0000","last_became_peered":"2024-04-20T01:47:20.038972+0000","last_unstale":"2024-04-20T01:48:20.722949+0000","last_undegraded":"2024-04-20T01:48:20.722949+0000","last_fullsized":"2024-04-20T01:48:20.722949+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:24.167832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"77'49","reported_seq":212,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-20T01:48:19.720268+0000","last_change":"2024-04-20T01:48:17.716164+0000","last_active":"2024-04-20T01:48:19.720268+0000","last_peered":"2024-04-20T01:48:19.720268+0000","last_clean":"2024-04-20T01:48:19.720268+0000","last_became_active":"2024-04-20T01:47:32.868365+0000","last_became_peered":"2024-04-20T01:47:32.868365+0000","last_unstale":"2024-04-20T01:48:19.720268+0000","last_undegraded":"2024-04-20T01:48:19.720268+0000","last_fullsized":"2024-04-20T01:48:19.720268+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:41.198793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052066309999999998,"stat_sum":{"num_bytes":6799360,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":26,"num_read_kb":4186,"num_write":43,"num_write_kb":7120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,1,5,15,4],"acting":[3,7,6,1,5,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"2"},{"start":"25","length":"2"},{"start":"2a","length":"1"}]},{"pgid":"2.3","version":"0'0","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-20T01:48:20.394105+0000","last_change":"2024-04-20T01:47:20.039014+0000","last_active":"2024-04-20T01:48:20.394105+0000","last_peered":"2024-04-20T01:48:20.394105+0000","last_clean":"2024-04-20T01:48:20.394105+0000","last_became_active":"2024-04-20T01:47:20.038764+0000","last_became_peered":"2024-04-20T01:47:20.038764+0000","last_unstale":"2024-04-20T01:48:20.394105+0000","last_undegraded":"2024-04-20T01:48:20.394105+0000","last_fullsized":"2024-04-20T01:48:20.394105+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:26.112165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"25'157","reported_seq":260,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-20T01:48:20.722913+0000","last_change":"2024-04-20T01:47:17.303029+0000","last_active":"2024-04-20T01:48:20.722913+0000","last_peered":"2024-04-20T01:48:20.722913+0000","last_clean":"2024-04-20T01:48:20.722913+0000","last_became_active":"2024-04-20T01:47:17.302105+0000","last_became_peered":"2024-04-20T01:47:17.302105+0000","last_unstale":"2024-04-20T01:48:20.722913+0000","last_undegraded":"2024-04-20T01:48:20.722913+0000","last_fullsized":"2024-04-20T01:48:20.722913+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_clean_scrub_stamp":"2024-04-20T01:47:15.986754+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:30.481506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,15],"acting":[13,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.2","version":"80'136","reported_seq":312,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-20T01:48:21.735503+0000","last_change":"2024-04-20T01:48:21.737158+0000","last_active":"2024-04-20T01:48:21.735503+0000","last_peered":"2024-04-20T01:48:21.735503+0000","last_clean":"2024-04-20T01:48:21.735503+0000","last_became_active":"2024-04-20T01:47:32.866509+0000","last_became_peered":"2024-04-20T01:47:32.866509+0000","last_unstale":"2024-04-20T01:48:21.735503+0000","last_undegraded":"2024-04-20T01:48:21.735503+0000","last_fullsized":"2024-04-20T01:48:21.735503+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.395078+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010652386,"stat_sum":{"num_bytes":18546688,"num_objects":7,"num_object_clones":2,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":60,"num_read_kb":21576,"num_write":103,"num_write_kb":22992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,6,10,14,1,7],"acting":[13,3,6,10,14,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"2"},{"start":"25","length":"2"},{"start":"2a","length":"1"},{"start":"2d","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-20T01:48:20.723784+0000","last_change":"2024-04-20T01:48:19.877116+0000","last_active":"2024-04-20T01:48:20.723784+0000","last_peered":"2024-04-20T01:48:20.723784+0000","last_clean":"2024-04-20T01:48:20.723784+0000","last_became_active":"2024-04-20T01:48:19.876908+0000","last_became_peered":"2024-04-20T01:48:19.876908+0000","last_unstale":"2024-04-20T01:48:20.723784+0000","last_undegraded":"2024-04-20T01:48:20.723784+0000","last_fullsized":"2024-04-20T01:48:20.723784+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:30.233309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"77'45","reported_seq":199,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-20T01:48:19.721913+0000","last_change":"2024-04-20T01:48:17.724663+0000","last_active":"2024-04-20T01:48:19.721913+0000","last_peered":"2024-04-20T01:48:19.721913+0000","last_clean":"2024-04-20T01:48:19.721913+0000","last_became_active":"2024-04-20T01:47:32.722189+0000","last_became_peered":"2024-04-20T01:47:32.722189+0000","last_unstale":"2024-04-20T01:48:19.721913+0000","last_undegraded":"2024-04-20T01:48:19.721913+0000","last_fullsized":"2024-04-20T01:48:19.721913+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:43.443403+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0094285170000000008,"stat_sum":{"num_bytes":2588672,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":12,"num_read_kb":1012,"num_write":36,"num_write_kb":3424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,14,9,5,6,3,13],"acting":[4,14,9,5,6,3,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"2"},{"start":"25","length":"2"},{"start":"2a","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-20T01:48:19.718838+0000","last_change":"2024-04-20T01:47:20.038409+0000","last_active":"2024-04-20T01:48:19.718838+0000","last_peered":"2024-04-20T01:48:19.718838+0000","last_clean":"2024-04-20T01:48:19.718838+0000","last_became_active":"2024-04-20T01:47:20.038107+0000","last_became_peered":"2024-04-20T01:47:20.038107+0000","last_unstale":"2024-04-20T01:48:19.718838+0000","last_undegraded":"2024-04-20T01:48:19.718838+0000","last_fullsized":"2024-04-20T01:48:19.718838+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:44.125577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"79'145","reported_seq":512,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-20T01:48:20.743736+0000","last_change":"2024-04-20T01:48:17.711906+0000","last_active":"2024-04-20T01:48:20.743736+0000","last_peered":"2024-04-20T01:48:20.743736+0000","last_clean":"2024-04-20T01:48:20.743736+0000","last_became_active":"2024-04-20T01:47:59.783367+0000","last_became_peered":"2024-04-20T01:47:59.783367+0000","last_unstale":"2024-04-20T01:48:20.743736+0000","last_undegraded":"2024-04-20T01:48:20.743736+0000","last_fullsized":"2024-04-20T01:48:20.743736+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:51.089820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070150099999999999,"stat_sum":{"num_bytes":13975552,"num_objects":9,"num_object_clones":3,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":16501,"num_write":102,"num_write_kb":20272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":24199168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,15,4,1,0,3,13],"acting":[8,15,4,1,0,3,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"2"},{"start":"25","length":"2"},{"start":"2a","length":"1"}]},{"pgid":"3.7","version":"79'28","reported_seq":184,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-20T01:48:20.735430+0000","last_change":"2024-04-20T01:48:17.716883+0000","last_active":"2024-04-20T01:48:20.735430+0000","last_peered":"2024-04-20T01:48:20.735430+0000","last_clean":"2024-04-20T01:48:20.735430+0000","last_became_active":"2024-04-20T01:47:32.722002+0000","last_became_peered":"2024-04-20T01:47:32.722002+0000","last_unstale":"2024-04-20T01:48:20.735430+0000","last_undegraded":"2024-04-20T01:48:20.735430+0000","last_fullsized":"2024-04-20T01:48:20.735430+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:43.990113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015405359999999999,"stat_sum":{"num_bytes":3702784,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":15,"num_read_kb":7460,"num_write":45,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,0,8,5,14],"acting":[4,11,9,0,8,5,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"2"},{"start":"25","length":"2"},{"start":"2a","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-20T01:48:19.713126+0000","last_change":"2024-04-20T01:47:20.110600+0000","last_active":"2024-04-20T01:48:19.713126+0000","last_peered":"2024-04-20T01:48:19.713126+0000","last_clean":"2024-04-20T01:48:19.713126+0000","last_became_active":"2024-04-20T01:47:20.110260+0000","last_became_peered":"2024-04-20T01:47:20.110260+0000","last_unstale":"2024-04-20T01:48:19.713126+0000","last_undegraded":"2024-04-20T01:48:19.713126+0000","last_fullsized":"2024-04-20T01:48:19.713126+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:47.718290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-20T01:48:19.719062+0000","last_change":"2024-04-20T01:47:20.036634+0000","last_active":"2024-04-20T01:48:19.719062+0000","last_peered":"2024-04-20T01:48:19.719062+0000","last_clean":"2024-04-20T01:48:19.719062+0000","last_became_active":"2024-04-20T01:47:20.036328+0000","last_became_peered":"2024-04-20T01:47:20.036328+0000","last_unstale":"2024-04-20T01:48:19.719062+0000","last_undegraded":"2024-04-20T01:48:19.719062+0000","last_fullsized":"2024-04-20T01:48:19.719062+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.385250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,13],"acting":[6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"73'20","reported_seq":180,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-20T01:48:19.721481+0000","last_change":"2024-04-20T01:48:17.726144+0000","last_active":"2024-04-20T01:48:19.721481+0000","last_peered":"2024-04-20T01:48:19.721481+0000","last_clean":"2024-04-20T01:48:19.721481+0000","last_became_active":"2024-04-20T01:47:32.866834+0000","last_became_peered":"2024-04-20T01:47:32.866834+0000","last_unstale":"2024-04-20T01:48:19.721481+0000","last_undegraded":"2024-04-20T01:48:19.721481+0000","last_fullsized":"2024-04-20T01:48:19.721481+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:54.424527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056270500000000004,"stat_sum":{"num_bytes":638976,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2260,"num_write":52,"num_write_kb":2720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,3,6,2,9],"acting":[0,13,1,3,6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"1"},{"start":"1f","length":"2"},{"start":"25","length":"2"},{"start":"2a","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":124305408,"num_objects":71,"num_object_clones":26,"num_object_copies":497,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":1,"num_read":416,"num_read_kb":155202,"num_write":934,"num_write_kb":158160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":24199168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219226112,"data_stored":315379712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1055,"ondisk_log_size":1055,"up":112,"acting":112,"num_store_stats":16},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280528,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44496,"kb_used_data":13372,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94327344,"statfs":{"total":96636764160,"available":96591200256,"internally_reserved":0,"allocated":13692928,"data_stored":18363138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.41699999999999998}]},{"osd":2,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.496},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.63600000000000001}]},{"osd":3,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.441},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.47099999999999997}]},{"osd":4,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":9.0129999999999999,"5min":9.0129999999999999,"15min":9.0129999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.66700000000000004}]},{"osd":5,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.51700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.57899999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":12,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":9.1389999999999993,"5min":9.1389999999999993,"15min":9.1389999999999993},"last":0.81499999999999995}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.68200000000000005}]},{"osd":15,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.8719999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.70699999999999996}]}]},{"osd":15,"up_from":21,"seq":90194313232,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35776,"kb_used_data":8752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336064,"statfs":{"total":96636764160,"available":96600129536,"internally_reserved":0,"allocated":8962048,"data_stored":10538242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":4.5259999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":2,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.433},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.35399999999999998}]},{"osd":3,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.24299999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.443}]},{"osd":5,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.33200000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.42199999999999999}]},{"osd":7,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.377}]},{"osd":8,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":4.5979999999999999,"5min":4.5979999999999999,"15min":4.5979999999999999},"last":0.504}]},{"osd":9,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.49399999999999999}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.45600000000000002}]}]},{"osd":14,"up_from":21,"seq":90194313233,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43092,"kb_used_data":16064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328748,"statfs":{"total":96636764160,"available":96592637952,"internally_reserved":0,"allocated":16449536,"data_stored":22819042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,4,5,6,7,8,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.46999999999999997}]},{"osd":4,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.499},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.50800000000000001}]},{"osd":5,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.57099999999999995}]},{"osd":6,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.41799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":9,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.64700000000000002}]},{"osd":12,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.67700000000000005}]},{"osd":13,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.56200000000000006}]},{"osd":15,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.58099999999999996}]}]},{"osd":12,"up_from":21,"seq":90194313232,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42068,"kb_used_data":15044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94329772,"statfs":{"total":96636764160,"available":96593686528,"internally_reserved":0,"allocated":15405056,"data_stored":22317794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.52700000000000002}]},{"osd":2,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.499}]},{"osd":3,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.48699999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.46999999999999997}]},{"osd":5,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.61599999999999999}]},{"osd":6,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.76500000000000001}]},{"osd":11,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":6.6779999999999999,"5min":6.6779999999999999,"15min":6.6779999999999999},"last":0.56000000000000005}]},{"osd":13,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.90700000000000003}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.878}]},{"osd":15,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.85799999999999998}]}]},{"osd":11,"up_from":21,"seq":90194313231,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47480,"kb_used_data":16360,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94324360,"statfs":{"total":96636764160,"available":96588144640,"internally_reserved":0,"allocated":16752640,"data_stored":23255778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,3,4,8,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.624},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.41599999999999998}]},{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.36699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":3,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.27000000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":8.5969999999999995,"5min":8.5969999999999995,"15min":8.5969999999999995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.52100000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":9,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.34899999999999998}]},{"osd":10,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.64800000000000002}]},{"osd":12,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":0.47399999999999998},{"interface":"front","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":12.385,"5min":12.385,"15min":12.385},"last":0.51000000000000001}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68300000000000005}]},{"osd":15,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.439}]}]},{"osd":10,"up_from":21,"seq":90194313233,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47404,"kb_used_data":16284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94324436,"statfs":{"total":96636764160,"available":96588222464,"internally_reserved":0,"allocated":16674816,"data_stored":24533730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,4,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":7,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.56899999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":9,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.496},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.52000000000000002}]},{"osd":11,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":6.0410000000000004,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":0.47099999999999997}]},{"osd":12,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.331,"15min":1.331},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":9.7509999999999994,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.41299999999999998},{"interface":"front","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":9.7759999999999998,"5min":9.7759999999999998,"15min":9.7759999999999998},"last":0.42199999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.372},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.33400000000000002}]},{"osd":15,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.48499999999999999}]}]},{"osd":9,"up_from":21,"seq":90194313233,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38136,"kb_used_data":11108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333704,"statfs":{"total":96636764160,"available":96597712896,"internally_reserved":0,"allocated":11374592,"data_stored":17031997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,4,6,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.47099999999999997}]},{"osd":4,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.497}]},{"osd":6,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.60199999999999998}]},{"osd":7,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.77200000000000002}]},{"osd":8,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.3319999999999999,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.751},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.58499999999999996}]},{"osd":10,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.54200000000000004}]},{"osd":11,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.84599999999999997}]},{"osd":12,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.621}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.79700000000000004}]},{"osd":15,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":1.002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.88800000000000001}]}]},{"osd":8,"up_from":21,"seq":90194313232,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42140,"kb_used_data":15116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94329700,"statfs":{"total":96636764160,"available":96593612800,"internally_reserved":0,"allocated":15478784,"data_stored":20847330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,6,7,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":1,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.44800000000000001}]},{"osd":3,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.62},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":0.48699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9630000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":0.54100000000000004}]},{"osd":7,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.752},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.64800000000000002}]},{"osd":9,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":1.139}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":11,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.5110000000000001,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":1.0069999999999999}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":15,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.89700000000000002}]}]},{"osd":7,"up_from":21,"seq":90194313233,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40844,"kb_used_data":13820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94330996,"statfs":{"total":96636764160,"available":96594939904,"internally_reserved":0,"allocated":14151680,"data_stored":19004130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,3,4,6,8,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":1,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.55800000000000005}]},{"osd":8,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":5.7679999999999998,"5min":5.7679999999999998,"15min":5.7679999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":9.952,"5min":9.952,"15min":9.952},"last":0.46300000000000002}]},{"osd":9,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.623}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":11,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.58499999999999996}]},{"osd":12,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":4.2720000000000002,"5min":4.2720000000000002,"15min":4.2720000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.70299999999999996}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63600000000000001}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":6,"up_from":21,"seq":90194313231,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40076,"kb_used_data":13052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331764,"statfs":{"total":96636764160,"available":96595726336,"internally_reserved":0,"allocated":13365248,"data_stored":18680546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":5,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.64300000000000002}]},{"osd":7,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.621}]},{"osd":8,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.72099999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":10,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.69099999999999995}]},{"osd":11,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.67000000000000004}]},{"osd":12,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":2.1880000000000002}]},{"osd":13,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.81499999999999995}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.79200000000000004}]},{"osd":15,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.86199999999999999}]}]},{"osd":1,"up_from":21,"seq":90194313232,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46236,"kb_used_data":19212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94325604,"statfs":{"total":96636764160,"available":96589418496,"internally_reserved":0,"allocated":19673088,"data_stored":27974370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.439},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.53000000000000003}]},{"osd":2,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.67100000000000004}]},{"osd":3,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.505},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.54400000000000004}]},{"osd":4,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.372},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.41499999999999998}]},{"osd":5,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.45500000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.70899999999999996}]},{"osd":7,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":1.3740000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":1.4139999999999999}]},{"osd":8,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.59699999999999998}]},{"osd":9,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.627},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.55900000000000005}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.752}]}]},{"osd":0,"up_from":21,"seq":90194313233,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43296,"kb_used_data":16272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328544,"statfs":{"total":96636764160,"available":96592429056,"internally_reserved":0,"allocated":16662528,"data_stored":22603928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.73499999999999999}]},{"osd":2,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":8.0619999999999994,"5min":8.0619999999999994,"15min":8.0619999999999994},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.48399999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":3.363,"5min":3.363,"15min":3.363},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":7.9710000000000001,"5min":7.9710000000000001,"15min":7.9710000000000001},"last":0.66600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":3.5339999999999998,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":0.45800000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":8.0239999999999991,"5min":8.0239999999999991,"15min":8.0239999999999991},"last":0.621},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.70999999999999996}]},{"osd":7,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":3.4020000000000001,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":7.8630000000000004,"5min":7.8630000000000004,"15min":7.8630000000000004},"last":0.80000000000000004}]},{"osd":8,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":8.1189999999999998,"5min":8.1189999999999998,"15min":8.1189999999999998},"last":0.77000000000000002}]},{"osd":9,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":3.5699999999999998,"5min":3.5699999999999998,"15min":3.5699999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.044}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":15,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":3.4369999999999998,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":0.69899999999999995}]}]},{"osd":2,"up_from":21,"seq":90194313231,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33532,"kb_used_data":6508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338308,"statfs":{"total":96636764160,"available":96602427392,"internally_reserved":0,"allocated":6664192,"data_stored":7588578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":1,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.58699999999999997}]},{"osd":3,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.55600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":7,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60899999999999999}]},{"osd":8,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.71599999999999997}]},{"osd":9,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.80700000000000005}]},{"osd":11,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.72999999999999998}]},{"osd":12,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":3.1190000000000002,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":3.492,"5min":3.492,"15min":3.492},"last":0.83999999999999997}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":3,"up_from":21,"seq":90194313234,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53484,"kb_used_data":22364,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94318356,"statfs":{"total":96636764160,"available":96581996544,"internally_reserved":0,"allocated":22900736,"data_stored":31025890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,4,5,6,7,8,9,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":2,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.44900000000000001}]},{"osd":4,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":9.3650000000000002,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":0.53600000000000003}]},{"osd":5,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.57499999999999996}]},{"osd":6,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.41899999999999998}]},{"osd":7,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.376},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.314}]},{"osd":8,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":9.4700000000000006,"5min":9.4700000000000006,"15min":9.4700000000000006},"last":0.627}]},{"osd":9,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.61699999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":12,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":10.959,"5min":10.959,"15min":10.959},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.51900000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]}]},{"osd":4,"up_from":21,"seq":90194313233,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42148,"kb_used_data":15124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94329692,"statfs":{"total":96636764160,"available":96593604608,"internally_reserved":0,"allocated":15486976,"data_stored":21068514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,5,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":1,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":10.507999999999999,"5min":10.507999999999999,"15min":10.507999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":11.019,"5min":11.019,"15min":11.019},"last":0.749}]},{"osd":3,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":4.9180000000000001,"5min":4.9180000000000001,"15min":4.9180000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":10.619,"5min":10.619,"15min":10.619},"last":0.502}]},{"osd":5,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":10.895,"5min":10.895,"15min":10.895},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":4.9589999999999996,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":0.73099999999999998}]},{"osd":6,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":11.051,"5min":11.051,"15min":11.051},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":10.589,"5min":10.589,"15min":10.589},"last":0.59499999999999997}]},{"osd":7,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":10.957000000000001,"5min":10.957000000000001,"15min":10.957000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":10.656000000000001,"5min":10.656000000000001,"15min":10.656000000000001},"last":0.65200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":9,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.385,"15min":1.385},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":10.929,"5min":10.929,"15min":10.929},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":10.555999999999999,"5min":10.555999999999999,"15min":10.555999999999999},"last":0.79800000000000004}]},{"osd":10,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":5.0229999999999997,"5min":5.0229999999999997,"15min":5.0229999999999997},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":10.69,"5min":10.69,"15min":10.69},"last":0.76000000000000001}]},{"osd":11,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":10.987,"5min":10.987,"15min":10.987},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":4.9900000000000002,"5min":4.9900000000000002,"15min":4.9900000000000002},"last":0.82099999999999995}]},{"osd":12,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":10.720000000000001,"5min":10.720000000000001,"15min":10.720000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.772,"5min":1.772,"15min":1.772},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":16.617000000000001,"5min":16.617000000000001,"15min":16.617000000000001},"last":0.88100000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]}]},{"osd":5,"up_from":21,"seq":90194313232,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38568,"kb_used_data":11540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333272,"statfs":{"total":96636764160,"available":96597270528,"internally_reserved":0,"allocated":11816960,"data_stored":15192893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":10.423999999999999,"5min":10.423999999999999,"15min":10.423999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":10.465,"5min":10.465,"15min":10.465},"last":0.48699999999999999}]},{"osd":2,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.505},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.51500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":4,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":8.5269999999999992,"5min":8.5269999999999992,"15min":8.5269999999999992},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.79,"5min":2.79,"15min":2.79},"last":0.46000000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.54000000000000004}]},{"osd":7,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.55600000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":10,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63300000000000001}]},{"osd":11,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.66400000000000003}]},{"osd":12,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":10.616,"5min":10.616,"15min":10.616},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":10.629,"5min":10.629,"15min":10.629},"last":0.64400000000000002}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.626}]}]}],"pool_statfs":[{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16097280,"data_stored":22999040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19083264,"data_stored":27869184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5541888,"data_stored":6950912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":22310912,"data_stored":30920704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":14917632,"data_stored":21463040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10678272,"data_stored":14544896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12242944,"data_stored":18042880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":13029376,"data_stored":18366464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":14909440,"data_stored":21241856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":10235904,"data_stored":16384000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":15552512,"data_stored":23896064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":15650816,"data_stored":23117824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":14282752,"data_stored":21680128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":11538432,"data_stored":15826944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":15314944,"data_stored":22171648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":7839744,"data_stored":9904128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T01:48:24.691 INFO:tasks.thrashosds.thrasher:Setting 3.4 to [7, 13, 12, 1, 3, 6, 5, 10, 11, 14, 2, 0, 15, 4] 2024-04-20T01:48:24.691 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.4', '7', '13', '12', '1', '3', '6', '5', '10', '11', '14', '2', '0', '15', '4'] 2024-04-20T01:48:24.691 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.4 7 13 12 1 3 6 5 10 11 14 2 0 15 4 2024-04-20T01:48:24.701 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:24.702 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:24.712 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-20T01:48:24.722 INFO:tasks.rados.rados.0.smithi165.stdout:353: done (0 left) 2024-04-20T01:48:24.723 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 43 current snap is 27 2024-04-20T01:48:24.723 INFO:tasks.rados.rados.0.smithi165.stdout:354: seq_num 149 ranges {2342912=98304} 2024-04-20T01:48:24.735 INFO:tasks.rados.rados.0.smithi165.stdout:354: writing smithi16579500-43 from 2342912 to 2441216 tid 1 2024-04-20T01:48:24.735 INFO:tasks.rados.rados.0.smithi165.stdout:355: setattr oid 19 current snap is 27 2024-04-20T01:48:24.738 INFO:tasks.rados.rados.0.smithi165.stdout:356: snap_remove snap 24 2024-04-20T01:48:24.796 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:24.810+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:24.865 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:24.878+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:24.880 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:24.880 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:24.891 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-20T01:48:24.966 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:24.980+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:25.062 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:25.062 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:25.066 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:25.080+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:25.069 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-20T01:48:25.167 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:25.181+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:25.211 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:25.212 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:25.221 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-20T01:48:25.268 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:25.281+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:25.349 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:25.349 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:25.359 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-20T01:48:25.368 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:25.382+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:25.470 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:25.484+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:25.528 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:25.528 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:25.537 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-20T01:48:25.571 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:25.585+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:25.674 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:25.689+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:25.707 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:25.707 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:25.717 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-20T01:48:25.723 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-20T01:48:25.737 INFO:tasks.rados.rados.0.smithi165.stdout:356: done (2 left) 2024-04-20T01:48:25.737 INFO:tasks.rados.rados.0.smithi165.stdout:357: read oid 25 snap -1 2024-04-20T01:48:25.737 INFO:tasks.rados.rados.0.smithi165.stdout:357: expect deleted 2024-04-20T01:48:25.737 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 41 current snap is 27 2024-04-20T01:48:25.737 INFO:tasks.rados.rados.0.smithi165.stdout:358: seq_num 150 ranges {1949696=114688} 2024-04-20T01:48:25.741 INFO:teuthology.orchestra.run.smithi044.stderr:set 3.4 pg_upmap_items mapping to [7->13,12->1,3->6,5->10,11->14,2->0,15->4] 2024-04-20T01:48:25.748 INFO:tasks.rados.rados.0.smithi165.stdout:358: writing smithi16579500-41 from 1949696 to 2064384 tid 1 2024-04-20T01:48:25.749 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 37 current snap is 27 2024-04-20T01:48:25.749 INFO:tasks.rados.rados.0.smithi165.stdout:359: seq_num 151 ranges {606208=131072} 2024-04-20T01:48:25.753 INFO:tasks.rados.rados.0.smithi165.stdout:359: writing smithi16579500-37 from 606208 to 737280 tid 1 2024-04-20T01:48:25.753 INFO:tasks.rados.rados.0.smithi165.stdout:360: read oid 34 snap -1 2024-04-20T01:48:25.754 INFO:tasks.rados.rados.0.smithi165.stdout:360: expect (ObjNum 131 snap 21 seq_num 131) 2024-04-20T01:48:25.754 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 46 current snap is 27 2024-04-20T01:48:25.754 INFO:tasks.rados.rados.0.smithi165.stdout:361: seq_num 152 ranges {770048=147456} 2024-04-20T01:48:25.759 INFO:tasks.rados.rados.0.smithi165.stdout:361: writing smithi16579500-46 from 770048 to 917504 tid 1 2024-04-20T01:48:25.759 INFO:tasks.rados.rados.0.smithi165.stdout:362: copy_from oid 3 from oid 5 current snap is 27 2024-04-20T01:48:25.759 INFO:tasks.rados.rados.0.smithi165.stdout:354: finishing write tid 1 to smithi16579500-43 2024-04-20T01:48:25.759 INFO:tasks.rados.rados.0.smithi165.stdout:354: finishing write tid 2 to smithi16579500-43 2024-04-20T01:48:25.760 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 19 v 157 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-20T01:48:25.760 INFO:tasks.rados.rados.0.smithi165.stdout:354: finishing write tid 3 to smithi16579500-43 2024-04-20T01:48:25.760 INFO:tasks.rados.rados.0.smithi165.stdout:354: oid 43 updating version 0 to 155 2024-04-20T01:48:25.760 INFO:tasks.rados.rados.0.smithi165.stdout:354: oid 43 updating version 155 to 156 2024-04-20T01:48:25.760 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 43 v 156 (ObjNum 149 snap 27 seq_num 149) dirty exists 2024-04-20T01:48:25.760 INFO:tasks.rados.rados.0.smithi165.stdout:354: left oid 43 (ObjNum 149 snap 27 seq_num 149) 2024-04-20T01:48:25.760 INFO:tasks.rados.rados.0.smithi165.stdout:354: done (7 left) 2024-04-20T01:48:25.760 INFO:tasks.rados.rados.0.smithi165.stdout:355: done (6 left) 2024-04-20T01:48:25.761 INFO:tasks.rados.rados.0.smithi165.stdout:363: rollback oid 1 current snap is 27 2024-04-20T01:48:25.761 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 1 to 26 2024-04-20T01:48:25.761 INFO:tasks.rados.rados.0.smithi165.stdout:357: done (6 left) 2024-04-20T01:48:25.761 INFO:tasks.rados.rados.0.smithi165.stdout:364: delete oid 16 current snap is 27 2024-04-20T01:48:25.762 INFO:tasks.rados.rados.0.smithi165.stdout:359: finishing write tid 1 to smithi16579500-37 2024-04-20T01:48:25.762 INFO:tasks.rados.rados.0.smithi165.stdout:359: finishing write tid 2 to smithi16579500-37 2024-04-20T01:48:25.765 INFO:tasks.rados.rados.0.smithi165.stdout:359: finishing write tid 3 to smithi16579500-37 2024-04-20T01:48:25.765 INFO:tasks.rados.rados.0.smithi165.stdout:359: oid 37 updating version 0 to 90 2024-04-20T01:48:25.765 INFO:tasks.rados.rados.0.smithi165.stdout:359: oid 37 updating version 90 to 91 2024-04-20T01:48:25.765 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 37 v 91 (ObjNum 151 snap 27 seq_num 151) dirty exists 2024-04-20T01:48:25.765 INFO:tasks.rados.rados.0.smithi165.stdout:359: left oid 37 (ObjNum 151 snap 27 seq_num 151) 2024-04-20T01:48:25.766 INFO:tasks.rados.rados.0.smithi165.stdout:359: done (6 left) 2024-04-20T01:48:25.766 INFO:tasks.rados.rados.0.smithi165.stdout:364: done (5 left) 2024-04-20T01:48:25.766 INFO:tasks.rados.rados.0.smithi165.stdout:365: snap_create 2024-04-20T01:48:25.767 INFO:tasks.rados.rados.0.smithi165.stdout:363: finishing rollback tid 0 to smithi16579500-1 2024-04-20T01:48:25.770 INFO:tasks.rados.rados.0.smithi165.stdout:363: finishing rollback tid 1 to smithi16579500-1 2024-04-20T01:48:25.772 INFO:tasks.rados.rados.0.smithi165.stdout:361: finishing write tid 1 to smithi16579500-46 2024-04-20T01:48:25.772 INFO:tasks.rados.rados.0.smithi165.stdout:361: finishing write tid 2 to smithi16579500-46 2024-04-20T01:48:25.774 INFO:tasks.rados.rados.0.smithi165.stdout:361: finishing write tid 3 to smithi16579500-46 2024-04-20T01:48:25.774 INFO:tasks.rados.rados.0.smithi165.stdout:361: oid 46 updating version 0 to 53 2024-04-20T01:48:25.774 INFO:tasks.rados.rados.0.smithi165.stdout:361: oid 46 updating version 53 to 54 2024-04-20T01:48:25.774 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 46 v 54 (ObjNum 152 snap 27 seq_num 152) dirty exists 2024-04-20T01:48:25.774 INFO:tasks.rados.rados.0.smithi165.stdout:361: left oid 46 (ObjNum 152 snap 27 seq_num 152) 2024-04-20T01:48:25.775 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:25.790+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:25.782 INFO:tasks.rados.rados.0.smithi165.stdout:362: finishing copy_from racing read to smithi16579500-3 2024-04-20T01:48:25.783 INFO:tasks.rados.rados.0.smithi165.stdout:363: finishing rollback tid 2 to smithi16579500-1 2024-04-20T01:48:25.783 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 1 v 38 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-04-20T01:48:25.788 INFO:tasks.rados.rados.0.smithi165.stdout:362: finishing copy_from to smithi16579500-3 2024-04-20T01:48:25.788 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 3 v 49 (ObjNum 142 snap 23 seq_num 142) dirty exists 2024-04-20T01:48:25.790 INFO:tasks.rados.rados.0.smithi165.stdout:360: expect (ObjNum 131 snap 21 seq_num 131) 2024-04-20T01:48:25.857 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-20T01:48:25.864 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:25.864 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:25.875 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-20T01:48:25.902 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:25.915+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:25.978 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:48:25.992+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:48:26.022 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:48:26.042 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:48:26.046 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:26.046 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:26.056 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-20T01:48:26.063 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:48:26.084 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:48:26.100 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:26.114+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:26.104 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:48:26.124 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:48:26.146 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:48:26.166 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:48:26.179 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:26.193+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:26.185 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:26.185 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:26.186 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:48:26.195 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-20T01:48:26.207 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:48:26.229 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:48:26.249 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:48:26.269 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:48:26.281 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:26.295+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:26.290 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:48:26.312 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:48:26.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:48:26.327+0000 7f1e65c1f700 0 -- 172.21.15.44:0/2516077953 >> [v2:172.21.15.165:6816/74562,v1:172.21.15.165:6817/74562] conn(0x7f1e60221780 msgr2=0x7f1e60223b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-20T01:48:26.333 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:48:26.334 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:48:26.334 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:48:26.334 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:48:26.334 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:48:26.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:48:26.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:48:26.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:48:26.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:48:26.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:48:26.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:48:26.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:48:26.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:48:26.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:48:26.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:48:26.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:48:26.335 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:48:26.364 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:26.364 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:26.364 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:48:26.364 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:48:26.365 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:26.375 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-20T01:48:26.418 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:26.432+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:26.484 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:26.497+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:26.505 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:26.505 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:48:26.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:48:26.505 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:48:26.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.506 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:104 3.es0 3:78f9000f:::smithi16579500-35:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:26.506 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:48:26.506 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 51.555995613999997, 2024-04-20T01:48:26.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012120811, 2024-04-20T01:48:26.506 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:26.506 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:26.506 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:26.506 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:26.507 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:26.507 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 104 2024-04-20T01:48:26.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.507 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:26.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:26.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:48:26.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:26.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:48:26.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:26.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:48:26.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:26.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:26.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962445+0000", 2024-04-20T01:48:26.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3210000000000001e-06 2024-04-20T01:48:26.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:26.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962451+0000", 2024-04-20T01:48:26.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.31e-06 2024-04-20T01:48:26.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:26.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962457+0000", 2024-04-20T01:48:26.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4819999999999999e-06 2024-04-20T01:48:26.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966434+0000", 2024-04-20T01:48:26.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039772239999999997 2024-04-20T01:48:26.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:26.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966501+0000", 2024-04-20T01:48:26.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6947999999999997e-05 2024-04-20T01:48:26.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:26.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966766+0000", 2024-04-20T01:48:26.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00026468999999999999 2024-04-20T01:48:26.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:26.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970614+0000", 2024-04-20T01:48:26.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0038481330000000001 2024-04-20T01:48:26.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:26.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974486+0000", 2024-04-20T01:48:26.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003871799 2024-04-20T01:48:26.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:26.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974563+0000", 2024-04-20T01:48:26.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7416999999999994e-05 2024-04-20T01:48:26.513 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.513 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.513 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.513 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:105 3.es0 3:78f9000f:::smithi16579500-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:26.513 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962461+0000", 2024-04-20T01:48:26.514 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 51.555977403999997, 2024-04-20T01:48:26.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014134643000000001, 2024-04-20T01:48:26.514 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:26.514 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:26.514 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:26.514 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:26.514 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:26.514 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 105 2024-04-20T01:48:26.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.515 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:26.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:26.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962461+0000", 2024-04-20T01:48:26.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:26.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962461+0000", 2024-04-20T01:48:26.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:26.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962460+0000", 2024-04-20T01:48:26.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:26.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:26.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962462+0000", 2024-04-20T01:48:26.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.897e-06 2024-04-20T01:48:26.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:26.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962466+0000", 2024-04-20T01:48:26.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4520000000000002e-06 2024-04-20T01:48:26.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:26.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962468+0000", 2024-04-20T01:48:26.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6570000000000001e-06 2024-04-20T01:48:26.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966956+0000", 2024-04-20T01:48:26.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044879730000000001 2024-04-20T01:48:26.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:26.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.967029+0000", 2024-04-20T01:48:26.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2657999999999995e-05 2024-04-20T01:48:26.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974602+0000", 2024-04-20T01:48:26.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075729370000000001 2024-04-20T01:48:26.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:26.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974650+0000", 2024-04-20T01:48:26.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8000000000000001e-05 2024-04-20T01:48:26.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:26.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976595+0000", 2024-04-20T01:48:26.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019453789999999999 2024-04-20T01:48:26.520 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.520 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.520 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.520 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:109 3.es0 3:7f925225:::smithi16579500-37:head [append 0~606208,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:26.521 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:48:26.521 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 51.555966972999997, 2024-04-20T01:48:26.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012453573000000001, 2024-04-20T01:48:26.521 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:26.521 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:26.521 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:26.521 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:26.521 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:26.521 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 109 2024-04-20T01:48:26.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.521 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:26.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:26.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:48:26.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:26.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:48:26.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:26.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:48:26.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:26.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:26.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962732+0000", 2024-04-20T01:48:26.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00026073200000000001 2024-04-20T01:48:26.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:26.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962742+0000", 2024-04-20T01:48:26.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0735e-05 2024-04-20T01:48:26.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:26.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962749+0000", 2024-04-20T01:48:26.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.365e-06 2024-04-20T01:48:26.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.967197+0000", 2024-04-20T01:48:26.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044487119999999996 2024-04-20T01:48:26.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:26.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.967324+0000", 2024-04-20T01:48:26.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126575 2024-04-20T01:48:26.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:26.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968044+0000", 2024-04-20T01:48:26.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00072015399999999998 2024-04-20T01:48:26.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:26.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970795+0000", 2024-04-20T01:48:26.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027507650000000001 2024-04-20T01:48:26.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:26.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974884+0000", 2024-04-20T01:48:26.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040886469999999999 2024-04-20T01:48:26.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:26.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974925+0000", 2024-04-20T01:48:26.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1094000000000003e-05 2024-04-20T01:48:26.526 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.526 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.526 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.526 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:110 3.es0 3:7f925225:::smithi16579500-37:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:26.526 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962755+0000", 2024-04-20T01:48:26.526 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 51.555683103, 2024-04-20T01:48:26.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012356723999999999, 2024-04-20T01:48:26.526 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:26.526 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:26.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:26.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:26.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:26.527 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 110 2024-04-20T01:48:26.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.527 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:26.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:26.527 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962755+0000", 2024-04-20T01:48:26.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:26.527 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962755+0000", 2024-04-20T01:48:26.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:26.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962754+0000", 2024-04-20T01:48:26.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:26.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:26.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962758+0000", 2024-04-20T01:48:26.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8750000000000002e-06 2024-04-20T01:48:26.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:26.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962765+0000", 2024-04-20T01:48:26.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2370000000000003e-06 2024-04-20T01:48:26.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:26.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962770+0000", 2024-04-20T01:48:26.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9030000000000003e-06 2024-04-20T01:48:26.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968614+0000", 2024-04-20T01:48:26.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0058437619999999997 2024-04-20T01:48:26.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:26.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968676+0000", 2024-04-20T01:48:26.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1592000000000003e-05 2024-04-20T01:48:26.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:26.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968935+0000", 2024-04-20T01:48:26.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025953599999999999 2024-04-20T01:48:26.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:26.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970828+0000", 2024-04-20T01:48:26.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0018928510000000001 2024-04-20T01:48:26.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:26.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975036+0000", 2024-04-20T01:48:26.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042079680000000003 2024-04-20T01:48:26.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:26.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975112+0000", 2024-04-20T01:48:26.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5558999999999999e-05 2024-04-20T01:48:26.531 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.531 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.531 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.531 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:111 3.es0 3:7f925225:::smithi16579500-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:26.531 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962775+0000", 2024-04-20T01:48:26.532 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 51.555663348000003, 2024-04-20T01:48:26.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014337756, 2024-04-20T01:48:26.532 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:26.532 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:26.532 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:26.532 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:26.532 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:26.532 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 111 2024-04-20T01:48:26.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.532 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:26.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:26.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962775+0000", 2024-04-20T01:48:26.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:26.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962775+0000", 2024-04-20T01:48:26.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:26.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962774+0000", 2024-04-20T01:48:26.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:26.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:26.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962777+0000", 2024-04-20T01:48:26.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7839999999999999e-06 2024-04-20T01:48:26.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:26.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962784+0000", 2024-04-20T01:48:26.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9809999999999997e-06 2024-04-20T01:48:26.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:26.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962790+0000", 2024-04-20T01:48:26.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5509999999999999e-06 2024-04-20T01:48:26.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970564+0000", 2024-04-20T01:48:26.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077740680000000003 2024-04-20T01:48:26.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:26.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970609+0000", 2024-04-20T01:48:26.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5059999999999999e-05 2024-04-20T01:48:26.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975165+0000", 2024-04-20T01:48:26.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0045564999999999998 2024-04-20T01:48:26.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:26.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975263+0000", 2024-04-20T01:48:26.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7689000000000006e-05 2024-04-20T01:48:26.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:26.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977112+0000", 2024-04-20T01:48:26.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001849552 2024-04-20T01:48:26.536 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.536 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.536 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.536 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:142 3.es0 3:797661d0:::smithi16579500-48:head [append 0~425984,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:26.536 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962794+0000", 2024-04-20T01:48:26.536 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 51.555644147000002, 2024-04-20T01:48:26.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013969364, 2024-04-20T01:48:26.537 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:26.537 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:26.537 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:26.537 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:26.537 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:26.537 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 142 2024-04-20T01:48:26.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.537 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:26.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:26.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962794+0000", 2024-04-20T01:48:26.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:26.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962794+0000", 2024-04-20T01:48:26.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:26.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962793+0000", 2024-04-20T01:48:26.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:26.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:26.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963014+0000", 2024-04-20T01:48:26.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000221015 2024-04-20T01:48:26.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:26.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963025+0000", 2024-04-20T01:48:26.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0739e-05 2024-04-20T01:48:26.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:26.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963032+0000", 2024-04-20T01:48:26.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7580000000000003e-06 2024-04-20T01:48:26.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.971338+0000", 2024-04-20T01:48:26.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008305659 2024-04-20T01:48:26.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:26.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.971477+0000", 2024-04-20T01:48:26.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013920100000000001 2024-04-20T01:48:26.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:26.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.972169+0000", 2024-04-20T01:48:26.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00069257800000000001 2024-04-20T01:48:26.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:26.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974577+0000", 2024-04-20T01:48:26.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024072820000000002 2024-04-20T01:48:26.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:26.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976694+0000", 2024-04-20T01:48:26.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021172629999999999 2024-04-20T01:48:26.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:26.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976763+0000", 2024-04-20T01:48:26.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9259999999999998e-05 2024-04-20T01:48:26.542 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.542 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.542 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.543 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:143 3.es0 3:797661d0:::smithi16579500-48:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:26.543 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.963039+0000", 2024-04-20T01:48:26.543 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 51.555399391999998, 2024-04-20T01:48:26.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013923471999999999, 2024-04-20T01:48:26.543 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:26.543 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:26.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:26.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:26.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:26.543 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 143 2024-04-20T01:48:26.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.544 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:26.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:26.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963039+0000", 2024-04-20T01:48:26.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:26.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963039+0000", 2024-04-20T01:48:26.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:26.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963038+0000", 2024-04-20T01:48:26.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:26.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:26.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963210+0000", 2024-04-20T01:48:26.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017155800000000001 2024-04-20T01:48:26.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:26.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963217+0000", 2024-04-20T01:48:26.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7309999999999993e-06 2024-04-20T01:48:26.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:26.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963230+0000", 2024-04-20T01:48:26.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.295e-05 2024-04-20T01:48:26.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.972654+0000", 2024-04-20T01:48:26.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094237369999999997 2024-04-20T01:48:26.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:26.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.972720+0000", 2024-04-20T01:48:26.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5958999999999996e-05 2024-04-20T01:48:26.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:26.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.973225+0000", 2024-04-20T01:48:26.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00050500399999999998 2024-04-20T01:48:26.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:26.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974578+0000", 2024-04-20T01:48:26.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001353295 2024-04-20T01:48:26.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:26.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976919+0000", 2024-04-20T01:48:26.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023405840000000002 2024-04-20T01:48:26.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:26.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976962+0000", 2024-04-20T01:48:26.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3226000000000003e-05 2024-04-20T01:48:26.549 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.550 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.550 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.550 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:144 3.es0 3:797661d0:::smithi16579500-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:26.550 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:48:26.550 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 51.555200757999998, 2024-04-20T01:48:26.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014456992, 2024-04-20T01:48:26.550 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:26.550 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:26.550 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:26.550 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:26.550 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:26.550 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 144 2024-04-20T01:48:26.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.551 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:26.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:26.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:48:26.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:26.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:48:26.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:26.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:48:26.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:26.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:26.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963240+0000", 2024-04-20T01:48:26.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6459999999999999e-06 2024-04-20T01:48:26.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:26.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963247+0000", 2024-04-20T01:48:26.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9650000000000002e-06 2024-04-20T01:48:26.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:26.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963253+0000", 2024-04-20T01:48:26.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1900000000000003e-06 2024-04-20T01:48:26.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.973649+0000", 2024-04-20T01:48:26.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010396865999999999 2024-04-20T01:48:26.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:26.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.973745+0000", 2024-04-20T01:48:26.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5345000000000003e-05 2024-04-20T01:48:26.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:26.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974151+0000", 2024-04-20T01:48:26.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000405893 2024-04-20T01:48:26.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:26.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976606+0000", 2024-04-20T01:48:26.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024551730000000002 2024-04-20T01:48:26.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:26.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977622+0000", 2024-04-20T01:48:26.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0010159570000000001 2024-04-20T01:48:26.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:26.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977694+0000", 2024-04-20T01:48:26.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2469e-05 2024-04-20T01:48:26.556 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.556 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.556 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.556 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:145 3.es0 3:797661d0:::smithi16579500-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:26.556 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:48:26.556 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 51.555178937000001, 2024-04-20T01:48:26.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016339408999999999, 2024-04-20T01:48:26.557 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:26.557 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:26.557 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:26.557 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:26.557 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:26.557 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 145 2024-04-20T01:48:26.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.557 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:26.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:26.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:48:26.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:26.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:48:26.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:26.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:48:26.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:26.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:26.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963267+0000", 2024-04-20T01:48:26.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2490000000000001e-06 2024-04-20T01:48:26.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:26.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963274+0000", 2024-04-20T01:48:26.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7599999999999997e-06 2024-04-20T01:48:26.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:26.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963279+0000", 2024-04-20T01:48:26.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5400000000000003e-06 2024-04-20T01:48:26.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977300+0000", 2024-04-20T01:48:26.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014020608 2024-04-20T01:48:26.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:26.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977365+0000", 2024-04-20T01:48:26.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5617999999999994e-05 2024-04-20T01:48:26.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.978057+0000", 2024-04-20T01:48:26.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00069179099999999998 2024-04-20T01:48:26.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:26.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.978136+0000", 2024-04-20T01:48:26.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9132000000000001e-05 2024-04-20T01:48:26.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:26.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.979598+0000", 2024-04-20T01:48:26.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0014621599999999999 2024-04-20T01:48:26.561 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.561 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.561 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.561 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:245 3.es0 3:78f9000f:::smithi16579500-35:head [copy-from ver 10] snapc 5=[2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:26.561 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785668+0000", 2024-04-20T01:48:26.561 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 46.732769750999999, 2024-04-20T01:48:26.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025155522, 2024-04-20T01:48:26.562 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:26.562 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:26.562 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:26.562 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:26.562 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:26.562 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 245 2024-04-20T01:48:26.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.562 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:26.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:26.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785668+0000", 2024-04-20T01:48:26.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:26.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785668+0000", 2024-04-20T01:48:26.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:26.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785666+0000", 2024-04-20T01:48:26.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:26.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:26.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785676+0000", 2024-04-20T01:48:26.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0139999999999999e-05 2024-04-20T01:48:26.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:26.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785692+0000", 2024-04-20T01:48:26.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5894999999999999e-05 2024-04-20T01:48:26.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:26.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785715+0000", 2024-04-20T01:48:26.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2654999999999999e-05 2024-04-20T01:48:26.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785771+0000", 2024-04-20T01:48:26.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6331e-05 2024-04-20T01:48:26.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:26.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785779+0000", 2024-04-20T01:48:26.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4789999999999999e-06 2024-04-20T01:48:26.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791776+0000", 2024-04-20T01:48:26.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059978840000000002 2024-04-20T01:48:26.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:26.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791905+0000", 2024-04-20T01:48:26.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000128147 2024-04-20T01:48:26.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:26.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.805603+0000", 2024-04-20T01:48:26.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013698824 2024-04-20T01:48:26.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:26.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.808612+0000", 2024-04-20T01:48:26.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003008541 2024-04-20T01:48:26.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:26.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.810754+0000", 2024-04-20T01:48:26.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021421840000000001 2024-04-20T01:48:26.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:26.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.810824+0000", 2024-04-20T01:48:26.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9627000000000003e-05 2024-04-20T01:48:26.567 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.567 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.567 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.567 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:246 3.es0 3:78f9000f:::smithi16579500-35:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:26.567 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:48:26.567 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 46.732709571999997, 2024-04-20T01:48:26.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020980167000000001, 2024-04-20T01:48:26.567 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:26.567 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:26.568 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:26.568 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:26.568 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:26.568 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 246 2024-04-20T01:48:26.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.568 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:26.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:26.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:48:26.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:26.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:48:26.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:26.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:48:26.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:26.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:26.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785731+0000", 2024-04-20T01:48:26.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3550000000000001e-06 2024-04-20T01:48:26.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:26.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785738+0000", 2024-04-20T01:48:26.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2080000000000003e-06 2024-04-20T01:48:26.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:26.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785747+0000", 2024-04-20T01:48:26.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.388e-06 2024-04-20T01:48:26.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785859+0000", 2024-04-20T01:48:26.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112432 2024-04-20T01:48:26.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:26.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785866+0000", 2024-04-20T01:48:26.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0890000000000002e-06 2024-04-20T01:48:26.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792104+0000", 2024-04-20T01:48:26.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062376970000000004 2024-04-20T01:48:26.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:26.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792157+0000", 2024-04-20T01:48:26.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3269000000000001e-05 2024-04-20T01:48:26.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.806521+0000", 2024-04-20T01:48:26.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014363651999999999 2024-04-20T01:48:26.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:26.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.806639+0000", 2024-04-20T01:48:26.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011755400000000001 2024-04-20T01:48:26.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:26.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.806709+0000", 2024-04-20T01:48:26.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0037999999999996e-05 2024-04-20T01:48:26.573 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.573 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.573 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.573 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:247 3.es0 3:79bd6b86:::smithi16579500-12:head [read 0~196608,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:26.573 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785760+0000", 2024-04-20T01:48:26.573 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 46.732678323000002, 2024-04-20T01:48:26.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011359487999999999, 2024-04-20T01:48:26.573 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:26.573 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:26.573 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:26.573 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:26.574 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:26.574 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 247 2024-04-20T01:48:26.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.574 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:26.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:26.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785760+0000", 2024-04-20T01:48:26.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:26.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785760+0000", 2024-04-20T01:48:26.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:26.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785759+0000", 2024-04-20T01:48:26.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:26.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:26.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785764+0000", 2024-04-20T01:48:26.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.194e-06 2024-04-20T01:48:26.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:26.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785772+0000", 2024-04-20T01:48:26.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9150000000000002e-06 2024-04-20T01:48:26.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:26.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785780+0000", 2024-04-20T01:48:26.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5639999999999999e-06 2024-04-20T01:48:26.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785905+0000", 2024-04-20T01:48:26.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012477000000000001 2024-04-20T01:48:26.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:26.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785909+0000", 2024-04-20T01:48:26.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1099999999999996e-06 2024-04-20T01:48:26.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792209+0000", 2024-04-20T01:48:26.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062998009999999998 2024-04-20T01:48:26.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:26.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792316+0000", 2024-04-20T01:48:26.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107161 2024-04-20T01:48:26.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:26.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.797119+0000", 2024-04-20T01:48:26.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0048032860000000004 2024-04-20T01:48:26.579 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.579 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.579 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.579 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:248 3.es0 3:79bd6b86:::smithi16579500-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:26.579 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:48:26.579 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 46.732653812000002, 2024-04-20T01:48:26.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011602988, 2024-04-20T01:48:26.579 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:26.579 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:26.579 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:26.580 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:26.580 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:26.580 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 248 2024-04-20T01:48:26.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.580 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:26.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:26.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:48:26.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:26.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:48:26.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:26.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:48:26.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:26.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:26.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785787+0000", 2024-04-20T01:48:26.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4079999999999998e-06 2024-04-20T01:48:26.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:26.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785791+0000", 2024-04-20T01:48:26.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9929999999999997e-06 2024-04-20T01:48:26.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:26.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785795+0000", 2024-04-20T01:48:26.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1559999999999997e-06 2024-04-20T01:48:26.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785936+0000", 2024-04-20T01:48:26.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014018 2024-04-20T01:48:26.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:26.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785939+0000", 2024-04-20T01:48:26.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-04-20T01:48:26.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794183+0000", 2024-04-20T01:48:26.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082434339999999991 2024-04-20T01:48:26.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:26.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794272+0000", 2024-04-20T01:48:26.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9246000000000006e-05 2024-04-20T01:48:26.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:26.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.797387+0000", 2024-04-20T01:48:26.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0031150269999999998 2024-04-20T01:48:26.585 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.585 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.586 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.586 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:249 3.es0 3:79bd6b86:::smithi16579500-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:26.586 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:48:26.586 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 46.732639515000002, 2024-04-20T01:48:26.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011999312, 2024-04-20T01:48:26.586 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:26.586 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:26.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:26.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:26.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:26.587 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 249 2024-04-20T01:48:26.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.587 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:26.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:26.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:48:26.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:26.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:48:26.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:26.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:48:26.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:26.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:26.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785800+0000", 2024-04-20T01:48:26.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.995e-06 2024-04-20T01:48:26.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:26.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785804+0000", 2024-04-20T01:48:26.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2990000000000001e-06 2024-04-20T01:48:26.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:26.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785807+0000", 2024-04-20T01:48:26.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.715e-06 2024-04-20T01:48:26.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785955+0000", 2024-04-20T01:48:26.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000147923 2024-04-20T01:48:26.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:26.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785958+0000", 2024-04-20T01:48:26.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8940000000000002e-06 2024-04-20T01:48:26.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794609+0000", 2024-04-20T01:48:26.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086509289999999999 2024-04-20T01:48:26.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:26.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.795125+0000", 2024-04-20T01:48:26.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00051564999999999998 2024-04-20T01:48:26.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:26.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.797798+0000", 2024-04-20T01:48:26.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0026731250000000002 2024-04-20T01:48:26.592 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.592 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.592 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.592 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:586 3.es0 3:79bd6b86:::smithi16579500-12:head [delete] snapc 20=[20,1f,1a,b] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e65)", 2024-04-20T01:48:26.592 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:06.762289+0000", 2024-04-20T01:48:26.592 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 19.756149436000001, 2024-04-20T01:48:26.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011713963000000001, 2024-04-20T01:48:26.592 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:26.592 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:26.592 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:26.592 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:26.592 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:26.593 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 586 2024-04-20T01:48:26.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.593 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:26.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:26.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762289+0000", 2024-04-20T01:48:26.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:26.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762289+0000", 2024-04-20T01:48:26.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:26.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762287+0000", 2024-04-20T01:48:26.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:26.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:26.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762294+0000", 2024-04-20T01:48:26.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6450000000000005e-06 2024-04-20T01:48:26.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:26.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762308+0000", 2024-04-20T01:48:26.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3652000000000001e-05 2024-04-20T01:48:26.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:26.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762318+0000", 2024-04-20T01:48:26.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6679999999999996e-06 2024-04-20T01:48:26.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762391+0000", 2024-04-20T01:48:26.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3823000000000001e-05 2024-04-20T01:48:26.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:26.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762400+0000", 2024-04-20T01:48:26.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6750000000000008e-06 2024-04-20T01:48:26.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.767267+0000", 2024-04-20T01:48:26.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0048666300000000003 2024-04-20T01:48:26.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:26.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.767389+0000", 2024-04-20T01:48:26.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012267699999999999 2024-04-20T01:48:26.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:26.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.774003+0000", 2024-04-20T01:48:26.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066131640000000004 2024-04-20T01:48:26.597 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.597 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.597 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.597 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.bs4 65/30 ECSubWrite(tid=174, reqid=osd.4.0:174, at_version=65'88, trim_to=0'0, roll_forward_to=63'84))", 2024-04-20T01:48:26.597 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:06.764164+0000", 2024-04-20T01:48:26.597 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 19.754274153000001, 2024-04-20T01:48:26.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011467442, 2024-04-20T01:48:26.597 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:26.597 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:26.597 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:26.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:26.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.764164+0000", 2024-04-20T01:48:26.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:26.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.764164+0000", 2024-04-20T01:48:26.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:26.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.764164+0000", 2024-04-20T01:48:26.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:26.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:26.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.764187+0000", 2024-04-20T01:48:26.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3390000000000001e-05 2024-04-20T01:48:26.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:26.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.764193+0000", 2024-04-20T01:48:26.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7010000000000001e-06 2024-04-20T01:48:26.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:26.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.764197+0000", 2024-04-20T01:48:26.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5750000000000002e-06 2024-04-20T01:48:26.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.767988+0000", 2024-04-20T01:48:26.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003791132 2024-04-20T01:48:26.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:26.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.767998+0000", 2024-04-20T01:48:26.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4900000000000006e-06 2024-04-20T01:48:26.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:26.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.775599+0000", 2024-04-20T01:48:26.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076008170000000002 2024-04-20T01:48:26.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:26.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.775631+0000", 2024-04-20T01:48:26.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2527000000000002e-05 2024-04-20T01:48:26.601 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.601 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.601 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.601 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:727 3.es0 3:797661d0:::smithi16579500-48:head [append 0~622592] snapc 26=[26,25,1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-20T01:48:26.601 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:11.814331+0000", 2024-04-20T01:48:26.601 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.704107404, 2024-04-20T01:48:26.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013760674000000001, 2024-04-20T01:48:26.602 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:26.602 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:26.602 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:26.602 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:26.602 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:26.602 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 727 2024-04-20T01:48:26.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.602 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:26.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:26.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814331+0000", 2024-04-20T01:48:26.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:26.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814331+0000", 2024-04-20T01:48:26.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:26.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814329+0000", 2024-04-20T01:48:26.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:26.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:26.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814688+0000", 2024-04-20T01:48:26.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00035975800000000001 2024-04-20T01:48:26.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:26.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814706+0000", 2024-04-20T01:48:26.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7248000000000002e-05 2024-04-20T01:48:26.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:26.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814716+0000", 2024-04-20T01:48:26.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9620000000000005e-06 2024-04-20T01:48:26.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814789+0000", 2024-04-20T01:48:26.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3734999999999998e-05 2024-04-20T01:48:26.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:26.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814800+0000", 2024-04-20T01:48:26.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0898e-05 2024-04-20T01:48:26.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.816951+0000", 2024-04-20T01:48:26.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021503379999999999 2024-04-20T01:48:26.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:26.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.817082+0000", 2024-04-20T01:48:26.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013159599999999999 2024-04-20T01:48:26.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:26.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.818406+0000", 2024-04-20T01:48:26.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013237749999999999 2024-04-20T01:48:26.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:26.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.822671+0000", 2024-04-20T01:48:26.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042655460000000003 2024-04-20T01:48:26.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:26.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.828006+0000", 2024-04-20T01:48:26.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005334999 2024-04-20T01:48:26.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:26.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.828091+0000", 2024-04-20T01:48:26.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4857000000000004e-05 2024-04-20T01:48:26.607 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.607 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.607 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.607 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:728 3.es0 3:797661d0:::smithi16579500-48:head [append 0~655360] snapc 26=[26,25,1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-20T01:48:26.607 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:11.818080+0000", 2024-04-20T01:48:26.608 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.700358297999999, 2024-04-20T01:48:26.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010815946999999999, 2024-04-20T01:48:26.608 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:26.608 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:26.608 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:26.608 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:26.608 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:26.608 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 728 2024-04-20T01:48:26.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.608 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:26.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:26.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.818080+0000", 2024-04-20T01:48:26.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:26.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.818080+0000", 2024-04-20T01:48:26.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:26.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.818078+0000", 2024-04-20T01:48:26.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:26.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:26.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.818644+0000", 2024-04-20T01:48:26.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00056597699999999995 2024-04-20T01:48:26.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:26.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.818667+0000", 2024-04-20T01:48:26.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2955e-05 2024-04-20T01:48:26.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:26.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.818704+0000", 2024-04-20T01:48:26.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6358e-05 2024-04-20T01:48:26.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.819553+0000", 2024-04-20T01:48:26.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00084911100000000005 2024-04-20T01:48:26.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:26.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.819687+0000", 2024-04-20T01:48:26.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013430899999999999 2024-04-20T01:48:26.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:26.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.820731+0000", 2024-04-20T01:48:26.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0010439430000000001 2024-04-20T01:48:26.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:26.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.828277+0000", 2024-04-20T01:48:26.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075456940000000004 2024-04-20T01:48:26.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:26.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.828824+0000", 2024-04-20T01:48:26.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054766299999999999 2024-04-20T01:48:26.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:26.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.828896+0000", 2024-04-20T01:48:26.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1407000000000003e-05 2024-04-20T01:48:26.613 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.614 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.614 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.614 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:729 3.es0 3:797661d0:::smithi16579500-48:head [append 0~688128] snapc 26=[26,25,1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-20T01:48:26.614 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:11.822329+0000", 2024-04-20T01:48:26.614 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.696109347, 2024-04-20T01:48:26.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011053970999999999, 2024-04-20T01:48:26.614 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:26.614 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:26.614 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:26.615 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:26.615 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:26.615 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 729 2024-04-20T01:48:26.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.615 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:26.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:26.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.822329+0000", 2024-04-20T01:48:26.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:26.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.822329+0000", 2024-04-20T01:48:26.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:26.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.822327+0000", 2024-04-20T01:48:26.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:26.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:26.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.822756+0000", 2024-04-20T01:48:26.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00042862500000000002 2024-04-20T01:48:26.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:26.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.822769+0000", 2024-04-20T01:48:26.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3022e-05 2024-04-20T01:48:26.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:26.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.822780+0000", 2024-04-20T01:48:26.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1327e-05 2024-04-20T01:48:26.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.822814+0000", 2024-04-20T01:48:26.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3550000000000002e-05 2024-04-20T01:48:26.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:26.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.822892+0000", 2024-04-20T01:48:26.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8342000000000003e-05 2024-04-20T01:48:26.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:26.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823588+0000", 2024-04-20T01:48:26.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00069574600000000002 2024-04-20T01:48:26.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:26.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.833222+0000", 2024-04-20T01:48:26.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0096334950000000006 2024-04-20T01:48:26.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:26.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.833299+0000", 2024-04-20T01:48:26.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7859000000000001e-05 2024-04-20T01:48:26.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:26.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.833383+0000", 2024-04-20T01:48:26.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3232999999999994e-05 2024-04-20T01:48:26.621 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.621 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.621 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.621 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:732 3.es0 3:797661d0:::smithi16579500-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e70)", 2024-04-20T01:48:26.621 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:11.823728+0000", 2024-04-20T01:48:26.621 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.694710253, 2024-04-20T01:48:26.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012035161000000001, 2024-04-20T01:48:26.622 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:26.622 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:26.622 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:26.622 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:26.622 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:26.622 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 732 2024-04-20T01:48:26.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.622 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:26.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:26.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823728+0000", 2024-04-20T01:48:26.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:26.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823728+0000", 2024-04-20T01:48:26.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:26.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:26.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823727+0000", 2024-04-20T01:48:26.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:26.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:26.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823731+0000", 2024-04-20T01:48:26.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8349999999999997e-06 2024-04-20T01:48:26.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:26.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823745+0000", 2024-04-20T01:48:26.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4211e-05 2024-04-20T01:48:26.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:26.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823756+0000", 2024-04-20T01:48:26.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0705e-05 2024-04-20T01:48:26.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.827650+0000", 2024-04-20T01:48:26.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003894163 2024-04-20T01:48:26.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:26.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.827758+0000", 2024-04-20T01:48:26.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107759 2024-04-20T01:48:26.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:26.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.834824+0000", 2024-04-20T01:48:26.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.007066587 2024-04-20T01:48:26.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:26.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.834920+0000", 2024-04-20T01:48:26.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5639999999999999e-05 2024-04-20T01:48:26.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:26.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:26.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:26.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.835763+0000", 2024-04-20T01:48:26.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000842835 2024-04-20T01:48:26.626 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.626 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.626 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.626 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:26.627 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:26.627 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:26.628 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:26.643+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:26.629 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-20T01:48:26.729 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:26.743+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:26.740 INFO:tasks.rados.rados.0.smithi165.stdout:360: done (5 left) 2024-04-20T01:48:26.740 INFO:tasks.rados.rados.0.smithi165.stdout:361: done (4 left) 2024-04-20T01:48:26.740 INFO:tasks.rados.rados.0.smithi165.stdout:362: done (3 left) 2024-04-20T01:48:26.740 INFO:tasks.rados.rados.0.smithi165.stdout:363: done (2 left) 2024-04-20T01:48:26.740 INFO:tasks.rados.rados.0.smithi165.stdout:365: done (1 left) 2024-04-20T01:48:26.740 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 5 current snap is 28 2024-04-20T01:48:26.740 INFO:tasks.rados.rados.0.smithi165.stdout:366: seq_num 154 ranges {3981312=180224} 2024-04-20T01:48:26.744 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is unset 2024-04-20T01:48:26.762 INFO:tasks.rados.rados.0.smithi165.stdout:366: writing smithi16579500-5 from 3981312 to 4161536 tid 1 2024-04-20T01:48:26.762 INFO:tasks.rados.rados.0.smithi165.stdout:367: snap_create 2024-04-20T01:48:26.768 INFO:tasks.rados.rados.0.smithi165.stdout:366: finishing write tid 1 to smithi16579500-5 2024-04-20T01:48:26.768 INFO:tasks.rados.rados.0.smithi165.stdout:366: finishing write tid 2 to smithi16579500-5 2024-04-20T01:48:26.769 INFO:tasks.rados.rados.0.smithi165.stdout:366: finishing write tid 3 to smithi16579500-5 2024-04-20T01:48:26.769 INFO:tasks.rados.rados.0.smithi165.stdout:366: oid 5 updating version 0 to 57 2024-04-20T01:48:26.769 INFO:tasks.rados.rados.0.smithi165.stdout:366: oid 5 version 57 is already newer than 56 2024-04-20T01:48:26.769 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 5 v 57 (ObjNum 154 snap 28 seq_num 154) dirty exists 2024-04-20T01:48:26.769 INFO:tasks.rados.rados.0.smithi165.stdout:366: left oid 5 (ObjNum 154 snap 28 seq_num 154) 2024-04-20T01:48:26.802 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:26.802 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:26.802 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-04-20T01:48:26.802 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:26.812 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-20T01:48:26.830 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:26.844+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:26.931 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:48:26.945+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:48:26.941 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:26.941 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:26.941 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:48:26.941 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:48:26.941 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:26.951 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-20T01:48:27.032 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:27.046+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:27.121 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:48:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:48:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:48:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 37/30 ECSubWrite(tid=43, reqid=client.4643.0:241, at_version=37'31, trim_to=0'0, roll_forward_to=36'28))", 2024-04-20T01:48:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.793501+0000", 2024-04-20T01:48:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 48.340846562000003, 2024-04-20T01:48:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005847517, 2024-04-20T01:48:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.121 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793501+0000", 2024-04-20T01:48:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793501+0000", 2024-04-20T01:48:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793500+0000", 2024-04-20T01:48:27.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793525+0000", 2024-04-20T01:48:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4641999999999999e-05 2024-04-20T01:48:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793537+0000", 2024-04-20T01:48:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2626000000000001e-05 2024-04-20T01:48:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793547+0000", 2024-04-20T01:48:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.842e-06 2024-04-20T01:48:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793599+0000", 2024-04-20T01:48:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2301e-05 2024-04-20T01:48:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793607+0000", 2024-04-20T01:48:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9070000000000007e-06 2024-04-20T01:48:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799321+0000", 2024-04-20T01:48:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005713358 2024-04-20T01:48:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799349+0000", 2024-04-20T01:48:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7832999999999999e-05 2024-04-20T01:48:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 37/30 ECSubWrite(tid=19, reqid=client.4643.0:235, at_version=37'9, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:48:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.793816+0000", 2024-04-20T01:48:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 48.340531087000002, 2024-04-20T01:48:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0055550790000000001, 2024-04-20T01:48:27.125 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793816+0000", 2024-04-20T01:48:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793816+0000", 2024-04-20T01:48:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793816+0000", 2024-04-20T01:48:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793838+0000", 2024-04-20T01:48:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2075000000000001e-05 2024-04-20T01:48:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793849+0000", 2024-04-20T01:48:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1433999999999999e-05 2024-04-20T01:48:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793859+0000", 2024-04-20T01:48:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4849999999999998e-06 2024-04-20T01:48:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793956+0000", 2024-04-20T01:48:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7114000000000006e-05 2024-04-20T01:48:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793965+0000", 2024-04-20T01:48:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0820000000000005e-06 2024-04-20T01:48:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799344+0000", 2024-04-20T01:48:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053794400000000001 2024-04-20T01:48:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799372+0000", 2024-04-20T01:48:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7231000000000001e-05 2024-04-20T01:48:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 37/30 ECSubWrite(tid=44, reqid=client.4643.0:242, at_version=37'32, trim_to=0'0, roll_forward_to=36'28))", 2024-04-20T01:48:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.794496+0000", 2024-04-20T01:48:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 48.339851195000001, 2024-04-20T01:48:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0049252380000000002, 2024-04-20T01:48:27.129 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794496+0000", 2024-04-20T01:48:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794496+0000", 2024-04-20T01:48:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794496+0000", 2024-04-20T01:48:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794516+0000", 2024-04-20T01:48:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0475999999999998e-05 2024-04-20T01:48:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794530+0000", 2024-04-20T01:48:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3565e-05 2024-04-20T01:48:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794539+0000", 2024-04-20T01:48:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9600000000000006e-06 2024-04-20T01:48:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794591+0000", 2024-04-20T01:48:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2574999999999997e-05 2024-04-20T01:48:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794602+0000", 2024-04-20T01:48:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1338999999999999e-05 2024-04-20T01:48:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799353+0000", 2024-04-20T01:48:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047508999999999997 2024-04-20T01:48:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799422+0000", 2024-04-20T01:48:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8281999999999994e-05 2024-04-20T01:48:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 38/30 ECSubWrite(tid=23, reqid=osd.3.0:22, at_version=0'0, trim_to=0'0, roll_forward_to=38'12))", 2024-04-20T01:48:27.133 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.788077+0000", 2024-04-20T01:48:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.346271033000001, 2024-04-20T01:48:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057844480000000002, 2024-04-20T01:48:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788077+0000", 2024-04-20T01:48:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788077+0000", 2024-04-20T01:48:27.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788076+0000", 2024-04-20T01:48:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788088+0000", 2024-04-20T01:48:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2672e-05 2024-04-20T01:48:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788101+0000", 2024-04-20T01:48:27.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2468e-05 2024-04-20T01:48:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788121+0000", 2024-04-20T01:48:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0315000000000001e-05 2024-04-20T01:48:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788181+0000", 2024-04-20T01:48:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0257999999999998e-05 2024-04-20T01:48:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788194+0000", 2024-04-20T01:48:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2714000000000001e-05 2024-04-20T01:48:27.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.793811+0000", 2024-04-20T01:48:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056169649999999998 2024-04-20T01:48:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.793861+0000", 2024-04-20T01:48:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9978000000000002e-05 2024-04-20T01:48:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 53/30 ECSubWrite(tid=32, reqid=client.4643.0:443, at_version=53'21, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.718947+0000", 2024-04-20T01:48:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 32.415400065, 2024-04-20T01:48:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009785129, 2024-04-20T01:48:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718947+0000", 2024-04-20T01:48:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:27.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718947+0000", 2024-04-20T01:48:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718947+0000", 2024-04-20T01:48:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719111+0000", 2024-04-20T01:48:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000164227 2024-04-20T01:48:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719125+0000", 2024-04-20T01:48:27.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.454e-05 2024-04-20T01:48:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719137+0000", 2024-04-20T01:48:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1783999999999999e-05 2024-04-20T01:48:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719206+0000", 2024-04-20T01:48:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9123999999999998e-05 2024-04-20T01:48:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719221+0000", 2024-04-20T01:48:27.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4963999999999999e-05 2024-04-20T01:48:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.728704+0000", 2024-04-20T01:48:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094822859999999995 2024-04-20T01:48:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.728733+0000", 2024-04-20T01:48:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8972e-05 2024-04-20T01:48:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 53/30 ECSubWrite(tid=33, reqid=client.4643.0:444, at_version=53'22, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.721427+0000", 2024-04-20T01:48:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 32.412920815, 2024-04-20T01:48:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010045943999999999, 2024-04-20T01:48:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721427+0000", 2024-04-20T01:48:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721427+0000", 2024-04-20T01:48:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721424+0000", 2024-04-20T01:48:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999967 2024-04-20T01:48:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721541+0000", 2024-04-20T01:48:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117309 2024-04-20T01:48:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721560+0000", 2024-04-20T01:48:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8559999999999998e-05 2024-04-20T01:48:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721571+0000", 2024-04-20T01:48:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1612e-05 2024-04-20T01:48:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721648+0000", 2024-04-20T01:48:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6534999999999998e-05 2024-04-20T01:48:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721660+0000", 2024-04-20T01:48:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2396e-05 2024-04-20T01:48:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731447+0000", 2024-04-20T01:48:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097865039999999997 2024-04-20T01:48:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731473+0000", 2024-04-20T01:48:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6129999999999999e-05 2024-04-20T01:48:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.145 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 53/30 ECSubWrite(tid=34, reqid=client.4643.0:445, at_version=53'23, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.722888+0000", 2024-04-20T01:48:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 32.411459335000004, 2024-04-20T01:48:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086321880000000007, 2024-04-20T01:48:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722888+0000", 2024-04-20T01:48:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722888+0000", 2024-04-20T01:48:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722887+0000", 2024-04-20T01:48:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722958+0000", 2024-04-20T01:48:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0167999999999996e-05 2024-04-20T01:48:27.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722970+0000", 2024-04-20T01:48:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2393e-05 2024-04-20T01:48:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722979+0000", 2024-04-20T01:48:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1509999999999993e-06 2024-04-20T01:48:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723935+0000", 2024-04-20T01:48:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00095590900000000001 2024-04-20T01:48:27.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723947+0000", 2024-04-20T01:48:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2405999999999999e-05 2024-04-20T01:48:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731503+0000", 2024-04-20T01:48:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075554990000000002 2024-04-20T01:48:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731520+0000", 2024-04-20T01:48:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7490999999999999e-05 2024-04-20T01:48:27.149 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 53/30 ECSubWrite(tid=35, reqid=client.4643.0:446, at_version=53'24, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.724462+0000", 2024-04-20T01:48:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 32.409885256000003, 2024-04-20T01:48:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0070986110000000003, 2024-04-20T01:48:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:27.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724462+0000", 2024-04-20T01:48:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724462+0000", 2024-04-20T01:48:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724461+0000", 2024-04-20T01:48:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:27.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724482+0000", 2024-04-20T01:48:27.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0557999999999999e-05 2024-04-20T01:48:27.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:27.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724502+0000", 2024-04-20T01:48:27.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9704999999999999e-05 2024-04-20T01:48:27.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724511+0000", 2024-04-20T01:48:27.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8699999999999998e-06 2024-04-20T01:48:27.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:27.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724537+0000", 2024-04-20T01:48:27.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.688e-05 2024-04-20T01:48:27.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724543+0000", 2024-04-20T01:48:27.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9909999999999999e-06 2024-04-20T01:48:27.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731548+0000", 2024-04-20T01:48:27.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0070045990000000002 2024-04-20T01:48:27.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:27.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731561+0000", 2024-04-20T01:48:27.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2857000000000001e-05 2024-04-20T01:48:27.154 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.154 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.154 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.154 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs2 55/30 ECSubWrite(tid=40, reqid=client.4643.0:458, at_version=55'22, trim_to=0'0, roll_forward_to=52'21))", 2024-04-20T01:48:27.154 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:56.729618+0000", 2024-04-20T01:48:27.154 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.404729315000001, 2024-04-20T01:48:27.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0048654960000000004, 2024-04-20T01:48:27.154 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:27.154 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.154 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:27.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:27.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.729618+0000", 2024-04-20T01:48:27.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:27.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.729618+0000", 2024-04-20T01:48:27.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:27.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.729618+0000", 2024-04-20T01:48:27.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:27.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:27.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.729643+0000", 2024-04-20T01:48:27.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4508999999999999e-05 2024-04-20T01:48:27.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:27.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.729655+0000", 2024-04-20T01:48:27.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2072999999999999e-05 2024-04-20T01:48:27.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.729665+0000", 2024-04-20T01:48:27.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9499999999999996e-06 2024-04-20T01:48:27.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:27.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.729752+0000", 2024-04-20T01:48:27.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7134999999999998e-05 2024-04-20T01:48:27.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.729762+0000", 2024-04-20T01:48:27.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0729e-05 2024-04-20T01:48:27.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.734446+0000", 2024-04-20T01:48:27.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046834499999999996 2024-04-20T01:48:27.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.158 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:27.158 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.734484+0000", 2024-04-20T01:48:27.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7870999999999997e-05 2024-04-20T01:48:27.158 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.158 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.158 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.158 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs2 55/30 ECSubWrite(tid=42, reqid=osd.2.0:42, at_version=55'26, trim_to=0'0, roll_forward_to=52'21))", 2024-04-20T01:48:27.158 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:56.731259+0000", 2024-04-20T01:48:27.158 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.403088529000001, 2024-04-20T01:48:27.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0064248020000000003, 2024-04-20T01:48:27.159 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:27.159 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.159 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:27.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:27.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731259+0000", 2024-04-20T01:48:27.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:27.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731259+0000", 2024-04-20T01:48:27.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:27.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731259+0000", 2024-04-20T01:48:27.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:27.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:27.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731283+0000", 2024-04-20T01:48:27.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4437e-05 2024-04-20T01:48:27.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:27.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731292+0000", 2024-04-20T01:48:27.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0880000000000001e-06 2024-04-20T01:48:27.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731300+0000", 2024-04-20T01:48:27.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8599999999999993e-06 2024-04-20T01:48:27.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:27.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731742+0000", 2024-04-20T01:48:27.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044220700000000002 2024-04-20T01:48:27.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731753+0000", 2024-04-20T01:48:27.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0968000000000001e-05 2024-04-20T01:48:27.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.737642+0000", 2024-04-20T01:48:27.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0058890239999999996 2024-04-20T01:48:27.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:27.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.737684+0000", 2024-04-20T01:48:27.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1683000000000001e-05 2024-04-20T01:48:27.162 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.162 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.162 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.162 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 55/30 ECSubWrite(tid=41, reqid=osd.3.0:41, at_version=55'30, trim_to=0'0, roll_forward_to=54'26))", 2024-04-20T01:48:27.163 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:56.732670+0000", 2024-04-20T01:48:27.163 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.401677518, 2024-04-20T01:48:27.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080432620000000007, 2024-04-20T01:48:27.163 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:27.163 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.163 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:27.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:27.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732670+0000", 2024-04-20T01:48:27.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:27.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732670+0000", 2024-04-20T01:48:27.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:27.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732669+0000", 2024-04-20T01:48:27.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:27.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:27.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732698+0000", 2024-04-20T01:48:27.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8818000000000001e-05 2024-04-20T01:48:27.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:27.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732711+0000", 2024-04-20T01:48:27.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2696e-05 2024-04-20T01:48:27.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732721+0000", 2024-04-20T01:48:27.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0006e-05 2024-04-20T01:48:27.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:27.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732805+0000", 2024-04-20T01:48:27.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4152000000000003e-05 2024-04-20T01:48:27.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732814+0000", 2024-04-20T01:48:27.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7110000000000007e-06 2024-04-20T01:48:27.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.740690+0000", 2024-04-20T01:48:27.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0078757670000000005 2024-04-20T01:48:27.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:27.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.740713+0000", 2024-04-20T01:48:27.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3141000000000001e-05 2024-04-20T01:48:27.166 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.166 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.166 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.167 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 55/30 ECSubWrite(tid=42, reqid=client.4643.0:451, at_version=55'31, trim_to=0'0, roll_forward_to=55'28))", 2024-04-20T01:48:27.167 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:56.739643+0000", 2024-04-20T01:48:27.167 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.394704824000002, 2024-04-20T01:48:27.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056333429999999999, 2024-04-20T01:48:27.167 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:27.167 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.167 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:27.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:27.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739643+0000", 2024-04-20T01:48:27.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:27.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739643+0000", 2024-04-20T01:48:27.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:27.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739641+0000", 2024-04-20T01:48:27.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:27.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:27.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739813+0000", 2024-04-20T01:48:27.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017258700000000001 2024-04-20T01:48:27.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:27.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739833+0000", 2024-04-20T01:48:27.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9312000000000001e-05 2024-04-20T01:48:27.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739844+0000", 2024-04-20T01:48:27.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1303e-05 2024-04-20T01:48:27.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:27.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.740000+0000", 2024-04-20T01:48:27.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015592899999999999 2024-04-20T01:48:27.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.740013+0000", 2024-04-20T01:48:27.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2659e-05 2024-04-20T01:48:27.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.745246+0000", 2024-04-20T01:48:27.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0052337290000000003 2024-04-20T01:48:27.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:27.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.745276+0000", 2024-04-20T01:48:27.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9688000000000001e-05 2024-04-20T01:48:27.170 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.170 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.171 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.171 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 58/57 ECSubWrite(tid=121, reqid=client.4643.0:485, at_version=58'81, trim_to=0'0, roll_forward_to=58'78))", 2024-04-20T01:48:27.171 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:59.819732+0000", 2024-04-20T01:48:27.171 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.314616047000001, 2024-04-20T01:48:27.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053001919999999996, 2024-04-20T01:48:27.171 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:27.171 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.171 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:27.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:27.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819732+0000", 2024-04-20T01:48:27.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:27.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819732+0000", 2024-04-20T01:48:27.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:27.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819731+0000", 2024-04-20T01:48:27.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:27.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:27.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819761+0000", 2024-04-20T01:48:27.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0620999999999997e-05 2024-04-20T01:48:27.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:27.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819775+0000", 2024-04-20T01:48:27.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4042999999999999e-05 2024-04-20T01:48:27.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819785+0000", 2024-04-20T01:48:27.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9369999999999999e-06 2024-04-20T01:48:27.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:27.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819822+0000", 2024-04-20T01:48:27.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6430000000000002e-05 2024-04-20T01:48:27.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819829+0000", 2024-04-20T01:48:27.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0299999999999996e-06 2024-04-20T01:48:27.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.825010+0000", 2024-04-20T01:48:27.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0051810900000000002 2024-04-20T01:48:27.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:27.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.825032+0000", 2024-04-20T01:48:27.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1759e-05 2024-04-20T01:48:27.174 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.175 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.175 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.175 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 60/57 ECSubWrite(tid=139, reqid=osd.8.0:138, at_version=0'0, trim_to=0'0, roll_forward_to=60'83))", 2024-04-20T01:48:27.175 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:01.785138+0000", 2024-04-20T01:48:27.175 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.349209534, 2024-04-20T01:48:27.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0054622710000000003, 2024-04-20T01:48:27.175 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:27.175 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.175 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:27.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:27.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785138+0000", 2024-04-20T01:48:27.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:27.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785138+0000", 2024-04-20T01:48:27.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:27.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785137+0000", 2024-04-20T01:48:27.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:27.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:27.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785149+0000", 2024-04-20T01:48:27.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1198e-05 2024-04-20T01:48:27.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:27.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785160+0000", 2024-04-20T01:48:27.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066e-05 2024-04-20T01:48:27.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785168+0000", 2024-04-20T01:48:27.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4880000000000007e-06 2024-04-20T01:48:27.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:27.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785217+0000", 2024-04-20T01:48:27.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8952000000000002e-05 2024-04-20T01:48:27.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785228+0000", 2024-04-20T01:48:27.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0915e-05 2024-04-20T01:48:27.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.790582+0000", 2024-04-20T01:48:27.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053536649999999996 2024-04-20T01:48:27.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:27.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.790600+0000", 2024-04-20T01:48:27.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8677000000000001e-05 2024-04-20T01:48:27.179 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.179 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.179 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.179 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 65/57 ECSubWrite(tid=165, reqid=osd.8.0:165, at_version=65'111, trim_to=0'0, roll_forward_to=65'107))", 2024-04-20T01:48:27.179 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:06.783738+0000", 2024-04-20T01:48:27.179 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 20.350609631000001, 2024-04-20T01:48:27.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053461910000000001, 2024-04-20T01:48:27.179 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:27.179 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.179 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:27.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:27.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783738+0000", 2024-04-20T01:48:27.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:27.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783738+0000", 2024-04-20T01:48:27.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:27.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783737+0000", 2024-04-20T01:48:27.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:27.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:27.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783756+0000", 2024-04-20T01:48:27.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8454e-05 2024-04-20T01:48:27.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:27.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783768+0000", 2024-04-20T01:48:27.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1950000000000001e-05 2024-04-20T01:48:27.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783777+0000", 2024-04-20T01:48:27.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0850000000000003e-06 2024-04-20T01:48:27.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:27.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783815+0000", 2024-04-20T01:48:27.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7857e-05 2024-04-20T01:48:27.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783822+0000", 2024-04-20T01:48:27.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1130000000000003e-06 2024-04-20T01:48:27.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.789064+0000", 2024-04-20T01:48:27.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0052421009999999999 2024-04-20T01:48:27.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:27.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.789084+0000", 2024-04-20T01:48:27.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0395e-05 2024-04-20T01:48:27.183 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.183 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.183 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.183 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 75/57 ECSubWrite(tid=185, reqid=osd.8.0:185, at_version=75'128, trim_to=0'0, roll_forward_to=68'123))", 2024-04-20T01:48:27.183 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.718372+0000", 2024-04-20T01:48:27.183 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.415975088, 2024-04-20T01:48:27.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.006709179, 2024-04-20T01:48:27.183 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:27.183 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.183 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:27.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:27.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718372+0000", 2024-04-20T01:48:27.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:27.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718372+0000", 2024-04-20T01:48:27.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:27.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718372+0000", 2024-04-20T01:48:27.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:27.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:27.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718398+0000", 2024-04-20T01:48:27.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5945e-05 2024-04-20T01:48:27.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:27.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718408+0000", 2024-04-20T01:48:27.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5540000000000003e-06 2024-04-20T01:48:27.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718416+0000", 2024-04-20T01:48:27.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4030000000000007e-06 2024-04-20T01:48:27.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:27.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718988+0000", 2024-04-20T01:48:27.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00057154199999999997 2024-04-20T01:48:27.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718999+0000", 2024-04-20T01:48:27.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1499e-05 2024-04-20T01:48:27.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725052+0000", 2024-04-20T01:48:27.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0060528350000000003 2024-04-20T01:48:27.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:27.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725082+0000", 2024-04-20T01:48:27.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9816999999999999e-05 2024-04-20T01:48:27.187 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.187 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.187 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.187 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs2 75/30 ECSubWrite(tid=64, reqid=osd.2.0:64, at_version=75'48, trim_to=0'0, roll_forward_to=73'46))", 2024-04-20T01:48:27.187 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.720225+0000", 2024-04-20T01:48:27.187 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.414123042, 2024-04-20T01:48:27.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0058277789999999999, 2024-04-20T01:48:27.187 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:27.187 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.188 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:27.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:27.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720225+0000", 2024-04-20T01:48:27.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:27.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720225+0000", 2024-04-20T01:48:27.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:27.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720224+0000", 2024-04-20T01:48:27.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:27.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:27.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720253+0000", 2024-04-20T01:48:27.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9485e-05 2024-04-20T01:48:27.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:27.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720280+0000", 2024-04-20T01:48:27.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.69e-05 2024-04-20T01:48:27.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720291+0000", 2024-04-20T01:48:27.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0589e-05 2024-04-20T01:48:27.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.190 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:27.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720522+0000", 2024-04-20T01:48:27.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023163799999999999 2024-04-20T01:48:27.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.190 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720532+0000", 2024-04-20T01:48:27.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7499999999999998e-06 2024-04-20T01:48:27.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.190 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726006+0000", 2024-04-20T01:48:27.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0054741770000000002 2024-04-20T01:48:27.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.191 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:27.191 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726052+0000", 2024-04-20T01:48:27.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.613e-05 2024-04-20T01:48:27.191 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.191 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.191 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.191 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs2 75/30 ECSubWrite(tid=65, reqid=osd.2.0:65, at_version=75'50, trim_to=0'0, roll_forward_to=73'46))", 2024-04-20T01:48:27.191 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.720305+0000", 2024-04-20T01:48:27.191 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.414042406, 2024-04-20T01:48:27.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0060264209999999997, 2024-04-20T01:48:27.192 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:27.192 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.192 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:27.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:27.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720305+0000", 2024-04-20T01:48:27.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:27.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720305+0000", 2024-04-20T01:48:27.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:27.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720305+0000", 2024-04-20T01:48:27.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:27.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:27.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720327+0000", 2024-04-20T01:48:27.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2181e-05 2024-04-20T01:48:27.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:27.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720335+0000", 2024-04-20T01:48:27.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7710000000000006e-06 2024-04-20T01:48:27.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720341+0000", 2024-04-20T01:48:27.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4690000000000002e-06 2024-04-20T01:48:27.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:27.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722262+0000", 2024-04-20T01:48:27.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001920843 2024-04-20T01:48:27.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722272+0000", 2024-04-20T01:48:27.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0481e-05 2024-04-20T01:48:27.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726304+0000", 2024-04-20T01:48:27.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040320979999999996 2024-04-20T01:48:27.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:27.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726332+0000", 2024-04-20T01:48:27.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7148000000000002e-05 2024-04-20T01:48:27.195 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.195 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.195 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.195 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 75/57 ECSubWrite(tid=188, reqid=osd.8.0:188, at_version=75'132, trim_to=0'0, roll_forward_to=75'128))", 2024-04-20T01:48:27.195 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.730972+0000", 2024-04-20T01:48:27.195 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.403375698, 2024-04-20T01:48:27.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059973420000000001, 2024-04-20T01:48:27.196 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:27.196 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.196 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:27.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:27.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.730972+0000", 2024-04-20T01:48:27.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:27.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.730972+0000", 2024-04-20T01:48:27.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:27.197 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.730971+0000", 2024-04-20T01:48:27.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:27.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.197 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:27.197 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731000+0000", 2024-04-20T01:48:27.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8756e-05 2024-04-20T01:48:27.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.197 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:27.197 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731009+0000", 2024-04-20T01:48:27.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6780000000000006e-06 2024-04-20T01:48:27.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.198 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731017+0000", 2024-04-20T01:48:27.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9880000000000004e-06 2024-04-20T01:48:27.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.198 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:27.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.732017+0000", 2024-04-20T01:48:27.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00099972800000000007 2024-04-20T01:48:27.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.198 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.732026+0000", 2024-04-20T01:48:27.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0299999999999999e-06 2024-04-20T01:48:27.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.199 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736942+0000", 2024-04-20T01:48:27.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0049165370000000003 2024-04-20T01:48:27.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.199 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:27.199 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736969+0000", 2024-04-20T01:48:27.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7005e-05 2024-04-20T01:48:27.199 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.199 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.199 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.199 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.199 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 75/30 ECSubWrite(tid=55, reqid=client.4643.0:781, at_version=75'41, trim_to=0'0, roll_forward_to=66'39))", 2024-04-20T01:48:27.200 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.735309+0000", 2024-04-20T01:48:27.200 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.399038587, 2024-04-20T01:48:27.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005306926, 2024-04-20T01:48:27.200 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:27.200 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.200 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:27.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:27.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735309+0000", 2024-04-20T01:48:27.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:27.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735309+0000", 2024-04-20T01:48:27.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:27.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:27.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735309+0000", 2024-04-20T01:48:27.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:27.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:27.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735445+0000", 2024-04-20T01:48:27.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013668299999999999 2024-04-20T01:48:27.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:27.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735459+0000", 2024-04-20T01:48:27.202 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3370000000000001e-05 2024-04-20T01:48:27.202 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.202 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.202 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.202 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735469+0000", 2024-04-20T01:48:27.202 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8700000000000004e-06 2024-04-20T01:48:27.202 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.202 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.202 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:27.202 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735568+0000", 2024-04-20T01:48:27.202 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9122000000000004e-05 2024-04-20T01:48:27.202 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.202 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.202 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.202 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735579+0000", 2024-04-20T01:48:27.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1294999999999999e-05 2024-04-20T01:48:27.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.203 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.203 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.740581+0000", 2024-04-20T01:48:27.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0050022410000000002 2024-04-20T01:48:27.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:27.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:27.203 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:27.203 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.740616+0000", 2024-04-20T01:48:27.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4706000000000003e-05 2024-04-20T01:48:27.203 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.203 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.203 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.204 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:27.204 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:27.204 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:27.205 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:27.219+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:27.206 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-20T01:48:27.305 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:27.320+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:27.362 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:27.362 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:27.362 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-04-20T01:48:27.362 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:27.372 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-20T01:48:27.406 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:27.421+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:27.507 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:27.521+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:27.515 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:27.515 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:27.515 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-04-20T01:48:27.515 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-04-20T01:48:27.515 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:27.527 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-20T01:48:27.608 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:27.623+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:27.673 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:27.673 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-04-20T01:48:27.673 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-04-20T01:48:27.673 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-04-20T01:48:27.673 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.673 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.7s3 36/30 ECSubWrite(tid=58, reqid=client.4643.0:211, at_version=36'8, trim_to=0'0, roll_forward_to=33'5))", 2024-04-20T01:48:27.673 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.782029+0000", 2024-04-20T01:48:27.673 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 49.904149894, 2024-04-20T01:48:27.674 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0082533060000000002, 2024-04-20T01:48:27.674 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:27.674 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.674 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:27.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.674 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:27.674 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782029+0000", 2024-04-20T01:48:27.674 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.674 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.674 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:27.674 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782029+0000", 2024-04-20T01:48:27.674 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.675 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:27.675 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782028+0000", 2024-04-20T01:48:27.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:27.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.675 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:27.675 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782164+0000", 2024-04-20T01:48:27.675 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013607300000000001 2024-04-20T01:48:27.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.675 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.675 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:27.675 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782181+0000", 2024-04-20T01:48:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6651999999999999e-05 2024-04-20T01:48:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782196+0000", 2024-04-20T01:48:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5370999999999999e-05 2024-04-20T01:48:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.783434+0000", 2024-04-20T01:48:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001237581 2024-04-20T01:48:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.676 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.783444+0000", 2024-04-20T01:48:27.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0686999999999999e-05 2024-04-20T01:48:27.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.677 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.677 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.790251+0000", 2024-04-20T01:48:27.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068071729999999997 2024-04-20T01:48:27.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.677 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:27.677 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.790283+0000", 2024-04-20T01:48:27.677 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1250000000000001e-05 2024-04-20T01:48:27.677 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.677 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.677 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.678 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs1 36/30 ECSubWrite(tid=59, reqid=client.4643.0:214, at_version=36'14, trim_to=0'0, roll_forward_to=33'12))", 2024-04-20T01:48:27.678 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.783999+0000", 2024-04-20T01:48:27.678 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 49.902180817999998, 2024-04-20T01:48:27.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011678028, 2024-04-20T01:48:27.678 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:27.678 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.678 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:27.678 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.678 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:27.678 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.783999+0000", 2024-04-20T01:48:27.678 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.679 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:27.679 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.783999+0000", 2024-04-20T01:48:27.679 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.679 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:27.679 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.783997+0000", 2024-04-20T01:48:27.679 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:27.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.679 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.679 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:27.679 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.784024+0000", 2024-04-20T01:48:27.679 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6801999999999999e-05 2024-04-20T01:48:27.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:27.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.784037+0000", 2024-04-20T01:48:27.680 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2361000000000001e-05 2024-04-20T01:48:27.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.784046+0000", 2024-04-20T01:48:27.680 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5009999999999993e-06 2024-04-20T01:48:27.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.680 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.680 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.680 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.785306+0000", 2024-04-20T01:48:27.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001259942 2024-04-20T01:48:27.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.785316+0000", 2024-04-20T01:48:27.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0309000000000001e-05 2024-04-20T01:48:27.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.681 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.795648+0000", 2024-04-20T01:48:27.681 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010331127000000001 2024-04-20T01:48:27.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.681 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.681 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:27.682 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.795677+0000", 2024-04-20T01:48:27.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9016000000000001e-05 2024-04-20T01:48:27.682 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.682 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.682 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.682 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.682 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs1 37/30 ECSubWrite(tid=64, reqid=client.4643.0:225, at_version=37'16, trim_to=0'0, roll_forward_to=36'14))", 2024-04-20T01:48:27.682 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.788088+0000", 2024-04-20T01:48:27.682 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 48.898091798000003, 2024-04-20T01:48:27.682 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010015382, 2024-04-20T01:48:27.682 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:27.682 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.682 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:27.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:27.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788088+0000", 2024-04-20T01:48:27.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:27.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788088+0000", 2024-04-20T01:48:27.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.683 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.683 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:27.683 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788087+0000", 2024-04-20T01:48:27.683 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:27.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788230+0000", 2024-04-20T01:48:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014296 2024-04-20T01:48:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788255+0000", 2024-04-20T01:48:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4790000000000002e-05 2024-04-20T01:48:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788276+0000", 2024-04-20T01:48:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0806999999999999e-05 2024-04-20T01:48:27.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.685 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.685 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791925+0000", 2024-04-20T01:48:27.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003649949 2024-04-20T01:48:27.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.685 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.685 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791937+0000", 2024-04-20T01:48:27.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1544e-05 2024-04-20T01:48:27.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.685 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.685 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.798083+0000", 2024-04-20T01:48:27.685 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0061460079999999997 2024-04-20T01:48:27.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.798103+0000", 2024-04-20T01:48:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9907000000000001e-05 2024-04-20T01:48:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.8s1 47/30 ECSubWrite(tid=40, reqid=osd.5.0:40, at_version=47'29, trim_to=0'0, roll_forward_to=46'25))", 2024-04-20T01:48:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.719174+0000", 2024-04-20T01:48:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 38.967005258, 2024-04-20T01:48:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0103924, 2024-04-20T01:48:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:27.686 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.687 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:27.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:27.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719174+0000", 2024-04-20T01:48:27.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:27.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719174+0000", 2024-04-20T01:48:27.687 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.687 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.687 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:27.687 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719173+0000", 2024-04-20T01:48:27.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:27.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:27.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719197+0000", 2024-04-20T01:48:27.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4196e-05 2024-04-20T01:48:27.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:27.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719214+0000", 2024-04-20T01:48:27.688 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6276000000000001e-05 2024-04-20T01:48:27.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.688 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.688 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.688 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719223+0000", 2024-04-20T01:48:27.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6919999999999997e-06 2024-04-20T01:48:27.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720898+0000", 2024-04-20T01:48:27.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0016744259999999999 2024-04-20T01:48:27.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.689 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720905+0000", 2024-04-20T01:48:27.689 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6660000000000008e-06 2024-04-20T01:48:27.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.689 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.689 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.729531+0000", 2024-04-20T01:48:27.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086254020000000008 2024-04-20T01:48:27.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.690 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:27.690 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.729566+0000", 2024-04-20T01:48:27.690 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5621000000000003e-05 2024-04-20T01:48:27.690 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.690 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.690 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.690 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.690 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s4 47/30 ECSubWrite(tid=80, reqid=osd.8.0:80, at_version=47'65, trim_to=0'0, roll_forward_to=44'61))", 2024-04-20T01:48:27.690 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.720292+0000", 2024-04-20T01:48:27.691 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 38.965886832999999, 2024-04-20T01:48:27.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0092900529999999995, 2024-04-20T01:48:27.691 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:27.691 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.691 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:27.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.691 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:27.691 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720292+0000", 2024-04-20T01:48:27.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.691 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.691 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:27.691 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720292+0000", 2024-04-20T01:48:27.691 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:27.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720292+0000", 2024-04-20T01:48:27.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:27.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:27.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720310+0000", 2024-04-20T01:48:27.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7717e-05 2024-04-20T01:48:27.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.692 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:27.692 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720318+0000", 2024-04-20T01:48:27.692 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3129999999999999e-06 2024-04-20T01:48:27.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720325+0000", 2024-04-20T01:48:27.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.832e-06 2024-04-20T01:48:27.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720837+0000", 2024-04-20T01:48:27.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051157600000000002 2024-04-20T01:48:27.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.693 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.693 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.693 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720846+0000", 2024-04-20T01:48:27.693 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3260000000000002e-06 2024-04-20T01:48:27.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.729552+0000", 2024-04-20T01:48:27.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0087057520000000006 2024-04-20T01:48:27.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.694 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:27.694 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.729583+0000", 2024-04-20T01:48:27.694 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0923999999999997e-05 2024-04-20T01:48:27.694 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.694 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.694 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.694 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.695 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s4 48/30 ECSubWrite(tid=85, reqid=client.4643.0:399, at_version=48'71, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:27.695 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.711859+0000", 2024-04-20T01:48:27.695 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.974319969, 2024-04-20T01:48:27.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0082489499999999997, 2024-04-20T01:48:27.695 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:27.695 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.695 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:27.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:27.695 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711859+0000", 2024-04-20T01:48:27.695 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.695 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.695 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:27.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711859+0000", 2024-04-20T01:48:27.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:27.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711858+0000", 2024-04-20T01:48:27.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:27.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:27.696 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711887+0000", 2024-04-20T01:48:27.696 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8487e-05 2024-04-20T01:48:27.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.696 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.696 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:27.697 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711908+0000", 2024-04-20T01:48:27.697 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1625e-05 2024-04-20T01:48:27.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.697 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.697 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711926+0000", 2024-04-20T01:48:27.697 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8113000000000001e-05 2024-04-20T01:48:27.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.697 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.697 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712063+0000", 2024-04-20T01:48:27.697 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000136907 2024-04-20T01:48:27.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.697 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.697 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712076+0000", 2024-04-20T01:48:27.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2145e-05 2024-04-20T01:48:27.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.720075+0000", 2024-04-20T01:48:27.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0079993730000000006 2024-04-20T01:48:27.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.698 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:27.698 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.720108+0000", 2024-04-20T01:48:27.698 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3386000000000002e-05 2024-04-20T01:48:27.698 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.698 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s4 48/30 ECSubWrite(tid=86, reqid=client.4643.0:400, at_version=48'72, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.713114+0000", 2024-04-20T01:48:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.973065707000003, 2024-04-20T01:48:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0088368690000000007, 2024-04-20T01:48:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713114+0000", 2024-04-20T01:48:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.699 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:27.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713114+0000", 2024-04-20T01:48:27.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:27.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713113+0000", 2024-04-20T01:48:27.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:27.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.700 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.700 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:27.700 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713136+0000", 2024-04-20T01:48:27.700 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2974999999999999e-05 2024-04-20T01:48:27.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:27.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713149+0000", 2024-04-20T01:48:27.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2945000000000001e-05 2024-04-20T01:48:27.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713159+0000", 2024-04-20T01:48:27.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9909999999999997e-06 2024-04-20T01:48:27.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.701 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.701 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.701 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713217+0000", 2024-04-20T01:48:27.701 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8085999999999999e-05 2024-04-20T01:48:27.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713228+0000", 2024-04-20T01:48:27.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1345999999999999e-05 2024-04-20T01:48:27.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.721915+0000", 2024-04-20T01:48:27.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086870850000000006 2024-04-20T01:48:27.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.702 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.702 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:27.702 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.721950+0000", 2024-04-20T01:48:27.702 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5352999999999998e-05 2024-04-20T01:48:27.703 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.703 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.703 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.703 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs1 52/30 ECSubWrite(tid=124, reqid=client.4643.0:434, at_version=52'55, trim_to=0'0, roll_forward_to=48'53))", 2024-04-20T01:48:27.703 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:53.736586+0000", 2024-04-20T01:48:27.703 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 33.949592828999997, 2024-04-20T01:48:27.703 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0094278839999999992, 2024-04-20T01:48:27.703 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:27.703 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.703 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:27.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.703 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:27.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736586+0000", 2024-04-20T01:48:27.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.704 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:27.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736586+0000", 2024-04-20T01:48:27.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.704 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:27.704 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736586+0000", 2024-04-20T01:48:27.704 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:27.704 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.704 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:27.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736681+0000", 2024-04-20T01:48:27.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4703000000000005e-05 2024-04-20T01:48:27.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:27.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736716+0000", 2024-04-20T01:48:27.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5434e-05 2024-04-20T01:48:27.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.705 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.705 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736737+0000", 2024-04-20T01:48:27.705 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0645999999999999e-05 2024-04-20T01:48:27.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.705 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736860+0000", 2024-04-20T01:48:27.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012254099999999999 2024-04-20T01:48:27.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736872+0000", 2024-04-20T01:48:27.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2824e-05 2024-04-20T01:48:27.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.706 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.706 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.706 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.745990+0000", 2024-04-20T01:48:27.706 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0091180250000000001 2024-04-20T01:48:27.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.707 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:27.707 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.746014+0000", 2024-04-20T01:48:27.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3983000000000001e-05 2024-04-20T01:48:27.707 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.707 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.707 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.707 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:573 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [create,setxattr POm]dpWYd`[NXWUkjTTrrjndlWhqT (15156) in=29b,setxattr ]bTTcZk (15156) in=7b,setxattr okg_granXfaiUobbllnc]Sk^[Vrj`[aRthnaRdoV (278) in=40b,setxattr gT`mrtdoaf^^ekYVVj`ZirVVqeUahfeOZdgUdQsV (382) in=40b,setxattr s\\mP`nbegjZ`iam]machaefN\\hcflgW[VmU^ik`_ (16840) in=40b,setxattr mPmpkek_rY\\scmjOek]degahQ^_SNk_epPi]p^fm (14456) in=40b,setxattr agOmkqtrc^nSastfh_^gPhplQZgnQkXRsXYbkW\\\\ (1789) in=40b,setxattr `Y[Zq^Pd_hpqld\\XRg`sTalN_ORNZ_aPUn_Nl\\ZQ (1330) in=40b,setxattr cnbkPlXlpjTSQl\\mdTVtmTO^ZjP[ra]OpgkWbsbZ (139) in=40b,setxattr Vjd]bV^O[pjq`XZai`go_goUTcrph[sfZXdaeo_d (453) in=40b,setxattr lPk`g]XVSTSfXOdjZeUQRqXTXgf]dUqZpmQ^hNh^ (841) in=40b,setxattr \\a\\rN^\\m`c^OlQbo^WerVQqiWZldlTX]n_o`^fcY (19224) in=40b,setxattr ehmtodtgcpX\\UmnfaWbtiaUQhSWb_lXn_qeXprOR (139) in=40b,setxattr VqR_VPdeQ]UeaTRh\\UfdplrbOaOSOsaSa_`pgZWX (10033) in=40b,setxattr U (15156) in=1b,setxattr `bXaUYb]oeUoqmrfqXgPlttVbU_laZ[rnYRZfRdd (418) in=40b,setxattr gPijt_qqRThf_ZmN`fYRO`UYUW^p_io^Sh\\_\\afh (1086) in=40b,setxattr WoqshhWZOap_beQ^YcYjaPktbdU[aac[`b\\d\\haa (734) in=40b,setxattr oOgQZWqrpnb_PhmYPbqgfRkbr_VjboWXsVpeUcPq (769) in=40b,setxattr t[S\\OOdRkdshlT\\lbkes]tbinQacc[aa^OYTep_d (1821) in=40b] snapc 1f=[1f,1b,1a,b] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-20T01:48:27.707 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:04.764695+0000", 2024-04-20T01:48:27.707 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 22.921484026999998, 2024-04-20T01:48:27.707 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0091856230000000004, 2024-04-20T01:48:27.707 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:27.707 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:27.708 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:27.708 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:27.708 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:27.708 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 573 2024-04-20T01:48:27.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.708 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:27.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.708 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:27.708 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.764695+0000", 2024-04-20T01:48:27.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.708 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.708 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:27.708 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.764695+0000", 2024-04-20T01:48:27.708 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:27.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.764694+0000", 2024-04-20T01:48:27.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:27.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:27.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.765516+0000", 2024-04-20T01:48:27.709 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00082247099999999997 2024-04-20T01:48:27.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.709 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.709 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:27.709 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.765528+0000", 2024-04-20T01:48:27.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2191999999999999e-05 2024-04-20T01:48:27.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.765535+0000", 2024-04-20T01:48:27.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4119999999999998e-06 2024-04-20T01:48:27.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.710 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.765582+0000", 2024-04-20T01:48:27.710 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7707999999999998e-05 2024-04-20T01:48:27.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.710 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.710 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:27.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.765695+0000", 2024-04-20T01:48:27.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000112675 2024-04-20T01:48:27.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.766572+0000", 2024-04-20T01:48:27.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00087748600000000002 2024-04-20T01:48:27.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.711 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.711 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.773424+0000", 2024-04-20T01:48:27.711 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068519499999999999 2024-04-20T01:48:27.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.711 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.712 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:27.712 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.773548+0000", 2024-04-20T01:48:27.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012316900000000001 2024-04-20T01:48:27.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.712 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:27.712 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.773881+0000", 2024-04-20T01:48:27.712 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033335299999999999 2024-04-20T01:48:27.712 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.712 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.712 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.712 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.713 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:619 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [rollback 1a] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:27.713 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.811454+0000", 2024-04-20T01:48:27.713 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 17.874725045000002, 2024-04-20T01:48:27.713 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0082693799999999998, 2024-04-20T01:48:27.713 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:27.713 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:27.713 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:27.713 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:27.713 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:27.713 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 619 2024-04-20T01:48:27.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.713 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:27.713 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.713 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:27.713 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.811454+0000", 2024-04-20T01:48:27.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:27.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.811454+0000", 2024-04-20T01:48:27.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:27.714 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.811454+0000", 2024-04-20T01:48:27.714 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:27.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.714 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.714 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:27.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.811457+0000", 2024-04-20T01:48:27.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9900000000000002e-06 2024-04-20T01:48:27.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.715 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:27.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.811464+0000", 2024-04-20T01:48:27.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8789999999999997e-06 2024-04-20T01:48:27.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.715 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.715 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.811470+0000", 2024-04-20T01:48:27.715 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-04-20T01:48:27.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.715 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.812533+0000", 2024-04-20T01:48:27.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0010629719999999999 2024-04-20T01:48:27.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:27.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.812647+0000", 2024-04-20T01:48:27.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000113911 2024-04-20T01:48:27.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.716 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.716 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.813311+0000", 2024-04-20T01:48:27.716 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00066420200000000002 2024-04-20T01:48:27.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.716 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.717 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.717 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.814350+0000", 2024-04-20T01:48:27.717 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0010393749999999999 2024-04-20T01:48:27.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.717 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:27.717 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.819649+0000", 2024-04-20T01:48:27.717 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052988430000000001 2024-04-20T01:48:27.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.717 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.717 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:27.717 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.819724+0000", 2024-04-20T01:48:27.717 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4702000000000004e-05 2024-04-20T01:48:27.717 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.717 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.718 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.718 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.718 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:641 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [append 0~507904] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:27.718 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.871779+0000", 2024-04-20T01:48:27.718 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 17.814399987000002, 2024-04-20T01:48:27.718 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0091809730000000003, 2024-04-20T01:48:27.718 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:27.718 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:27.718 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:27.718 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:27.718 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:27.718 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 641 2024-04-20T01:48:27.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.718 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:27.719 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.719 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:27.719 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.871779+0000", 2024-04-20T01:48:27.719 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.719 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.719 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:27.719 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.871779+0000", 2024-04-20T01:48:27.719 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.719 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.719 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:27.719 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.871778+0000", 2024-04-20T01:48:27.719 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:27.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.720 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.720 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:27.720 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.872406+0000", 2024-04-20T01:48:27.720 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00062772399999999997 2024-04-20T01:48:27.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.720 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.720 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:27.720 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.872415+0000", 2024-04-20T01:48:27.720 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1460000000000002e-06 2024-04-20T01:48:27.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.720 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.720 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.720 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.872422+0000", 2024-04-20T01:48:27.720 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2430000000000002e-06 2024-04-20T01:48:27.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.721 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.721 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.721 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.872471+0000", 2024-04-20T01:48:27.721 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9867999999999998e-05 2024-04-20T01:48:27.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.721 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.721 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:27.721 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.872544+0000", 2024-04-20T01:48:27.721 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2841000000000003e-05 2024-04-20T01:48:27.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.721 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.721 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.721 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.873203+0000", 2024-04-20T01:48:27.721 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00065825699999999998 2024-04-20T01:48:27.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.722 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.722 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.875894+0000", 2024-04-20T01:48:27.722 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0026911959999999999 2024-04-20T01:48:27.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.722 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:27.722 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.880878+0000", 2024-04-20T01:48:27.722 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0049847559999999999 2024-04-20T01:48:27.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.722 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.722 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:27.722 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.880960+0000", 2024-04-20T01:48:27.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1835000000000005e-05 2024-04-20T01:48:27.723 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.723 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.723 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.723 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.723 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:642 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [append 0~507904] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:27.723 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.874202+0000", 2024-04-20T01:48:27.723 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 17.811977340999999, 2024-04-20T01:48:27.723 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0093451449999999991, 2024-04-20T01:48:27.723 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:27.723 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:27.723 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:27.723 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:27.723 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:27.724 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 642 2024-04-20T01:48:27.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.724 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:27.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:27.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874202+0000", 2024-04-20T01:48:27.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:27.724 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874202+0000", 2024-04-20T01:48:27.724 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.724 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.724 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:27.725 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874201+0000", 2024-04-20T01:48:27.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:27.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.725 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:27.725 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874787+0000", 2024-04-20T01:48:27.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00058594599999999995 2024-04-20T01:48:27.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.725 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:27.725 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874796+0000", 2024-04-20T01:48:27.725 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2730000000000008e-06 2024-04-20T01:48:27.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.725 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.726 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874802+0000", 2024-04-20T01:48:27.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3619999999999997e-06 2024-04-20T01:48:27.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.726 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874835+0000", 2024-04-20T01:48:27.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3577000000000001e-05 2024-04-20T01:48:27.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.726 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.726 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:27.726 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874902+0000", 2024-04-20T01:48:27.726 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6989999999999994e-05 2024-04-20T01:48:27.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.727 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.875456+0000", 2024-04-20T01:48:27.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00055350099999999997 2024-04-20T01:48:27.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.727 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.876715+0000", 2024-04-20T01:48:27.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0012589260000000001 2024-04-20T01:48:27.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.727 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.727 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:27.727 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.883407+0000", 2024-04-20T01:48:27.727 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0066922270000000002 2024-04-20T01:48:27.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.728 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:27.728 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.883547+0000", 2024-04-20T01:48:27.728 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014018 2024-04-20T01:48:27.728 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.728 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.728 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.728 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:643 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [append 0~458752] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:27.728 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.876828+0000", 2024-04-20T01:48:27.728 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 17.809351133, 2024-04-20T01:48:27.728 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096858259999999998, 2024-04-20T01:48:27.729 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:27.729 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:27.729 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:27.729 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:27.729 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:27.729 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 643 2024-04-20T01:48:27.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.729 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:27.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:27.729 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.876828+0000", 2024-04-20T01:48:27.729 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.729 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:27.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.876828+0000", 2024-04-20T01:48:27.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:27.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.876827+0000", 2024-04-20T01:48:27.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:27.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.730 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:27.730 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877322+0000", 2024-04-20T01:48:27.730 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049457899999999998 2024-04-20T01:48:27.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.730 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.731 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:27.731 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877333+0000", 2024-04-20T01:48:27.731 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1316e-05 2024-04-20T01:48:27.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.731 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.731 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.731 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877341+0000", 2024-04-20T01:48:27.731 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9910000000000002e-06 2024-04-20T01:48:27.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.731 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.731 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.731 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877374+0000", 2024-04-20T01:48:27.731 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3173000000000002e-05 2024-04-20T01:48:27.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.731 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.732 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:27.732 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877450+0000", 2024-04-20T01:48:27.732 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5864000000000004e-05 2024-04-20T01:48:27.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.732 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.732 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.878174+0000", 2024-04-20T01:48:27.732 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00072389000000000004 2024-04-20T01:48:27.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.732 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.732 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.880693+0000", 2024-04-20T01:48:27.732 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025188430000000002 2024-04-20T01:48:27.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.732 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.733 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:27.733 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.886426+0000", 2024-04-20T01:48:27.733 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0057329540000000002 2024-04-20T01:48:27.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.733 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.733 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:27.733 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.886514+0000", 2024-04-20T01:48:27.733 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8096000000000005e-05 2024-04-20T01:48:27.733 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.733 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.733 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.733 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.733 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:644 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [append 0~81920] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:27.734 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.877354+0000", 2024-04-20T01:48:27.734 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 17.808825148, 2024-04-20T01:48:27.734 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0094386420000000006, 2024-04-20T01:48:27.734 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:27.734 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:27.734 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:27.734 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:27.734 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:27.734 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 644 2024-04-20T01:48:27.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.734 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:27.734 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.734 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:27.734 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877354+0000", 2024-04-20T01:48:27.735 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:27.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877354+0000", 2024-04-20T01:48:27.735 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:27.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877353+0000", 2024-04-20T01:48:27.735 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:27.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.735 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:27.735 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877455+0000", 2024-04-20T01:48:27.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000101229 2024-04-20T01:48:27.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.736 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:27.736 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877467+0000", 2024-04-20T01:48:27.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2061e-05 2024-04-20T01:48:27.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.736 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.736 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877474+0000", 2024-04-20T01:48:27.736 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5730000000000002e-06 2024-04-20T01:48:27.736 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.736 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.736 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.737 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.878740+0000", 2024-04-20T01:48:27.737 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001266145 2024-04-20T01:48:27.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.737 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.737 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:27.737 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.878801+0000", 2024-04-20T01:48:27.737 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0432e-05 2024-04-20T01:48:27.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.737 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.737 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.737 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.879293+0000", 2024-04-20T01:48:27.737 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049227199999999998 2024-04-20T01:48:27.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.737 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.737 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.738 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.881226+0000", 2024-04-20T01:48:27.738 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00193319 2024-04-20T01:48:27.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.738 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.738 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:27.738 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.886717+0000", 2024-04-20T01:48:27.738 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0054906019999999998 2024-04-20T01:48:27.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.738 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.738 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:27.738 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.886793+0000", 2024-04-20T01:48:27.738 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5957999999999993e-05 2024-04-20T01:48:27.738 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.738 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.739 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.739 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.739 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:645 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [setxattr _header (56) in=7b] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:27.739 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.877483+0000", 2024-04-20T01:48:27.739 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 17.808696243, 2024-04-20T01:48:27.739 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011773402000000001, 2024-04-20T01:48:27.739 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:27.739 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:27.739 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:27.739 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:27.739 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:27.739 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 645 2024-04-20T01:48:27.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.740 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:27.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.740 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:27.740 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877483+0000", 2024-04-20T01:48:27.740 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.740 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:27.740 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877483+0000", 2024-04-20T01:48:27.740 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.740 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.740 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:27.740 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877482+0000", 2024-04-20T01:48:27.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:27.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:27.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877496+0000", 2024-04-20T01:48:27.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3591000000000001e-05 2024-04-20T01:48:27.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:27.741 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877504+0000", 2024-04-20T01:48:27.741 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9880000000000004e-06 2024-04-20T01:48:27.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.741 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.741 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877510+0000", 2024-04-20T01:48:27.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1970000000000001e-06 2024-04-20T01:48:27.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.879785+0000", 2024-04-20T01:48:27.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022748339999999999 2024-04-20T01:48:27.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.742 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:27.742 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.879861+0000", 2024-04-20T01:48:27.742 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5965999999999997e-05 2024-04-20T01:48:27.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.742 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.743 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.743 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.880287+0000", 2024-04-20T01:48:27.743 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00042561299999999998 2024-04-20T01:48:27.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.743 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.743 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.881229+0000", 2024-04-20T01:48:27.743 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00094196299999999998 2024-04-20T01:48:27.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.743 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:27.743 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.889146+0000", 2024-04-20T01:48:27.743 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0079171320000000003 2024-04-20T01:48:27.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.743 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.744 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:27.744 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.889256+0000", 2024-04-20T01:48:27.744 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011071899999999999 2024-04-20T01:48:27.744 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.744 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.744 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.744 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.744 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:646 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:27.744 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.877516+0000", 2024-04-20T01:48:27.744 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 17.808663471999999, 2024-04-20T01:48:27.744 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.017708380999999999, 2024-04-20T01:48:27.744 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:27.744 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:27.745 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:27.745 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:27.745 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:27.745 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 646 2024-04-20T01:48:27.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.745 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:27.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:27.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877516+0000", 2024-04-20T01:48:27.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.745 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:27.745 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877516+0000", 2024-04-20T01:48:27.745 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.746 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:27.746 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877515+0000", 2024-04-20T01:48:27.746 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:27.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.746 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:27.746 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877518+0000", 2024-04-20T01:48:27.746 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8940000000000002e-06 2024-04-20T01:48:27.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.746 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.746 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:27.746 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877530+0000", 2024-04-20T01:48:27.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2103000000000001e-05 2024-04-20T01:48:27.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877536+0000", 2024-04-20T01:48:27.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8390000000000003e-06 2024-04-20T01:48:27.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.880487+0000", 2024-04-20T01:48:27.747 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0029510949999999999 2024-04-20T01:48:27.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.747 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.747 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:27.747 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.880557+0000", 2024-04-20T01:48:27.748 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9443999999999994e-05 2024-04-20T01:48:27.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.748 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.748 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.889586+0000", 2024-04-20T01:48:27.748 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0090290790000000006 2024-04-20T01:48:27.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.748 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:27.748 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.889691+0000", 2024-04-20T01:48:27.748 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105386 2024-04-20T01:48:27.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.748 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.748 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:27.748 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.895224+0000", 2024-04-20T01:48:27.748 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.005533044 2024-04-20T01:48:27.749 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.749 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.749 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.749 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.749 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:679 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [copy-from ver 32] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:27.749 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.962971+0000", 2024-04-20T01:48:27.749 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 17.723208539000002, 2024-04-20T01:48:27.749 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011067029000000001, 2024-04-20T01:48:27.749 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:27.749 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:27.749 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:27.749 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:27.749 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:27.750 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 679 2024-04-20T01:48:27.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.750 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:27.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:27.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.962971+0000", 2024-04-20T01:48:27.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.750 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:27.750 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.962971+0000", 2024-04-20T01:48:27.750 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.750 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:27.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.962969+0000", 2024-04-20T01:48:27.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:27.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:27.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.962976+0000", 2024-04-20T01:48:27.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9179999999999997e-06 2024-04-20T01:48:27.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.751 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.751 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:27.751 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.962987+0000", 2024-04-20T01:48:27.751 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0533000000000001e-05 2024-04-20T01:48:27.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.752 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.752 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.962993+0000", 2024-04-20T01:48:27.752 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2090000000000002e-06 2024-04-20T01:48:27.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.752 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.752 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963033+0000", 2024-04-20T01:48:27.752 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0207000000000003e-05 2024-04-20T01:48:27.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.752 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.752 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:27.752 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963106+0000", 2024-04-20T01:48:27.752 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3095000000000004e-05 2024-04-20T01:48:27.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.753 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.753 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.753 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.971796+0000", 2024-04-20T01:48:27.753 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086893639999999998 2024-04-20T01:48:27.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.753 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.753 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.753 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.972929+0000", 2024-04-20T01:48:27.753 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011335430000000001 2024-04-20T01:48:27.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.753 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.753 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:27.753 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.973994+0000", 2024-04-20T01:48:27.753 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001064923 2024-04-20T01:48:27.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.754 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:27.754 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.974038+0000", 2024-04-20T01:48:27.754 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3748000000000002e-05 2024-04-20T01:48:27.754 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.754 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.754 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.754 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.754 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:680 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:27.754 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.963000+0000", 2024-04-20T01:48:27.754 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 17.723179256000002, 2024-04-20T01:48:27.754 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0095293779999999998, 2024-04-20T01:48:27.754 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:27.755 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:27.755 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:27.755 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:27.755 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:27.755 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 680 2024-04-20T01:48:27.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.755 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:27.755 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.755 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:27.755 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963000+0000", 2024-04-20T01:48:27.755 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.755 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.755 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:27.755 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963000+0000", 2024-04-20T01:48:27.756 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.756 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:27.756 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963000+0000", 2024-04-20T01:48:27.756 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:27.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.756 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.756 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:27.756 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963002+0000", 2024-04-20T01:48:27.756 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.125e-06 2024-04-20T01:48:27.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.757 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:27.757 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963007+0000", 2024-04-20T01:48:27.757 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0529999999999997e-06 2024-04-20T01:48:27.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.757 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.757 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963010+0000", 2024-04-20T01:48:27.757 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4680000000000001e-06 2024-04-20T01:48:27.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.757 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.757 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.757 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963239+0000", 2024-04-20T01:48:27.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00022873699999999999 2024-04-20T01:48:27.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:27.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963278+0000", 2024-04-20T01:48:27.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8733000000000003e-05 2024-04-20T01:48:27.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.972372+0000", 2024-04-20T01:48:27.758 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0090944210000000001 2024-04-20T01:48:27.758 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.758 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.758 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:27.758 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.972460+0000", 2024-04-20T01:48:27.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7349999999999995e-05 2024-04-20T01:48:27.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.759 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:27.759 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.972529+0000", 2024-04-20T01:48:27.759 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9809999999999997e-05 2024-04-20T01:48:27.759 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.759 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.759 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.759 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.759 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s4 80/57 ECSubWrite(tid=206, reqid=osd.8.0:206, at_version=80'149, trim_to=0'0, roll_forward_to=79'145))", 2024-04-20T01:48:27.759 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:21.738326+0000", 2024-04-20T01:48:27.759 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.9478534339999998, 2024-04-20T01:48:27.760 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0085236089999999997, 2024-04-20T01:48:27.760 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:27.760 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.760 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:27.760 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.760 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:27.760 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.738326+0000", 2024-04-20T01:48:27.760 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.760 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.760 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.760 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:27.760 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.738326+0000", 2024-04-20T01:48:27.760 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.760 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.760 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.761 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:27.761 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.738325+0000", 2024-04-20T01:48:27.761 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:27.761 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.761 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.761 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:27.761 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.738349+0000", 2024-04-20T01:48:27.761 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4179999999999999e-05 2024-04-20T01:48:27.761 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.761 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.761 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:27.761 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.738366+0000", 2024-04-20T01:48:27.761 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6781000000000001e-05 2024-04-20T01:48:27.761 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.762 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.762 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.762 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.738377+0000", 2024-04-20T01:48:27.762 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0511e-05 2024-04-20T01:48:27.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.762 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.762 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.762 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.738474+0000", 2024-04-20T01:48:27.762 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7399999999999996e-05 2024-04-20T01:48:27.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.762 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.762 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.762 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.738486+0000", 2024-04-20T01:48:27.762 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1463e-05 2024-04-20T01:48:27.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.763 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.763 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.746818+0000", 2024-04-20T01:48:27.763 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0083327639999999994 2024-04-20T01:48:27.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.763 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:27.763 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.746849+0000", 2024-04-20T01:48:27.763 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1124999999999998e-05 2024-04-20T01:48:27.763 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.763 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.763 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.764 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.7s3 80/30 ECSubWrite(tid=308, reqid=osd.4.0:308, at_version=80'30, trim_to=0'0, roll_forward_to=79'28))", 2024-04-20T01:48:27.764 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:21.741270+0000", 2024-04-20T01:48:27.764 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 5.944909601, 2024-04-20T01:48:27.764 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086490270000000001, 2024-04-20T01:48:27.764 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:27.764 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:27.764 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:27.764 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.764 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:27.764 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.741270+0000", 2024-04-20T01:48:27.764 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.764 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.764 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:27.765 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.741270+0000", 2024-04-20T01:48:27.765 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:27.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.765 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:27.765 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.741269+0000", 2024-04-20T01:48:27.765 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:27.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.765 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:27.765 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.741305+0000", 2024-04-20T01:48:27.765 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6591000000000003e-05 2024-04-20T01:48:27.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.765 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:27.766 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.741323+0000", 2024-04-20T01:48:27.766 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7734999999999999e-05 2024-04-20T01:48:27.766 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.766 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.766 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:27.766 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.741334+0000", 2024-04-20T01:48:27.766 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0542e-05 2024-04-20T01:48:27.766 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.766 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.766 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:27.766 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.741388+0000", 2024-04-20T01:48:27.766 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4307e-05 2024-04-20T01:48:27.766 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.766 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.767 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:27.767 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.741400+0000", 2024-04-20T01:48:27.767 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2004999999999999e-05 2024-04-20T01:48:27.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.767 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.767 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:27.767 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.749892+0000", 2024-04-20T01:48:27.767 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0084922959999999999 2024-04-20T01:48:27.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:27.767 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:27.767 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:27.767 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.749919+0000", 2024-04-20T01:48:27.767 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6641000000000001e-05 2024-04-20T01:48:27.767 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.768 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.768 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.768 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:27.768 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:27.768 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:27.770 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-20T01:48:27.771 INFO:tasks.rados.rados.0.smithi165.stdout:366: done (2 left) 2024-04-20T01:48:27.771 INFO:tasks.rados.rados.0.smithi165.stdout:367: done (1 left) 2024-04-20T01:48:27.771 INFO:tasks.rados.rados.0.smithi165.stdout:368: delete oid 7 current snap is 29 2024-04-20T01:48:27.771 INFO:tasks.rados.rados.0.smithi165.stdout:368: done (1 left) 2024-04-20T01:48:27.771 INFO:tasks.rados.rados.0.smithi165.stdout:369: read oid 46 snap -1 2024-04-20T01:48:27.771 INFO:tasks.rados.rados.0.smithi165.stdout:369: expect (ObjNum 152 snap 27 seq_num 152) 2024-04-20T01:48:27.771 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 49 current snap is 29 2024-04-20T01:48:27.771 INFO:tasks.rados.rados.0.smithi165.stdout:370: seq_num 156 ranges {1998848=229376} 2024-04-20T01:48:27.771 INFO:tasks.rados.rados.0.smithi165.stdout:370: writing smithi16579500-49 from 1998848 to 2228224 tid 1 2024-04-20T01:48:27.771 INFO:tasks.rados.rados.0.smithi165.stdout:371: rollback oid 23 current snap is 29 2024-04-20T01:48:27.771 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 23 to 20 2024-04-20T01:48:27.771 INFO:tasks.rados.rados.0.smithi165.stdout:369: expect (ObjNum 152 snap 27 seq_num 152) 2024-04-20T01:48:27.772 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:27.784+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:27.803 INFO:tasks.rados.rados.0.smithi165.stdout:358: finishing write tid 1 to smithi16579500-41 2024-04-20T01:48:27.804 INFO:tasks.rados.rados.0.smithi165.stdout:358: finishing write tid 2 to smithi16579500-41 2024-04-20T01:48:27.804 INFO:tasks.rados.rados.0.smithi165.stdout:369: done (3 left) 2024-04-20T01:48:27.804 INFO:tasks.rados.rados.0.smithi165.stdout:372: snap_remove snap 26 2024-04-20T01:48:27.870 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:27.884+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:27.923 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:27.923 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:27.923 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-04-20T01:48:27.923 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:27.935 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-20T01:48:27.971 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:27.985+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:28.072 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:28.086+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:28.078 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:28.078 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:28.078 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-04-20T01:48:28.078 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-04-20T01:48:28.078 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:28.091 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-20T01:48:28.173 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:28.187+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:28.237 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-04-20T01:48:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-04-20T01:48:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-04-20T01:48:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:211 3.7s0 3:eb044288:::smithi16579500-44:head [append 0~376832] snapc 3=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:48:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.778254+0000", 2024-04-20T01:48:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 50.471867250999999, 2024-04-20T01:48:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01403427, 2024-04-20T01:48:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:28.237 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 211 2024-04-20T01:48:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778254+0000", 2024-04-20T01:48:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:28.238 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778254+0000", 2024-04-20T01:48:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778252+0000", 2024-04-20T01:48:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780895+0000", 2024-04-20T01:48:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002642658 2024-04-20T01:48:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:28.239 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780910+0000", 2024-04-20T01:48:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4949999999999999e-05 2024-04-20T01:48:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780919+0000", 2024-04-20T01:48:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7770000000000008e-06 2024-04-20T01:48:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780949+0000", 2024-04-20T01:48:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0639999999999998e-05 2024-04-20T01:48:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.240 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781077+0000", 2024-04-20T01:48:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000127565 2024-04-20T01:48:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781928+0000", 2024-04-20T01:48:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00085169599999999999 2024-04-20T01:48:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.787994+0000", 2024-04-20T01:48:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00606545 2024-04-20T01:48:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.792213+0000", 2024-04-20T01:48:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042187199999999996 2024-04-20T01:48:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.792289+0000", 2024-04-20T01:48:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5980000000000001e-05 2024-04-20T01:48:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.242 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:213 3.7s0 3:eb044288:::smithi16579500-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:48:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.780960+0000", 2024-04-20T01:48:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 50.46916117, 2024-04-20T01:48:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015313724000000001, 2024-04-20T01:48:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 213 2024-04-20T01:48:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780960+0000", 2024-04-20T01:48:28.243 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780960+0000", 2024-04-20T01:48:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780960+0000", 2024-04-20T01:48:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:28.244 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780964+0000", 2024-04-20T01:48:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6610000000000002e-06 2024-04-20T01:48:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780971+0000", 2024-04-20T01:48:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9270000000000005e-06 2024-04-20T01:48:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780978+0000", 2024-04-20T01:48:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1670000000000004e-06 2024-04-20T01:48:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.245 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.786222+0000", 2024-04-20T01:48:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052446979999999999 2024-04-20T01:48:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.786324+0000", 2024-04-20T01:48:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000101801 2024-04-20T01:48:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.793221+0000", 2024-04-20T01:48:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068970510000000004 2024-04-20T01:48:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.793323+0000", 2024-04-20T01:48:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102289 2024-04-20T01:48:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.796274+0000", 2024-04-20T01:48:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0029506409999999999 2024-04-20T01:48:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.247 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:214 3.bs0 3:dc0bbe39:::smithi16579500-13:head [stat,delete] snapc 3=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:48:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.781222+0000", 2024-04-20T01:48:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 50.468899723, 2024-04-20T01:48:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015584361, 2024-04-20T01:48:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 214 2024-04-20T01:48:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781222+0000", 2024-04-20T01:48:28.248 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781222+0000", 2024-04-20T01:48:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781221+0000", 2024-04-20T01:48:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781228+0000", 2024-04-20T01:48:28.249 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6699999999999997e-06 2024-04-20T01:48:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781239+0000", 2024-04-20T01:48:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1574e-05 2024-04-20T01:48:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781247+0000", 2024-04-20T01:48:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.593e-06 2024-04-20T01:48:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.250 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781288+0000", 2024-04-20T01:48:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1253e-05 2024-04-20T01:48:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781383+0000", 2024-04-20T01:48:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4788000000000001e-05 2024-04-20T01:48:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782250+0000", 2024-04-20T01:48:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00086664399999999996 2024-04-20T01:48:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.251 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.786611+0000", 2024-04-20T01:48:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004361714 2024-04-20T01:48:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.796739+0000", 2024-04-20T01:48:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010127519 2024-04-20T01:48:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.796806+0000", 2024-04-20T01:48:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7414000000000002e-05 2024-04-20T01:48:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.252 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:225 3.bs0 3:df90e529:::smithi16579500-45:head [append 0~622592] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.785410+0000", 2024-04-20T01:48:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 49.464711211999997, 2024-04-20T01:48:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013791425, 2024-04-20T01:48:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 225 2024-04-20T01:48:28.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.785410+0000", 2024-04-20T01:48:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.785410+0000", 2024-04-20T01:48:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.785407+0000", 2024-04-20T01:48:28.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999967 2024-04-20T01:48:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.786402+0000", 2024-04-20T01:48:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00099532000000000006 2024-04-20T01:48:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.786427+0000", 2024-04-20T01:48:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5304e-05 2024-04-20T01:48:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.786440+0000", 2024-04-20T01:48:28.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2238999999999999e-05 2024-04-20T01:48:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.786515+0000", 2024-04-20T01:48:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4923000000000001e-05 2024-04-20T01:48:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.786652+0000", 2024-04-20T01:48:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137522 2024-04-20T01:48:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.256 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788170+0000", 2024-04-20T01:48:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001518003 2024-04-20T01:48:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796858+0000", 2024-04-20T01:48:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086877320000000001 2024-04-20T01:48:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.799118+0000", 2024-04-20T01:48:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022599600000000001 2024-04-20T01:48:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.799202+0000", 2024-04-20T01:48:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3954000000000003e-05 2024-04-20T01:48:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:227 3.bs0 3:df90e529:::smithi16579500-45:head [setxattr _header (56) in=7b] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787843+0000", 2024-04-20T01:48:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 49.462278935000001, 2024-04-20T01:48:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014686189000000001, 2024-04-20T01:48:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:28.258 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 227 2024-04-20T01:48:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787843+0000", 2024-04-20T01:48:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787843+0000", 2024-04-20T01:48:28.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787842+0000", 2024-04-20T01:48:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787847+0000", 2024-04-20T01:48:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8130000000000001e-06 2024-04-20T01:48:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787853+0000", 2024-04-20T01:48:28.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5119999999999999e-06 2024-04-20T01:48:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787859+0000", 2024-04-20T01:48:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8119999999999996e-06 2024-04-20T01:48:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.790718+0000", 2024-04-20T01:48:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002859267 2024-04-20T01:48:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.790827+0000", 2024-04-20T01:48:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000109471 2024-04-20T01:48:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791236+0000", 2024-04-20T01:48:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000408637 2024-04-20T01:48:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796922+0000", 2024-04-20T01:48:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0056861139999999999 2024-04-20T01:48:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:28.262 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.802464+0000", 2024-04-20T01:48:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055420449999999998 2024-04-20T01:48:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.802529+0000", 2024-04-20T01:48:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4554000000000006e-05 2024-04-20T01:48:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:228 3.bs0 3:df90e529:::smithi16579500-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787871+0000", 2024-04-20T01:48:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 49.462250220000001, 2024-04-20T01:48:28.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.016315307000000001, 2024-04-20T01:48:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 228 2024-04-20T01:48:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787871+0000", 2024-04-20T01:48:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787871+0000", 2024-04-20T01:48:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787871+0000", 2024-04-20T01:48:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787873+0000", 2024-04-20T01:48:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.08e-06 2024-04-20T01:48:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787878+0000", 2024-04-20T01:48:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7299999999999996e-06 2024-04-20T01:48:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787881+0000", 2024-04-20T01:48:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5209999999999998e-06 2024-04-20T01:48:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791461+0000", 2024-04-20T01:48:28.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003579905 2024-04-20T01:48:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791554+0000", 2024-04-20T01:48:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2485000000000006e-05 2024-04-20T01:48:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.802822+0000", 2024-04-20T01:48:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011268251 2024-04-20T01:48:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.802922+0000", 2024-04-20T01:48:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010029700000000001 2024-04-20T01:48:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.804187+0000", 2024-04-20T01:48:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001264362 2024-04-20T01:48:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.268 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:385 3.1s0 3:8bb12819:::smithi16579500-21:head [copy-from ver 2] snapc d=[c,b] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-20T01:48:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.710205+0000", 2024-04-20T01:48:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 39.539916169000001, 2024-04-20T01:48:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.016348672000000002, 2024-04-20T01:48:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 385 2024-04-20T01:48:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.269 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710205+0000", 2024-04-20T01:48:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710205+0000", 2024-04-20T01:48:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710204+0000", 2024-04-20T01:48:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710212+0000", 2024-04-20T01:48:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8250000000000005e-06 2024-04-20T01:48:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710222+0000", 2024-04-20T01:48:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0071000000000001e-05 2024-04-20T01:48:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710232+0000", 2024-04-20T01:48:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0583e-05 2024-04-20T01:48:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710273+0000", 2024-04-20T01:48:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0407999999999997e-05 2024-04-20T01:48:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710360+0000", 2024-04-20T01:48:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6778000000000002e-05 2024-04-20T01:48:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719733+0000", 2024-04-20T01:48:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0093733800000000006 2024-04-20T01:48:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.723689+0000", 2024-04-20T01:48:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0039562620000000003 2024-04-20T01:48:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.726462+0000", 2024-04-20T01:48:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027724989999999999 2024-04-20T01:48:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.726554+0000", 2024-04-20T01:48:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2262000000000005e-05 2024-04-20T01:48:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.273 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s2 48/30 ECSubWrite(tid=85, reqid=client.4643.0:399, at_version=48'71, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.711991+0000", 2024-04-20T01:48:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 38.538130512999999, 2024-04-20T01:48:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013597662, 2024-04-20T01:48:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711991+0000", 2024-04-20T01:48:28.274 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711991+0000", 2024-04-20T01:48:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711990+0000", 2024-04-20T01:48:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712015+0000", 2024-04-20T01:48:28.275 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4516999999999999e-05 2024-04-20T01:48:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712032+0000", 2024-04-20T01:48:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7425999999999999e-05 2024-04-20T01:48:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712042+0000", 2024-04-20T01:48:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9000000000000001e-06 2024-04-20T01:48:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713222+0000", 2024-04-20T01:48:28.276 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001179776 2024-04-20T01:48:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713248+0000", 2024-04-20T01:48:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5990999999999999e-05 2024-04-20T01:48:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.725562+0000", 2024-04-20T01:48:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01231369 2024-04-20T01:48:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.725589+0000", 2024-04-20T01:48:28.277 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7137000000000001e-05 2024-04-20T01:48:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s2 48/30 ECSubWrite(tid=86, reqid=client.4643.0:400, at_version=48'72, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.713256+0000", 2024-04-20T01:48:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 38.536865421000002, 2024-04-20T01:48:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.017021191000000001, 2024-04-20T01:48:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:28.278 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713256+0000", 2024-04-20T01:48:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713256+0000", 2024-04-20T01:48:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713255+0000", 2024-04-20T01:48:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.279 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713277+0000", 2024-04-20T01:48:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1348e-05 2024-04-20T01:48:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713289+0000", 2024-04-20T01:48:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2692e-05 2024-04-20T01:48:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713299+0000", 2024-04-20T01:48:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0071999999999999e-05 2024-04-20T01:48:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.280 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714467+0000", 2024-04-20T01:48:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001167581 2024-04-20T01:48:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714478+0000", 2024-04-20T01:48:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1029999999999999e-05 2024-04-20T01:48:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730237+0000", 2024-04-20T01:48:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015758635 2024-04-20T01:48:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.281 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730277+0000", 2024-04-20T01:48:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0788999999999999e-05 2024-04-20T01:48:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s2 48/30 ECSubWrite(tid=87, reqid=client.4643.0:401, at_version=48'73, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.714498+0000", 2024-04-20T01:48:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 38.535623827999999, 2024-04-20T01:48:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015921002, 2024-04-20T01:48:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:28.282 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714498+0000", 2024-04-20T01:48:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714498+0000", 2024-04-20T01:48:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714497+0000", 2024-04-20T01:48:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:28.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714519+0000", 2024-04-20T01:48:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2574000000000001e-05 2024-04-20T01:48:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714531+0000", 2024-04-20T01:48:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1361e-05 2024-04-20T01:48:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714540+0000", 2024-04-20T01:48:28.284 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4769999999999992e-06 2024-04-20T01:48:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.716648+0000", 2024-04-20T01:48:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002108183 2024-04-20T01:48:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.285 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.716661+0000", 2024-04-20T01:48:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2490000000000001e-05 2024-04-20T01:48:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730395+0000", 2024-04-20T01:48:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013734333 2024-04-20T01:48:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.286 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730419+0000", 2024-04-20T01:48:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3489000000000002e-05 2024-04-20T01:48:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.3s2 48/30 ECSubWrite(tid=254, reqid=osd.13.0:251, at_version=0'0, trim_to=0'0, roll_forward_to=48'22))", 2024-04-20T01:48:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.719206+0000", 2024-04-20T01:48:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 38.530915329000003, 2024-04-20T01:48:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014281428000000001, 2024-04-20T01:48:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:28.287 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719206+0000", 2024-04-20T01:48:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719206+0000", 2024-04-20T01:48:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719206+0000", 2024-04-20T01:48:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.288 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719215+0000", 2024-04-20T01:48:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6229999999999992e-06 2024-04-20T01:48:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719237+0000", 2024-04-20T01:48:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1877000000000001e-05 2024-04-20T01:48:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719249+0000", 2024-04-20T01:48:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1736000000000001e-05 2024-04-20T01:48:28.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.726201+0000", 2024-04-20T01:48:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0069514930000000004 2024-04-20T01:48:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.726206+0000", 2024-04-20T01:48:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7830000000000004e-06 2024-04-20T01:48:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.733466+0000", 2024-04-20T01:48:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072593550000000003 2024-04-20T01:48:28.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.733488+0000", 2024-04-20T01:48:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2007000000000001e-05 2024-04-20T01:48:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:493 3.1s0 3:8bb12819:::smithi16579500-21:head [copy-from ver 27] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.773406+0000", 2024-04-20T01:48:28.291 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 28.476715917, 2024-04-20T01:48:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014247365999999999, 2024-04-20T01:48:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 493 2024-04-20T01:48:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773406+0000", 2024-04-20T01:48:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.292 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773406+0000", 2024-04-20T01:48:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773404+0000", 2024-04-20T01:48:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773412+0000", 2024-04-20T01:48:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7309999999999993e-06 2024-04-20T01:48:28.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773427+0000", 2024-04-20T01:48:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5167999999999999e-05 2024-04-20T01:48:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773435+0000", 2024-04-20T01:48:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4710000000000007e-06 2024-04-20T01:48:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773496+0000", 2024-04-20T01:48:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0325000000000003e-05 2024-04-20T01:48:28.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op must wait for map", 2024-04-20T01:48:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773506+0000", 2024-04-20T01:48:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9739999999999997e-06 2024-04-20T01:48:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.775199+0000", 2024-04-20T01:48:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001693214 2024-04-20T01:48:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.775299+0000", 2024-04-20T01:48:28.295 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000100079 2024-04-20T01:48:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.782700+0000", 2024-04-20T01:48:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0074012640000000003 2024-04-20T01:48:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.787277+0000", 2024-04-20T01:48:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045765349999999996 2024-04-20T01:48:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.787341+0000", 2024-04-20T01:48:28.296 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3998000000000006e-05 2024-04-20T01:48:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.787653+0000", 2024-04-20T01:48:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00031239000000000001 2024-04-20T01:48:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:584 3.bs0 3:d86ca78e:::smithi16579500-26:1f [read 0~4816896,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-04-20T01:48:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:05.758316+0000", 2024-04-20T01:48:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 22.491805347, 2024-04-20T01:48:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015613514, 2024-04-20T01:48:28.297 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 584 2024-04-20T01:48:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.758316+0000", 2024-04-20T01:48:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.298 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.758316+0000", 2024-04-20T01:48:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.758316+0000", 2024-04-20T01:48:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.758321+0000", 2024-04-20T01:48:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9099999999999996e-06 2024-04-20T01:48:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.299 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.758328+0000", 2024-04-20T01:48:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4619999999999999e-06 2024-04-20T01:48:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.758334+0000", 2024-04-20T01:48:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7960000000000001e-06 2024-04-20T01:48:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.759436+0000", 2024-04-20T01:48:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011024730000000001 2024-04-20T01:48:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.300 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.759539+0000", 2024-04-20T01:48:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102896 2024-04-20T01:48:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.773930+0000", 2024-04-20T01:48:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014390464 2024-04-20T01:48:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:650 3.bs0 3:da81107c:::smithi16579500-34:head [copy-from ver 38] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.877476+0000", 2024-04-20T01:48:28.301 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.372646060000001, 2024-04-20T01:48:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014109823000000001, 2024-04-20T01:48:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 650 2024-04-20T01:48:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877476+0000", 2024-04-20T01:48:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877476+0000", 2024-04-20T01:48:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877470+0000", 2024-04-20T01:48:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999943 2024-04-20T01:48:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877485+0000", 2024-04-20T01:48:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4973e-05 2024-04-20T01:48:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.303 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877509+0000", 2024-04-20T01:48:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4056000000000001e-05 2024-04-20T01:48:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877521+0000", 2024-04-20T01:48:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2462e-05 2024-04-20T01:48:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877587+0000", 2024-04-20T01:48:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6008999999999998e-05 2024-04-20T01:48:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.304 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877709+0000", 2024-04-20T01:48:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000121537 2024-04-20T01:48:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.887306+0000", 2024-04-20T01:48:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0095964940000000006 2024-04-20T01:48:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.889750+0000", 2024-04-20T01:48:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0024441659999999998 2024-04-20T01:48:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.305 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.891529+0000", 2024-04-20T01:48:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001778916 2024-04-20T01:48:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.891585+0000", 2024-04-20T01:48:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6725999999999999e-05 2024-04-20T01:48:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.306 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:715 3.bs0 3:dc0bbe39:::smithi16579500-13:head [copy-from ver 32] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:10.113717+0000", 2024-04-20T01:48:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.136404362, 2024-04-20T01:48:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.020694802000000002, 2024-04-20T01:48:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 715 2024-04-20T01:48:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113717+0000", 2024-04-20T01:48:28.307 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113717+0000", 2024-04-20T01:48:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113715+0000", 2024-04-20T01:48:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113731+0000", 2024-04-20T01:48:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6231000000000001e-05 2024-04-20T01:48:28.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113751+0000", 2024-04-20T01:48:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9666000000000001e-05 2024-04-20T01:48:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113762+0000", 2024-04-20T01:48:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0596e-05 2024-04-20T01:48:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113837+0000", 2024-04-20T01:48:28.309 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5783000000000003e-05 2024-04-20T01:48:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113990+0000", 2024-04-20T01:48:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000152118 2024-04-20T01:48:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.127616+0000", 2024-04-20T01:48:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013625906 2024-04-20T01:48:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.129856+0000", 2024-04-20T01:48:28.310 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022408490000000001 2024-04-20T01:48:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.134361+0000", 2024-04-20T01:48:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045043349999999999 2024-04-20T01:48:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.134412+0000", 2024-04-20T01:48:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1310999999999997e-05 2024-04-20T01:48:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.311 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:716 3.bs0 3:dc0bbe39:::smithi16579500-13:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:10.113774+0000", 2024-04-20T01:48:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.136347917999998, 2024-04-20T01:48:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015615891, 2024-04-20T01:48:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 716 2024-04-20T01:48:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.312 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113774+0000", 2024-04-20T01:48:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113774+0000", 2024-04-20T01:48:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113773+0000", 2024-04-20T01:48:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.313 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113776+0000", 2024-04-20T01:48:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.61e-06 2024-04-20T01:48:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113781+0000", 2024-04-20T01:48:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6369999999999998e-06 2024-04-20T01:48:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113784+0000", 2024-04-20T01:48:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4489999999999999e-06 2024-04-20T01:48:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.314 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.114218+0000", 2024-04-20T01:48:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00043373499999999998 2024-04-20T01:48:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.114253+0000", 2024-04-20T01:48:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5410000000000001e-05 2024-04-20T01:48:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.129136+0000", 2024-04-20T01:48:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014882532 2024-04-20T01:48:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.315 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.129263+0000", 2024-04-20T01:48:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000126919 2024-04-20T01:48:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.129390+0000", 2024-04-20T01:48:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000126957 2024-04-20T01:48:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:720 3.7s0 3:e950587e:::smithi16579500-49:head [copy-from ver 100] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:10.113830+0000", 2024-04-20T01:48:28.316 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.136291862, 2024-04-20T01:48:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.016212712000000001, 2024-04-20T01:48:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 720 2024-04-20T01:48:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113830+0000", 2024-04-20T01:48:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.317 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113830+0000", 2024-04-20T01:48:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113829+0000", 2024-04-20T01:48:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113833+0000", 2024-04-20T01:48:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2239999999999997e-06 2024-04-20T01:48:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.318 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113843+0000", 2024-04-20T01:48:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6220000000000004e-06 2024-04-20T01:48:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113847+0000", 2024-04-20T01:48:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4150000000000001e-06 2024-04-20T01:48:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113890+0000", 2024-04-20T01:48:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2287999999999999e-05 2024-04-20T01:48:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.319 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113987+0000", 2024-04-20T01:48:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7149000000000001e-05 2024-04-20T01:48:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.124314+0000", 2024-04-20T01:48:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010327325 2024-04-20T01:48:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.128903+0000", 2024-04-20T01:48:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045891949999999999 2024-04-20T01:48:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.320 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.129948+0000", 2024-04-20T01:48:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0010448409999999999 2024-04-20T01:48:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.130042+0000", 2024-04-20T01:48:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4158000000000002e-05 2024-04-20T01:48:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.321 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:723 3.7s0 3:eb044288:::smithi16579500-44:head [rollback 1a] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:10.113898+0000", 2024-04-20T01:48:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.136223813000001, 2024-04-20T01:48:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013284504000000001, 2024-04-20T01:48:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 723 2024-04-20T01:48:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:28.322 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113898+0000", 2024-04-20T01:48:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113898+0000", 2024-04-20T01:48:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113898+0000", 2024-04-20T01:48:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967296 2024-04-20T01:48:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:28.323 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113900+0000", 2024-04-20T01:48:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1569999999999998e-06 2024-04-20T01:48:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113904+0000", 2024-04-20T01:48:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4039999999999997e-06 2024-04-20T01:48:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113907+0000", 2024-04-20T01:48:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.943e-06 2024-04-20T01:48:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.324 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.115280+0000", 2024-04-20T01:48:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0013727259999999999 2024-04-20T01:48:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.115377+0000", 2024-04-20T01:48:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7275000000000006e-05 2024-04-20T01:48:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.116058+0000", 2024-04-20T01:48:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00068105100000000001 2024-04-20T01:48:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:28.325 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.126887+0000", 2024-04-20T01:48:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010828723 2024-04-20T01:48:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.127106+0000", 2024-04-20T01:48:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00021916000000000001 2024-04-20T01:48:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.127182+0000", 2024-04-20T01:48:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6301e-05 2024-04-20T01:48:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:724 3.7s0 3:eb044288:::smithi16579500-44:head [append 0~0] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:10.113910+0000", 2024-04-20T01:48:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 18.136211741, 2024-04-20T01:48:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013446279, 2024-04-20T01:48:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 724 2024-04-20T01:48:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.327 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113910+0000", 2024-04-20T01:48:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113910+0000", 2024-04-20T01:48:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113909+0000", 2024-04-20T01:48:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.328 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:28.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113911+0000", 2024-04-20T01:48:28.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.784e-06 2024-04-20T01:48:28.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:28.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113914+0000", 2024-04-20T01:48:28.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1690000000000001e-06 2024-04-20T01:48:28.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.329 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:28.329 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113919+0000", 2024-04-20T01:48:28.329 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.053e-06 2024-04-20T01:48:28.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.329 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.116676+0000", 2024-04-20T01:48:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027574240000000001 2024-04-20T01:48:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.116803+0000", 2024-04-20T01:48:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000126654 2024-04-20T01:48:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.117277+0000", 2024-04-20T01:48:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00047413799999999998 2024-04-20T01:48:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.330 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:28.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.126888+0000", 2024-04-20T01:48:28.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096116610000000005 2024-04-20T01:48:28.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:28.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.127280+0000", 2024-04-20T01:48:28.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000391264 2024-04-20T01:48:28.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.331 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.331 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:28.331 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.127356+0000", 2024-04-20T01:48:28.331 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6470999999999993e-05 2024-04-20T01:48:28.331 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:783 3.bs0 3:d8d507c1:::smithi16579500-31:head [create,setxattr ]pZOqXa[dZ]SgadRhTYkaVmms]loYXV]OhjYgpQb (558) in=40b,setxattr tY^OWiahkbeO^^QXeQkskkmiqmg]^idV]]Wa (945) in=36b,setxattr ddsnSab`pWQQbVa^PY_ZP\\YZWq`iW]YPicUPUh (945) in=38b,setxattr f]\\heUc_VXgVXWYScTg_cOpgRekOSNq (945) in=31b,setxattr acUl[g`Obg (945) in=10b,setxattr OfSUabmdN^X_rZU`aai`O]e^cSeoSjpUR]obY_Y^ (17824) in=40b,setxattr ^Zotgjmkt]eXjWi\\Tsd]sWTiUkT\\r_diXa^ecsVT (557) in=40b,setxattr qUenNgU\\cb`[WW_fr]XX]kX[cPqhYt\\YtUSfs]T[ (1227) in=40b,setxattr SNqcNPWmeZRq_\\cnZRprZ_QZ]Pnnga_fpsVl (945) in=36b,setxattr _lqa`PaWRk\\gjqtki]amP[Uj]Pp\\T]TkSrdd]gai (771) in=40b,setxattr _S]YVmre[VNi]bcbZ[Rrgql (945) in=23b,setxattr pWktdcOsRSfRQ[_p^[csnUlhN`tgS]imf\\[`oleX (1015) in=40b,setxattr kcaO\\qSco_ObZP^qWjYlY[a]bpYRO`t\\cnQZNqsS (877) in=40b,setxattr ]lcnY\\pohsUfNjfgp`jb]^oXfRta__l`jjOY`iOk (174) in=40b,setxattr T`NOae[on`a_soaoWQ_irbZpdd_^NT]aVpXTr^fa (1438) in=40b,setxattr `aheahaRqeqg_YgcYglVNh\\fkl`gd]opppUSOR[e (11996) in=40b,setxattr n[QWooeNtePf_\\]^TpS^_YmQgQh]dmQSWqUVNrl^ (243) in=40b,setxattr o_hhpPi[PnXZtoj\\Q] (945) in=18b,setxattr PiUqpcbjVsW_[lgpZXliUgOpPRas\\\\a[TZQm[onO (487) in=40b,setxattr gjp\\eY`ptqelkrVpfW[r^ilVhqmpc[^Un\\^Tgfhq (1610) in=40b] snapc 2a=[2a,28] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-04-20T01:48:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:16.724535+0000", 2024-04-20T01:48:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 11.525586323000001, 2024-04-20T01:48:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.026026878999999999, 2024-04-20T01:48:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 783 2024-04-20T01:48:28.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.724535+0000", 2024-04-20T01:48:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.724535+0000", 2024-04-20T01:48:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.724533+0000", 2024-04-20T01:48:28.333 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:28.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:28.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.725134+0000", 2024-04-20T01:48:28.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00060092299999999995 2024-04-20T01:48:28.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:28.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.725152+0000", 2024-04-20T01:48:28.334 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8297000000000001e-05 2024-04-20T01:48:28.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.334 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.334 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:28.334 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.725164+0000", 2024-04-20T01:48:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.204e-05 2024-04-20T01:48:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.729187+0000", 2024-04-20T01:48:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0040223999999999998 2024-04-20T01:48:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.729293+0000", 2024-04-20T01:48:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000106703 2024-04-20T01:48:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.730663+0000", 2024-04-20T01:48:28.335 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0013691300000000001 2024-04-20T01:48:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.734534+0000", 2024-04-20T01:48:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0038714470000000001 2024-04-20T01:48:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.750462+0000", 2024-04-20T01:48:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015927802000000001 2024-04-20T01:48:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.750562+0000", 2024-04-20T01:48:28.336 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010027900000000001 2024-04-20T01:48:28.337 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.337 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.337 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.337 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:28.337 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:28.337 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:28.339 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-20T01:48:28.339 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:28.352+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:28.347 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-20T01:48:28.439 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:28.454+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:28.534 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:28.534 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:28.534 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-04-20T01:48:28.534 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:28.543 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:28.557+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:28.546 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-20T01:48:28.551 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:48:28.573 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:48:28.594 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:48:28.616 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:48:28.636 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:48:28.644 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:48:28.659+0000 7f9dfad7b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 43105) UID: 0 2024-04-20T01:48:28.657 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:48:28.678 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:48:28.704 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:28.704 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:28.704 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-04-20T01:48:28.704 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-04-20T01:48:28.704 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:28.718 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:48:28.720 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-20T01:48:28.738 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:48:28.745 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:28.760+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:28.753 INFO:tasks.rados.rados.0.smithi165.stdout:372: done (3 left) 2024-04-20T01:48:28.753 INFO:tasks.rados.rados.0.smithi165.stdout:373: rmattr oid 36 current snap is 29 2024-04-20T01:48:28.753 INFO:tasks.rados.rados.0.smithi165.stdout:373: done (3 left) 2024-04-20T01:48:28.753 INFO:tasks.rados.rados.0.smithi165.stdout:374: read oid 20 snap -1 2024-04-20T01:48:28.753 INFO:tasks.rados.rados.0.smithi165.stdout:374: expect (ObjNum 143 snap 23 seq_num 143) 2024-04-20T01:48:28.753 INFO:tasks.rados.rados.0.smithi165.stdout:375: delete oid 34 current snap is 29 2024-04-20T01:48:28.753 INFO:tasks.rados.rados.0.smithi165.stdout:358: finishing write tid 3 to smithi16579500-41 2024-04-20T01:48:28.753 INFO:tasks.rados.rados.0.smithi165.stdout:358: oid 41 updating version 0 to 159 2024-04-20T01:48:28.753 INFO:tasks.rados.rados.0.smithi165.stdout:358: oid 41 updating version 159 to 160 2024-04-20T01:48:28.753 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 41 v 160 (ObjNum 150 snap 27 seq_num 150) dirty exists 2024-04-20T01:48:28.754 INFO:tasks.rados.rados.0.smithi165.stdout:358: left oid 41 (ObjNum 150 snap 27 seq_num 150) 2024-04-20T01:48:28.754 INFO:tasks.rados.rados.0.smithi165.stdout:371: finishing rollback tid 0 to smithi16579500-23 2024-04-20T01:48:28.754 INFO:tasks.rados.rados.0.smithi165.stdout:371: finishing rollback tid 1 to smithi16579500-23 2024-04-20T01:48:28.754 INFO:tasks.rados.rados.0.smithi165.stdout:371: finishing rollback tid 2 to smithi16579500-23 2024-04-20T01:48:28.754 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 23 v 108 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-04-20T01:48:28.754 INFO:tasks.rados.rados.0.smithi165.stdout:370: finishing write tid 1 to smithi16579500-49 2024-04-20T01:48:28.754 INFO:tasks.rados.rados.0.smithi165.stdout:370: finishing write tid 2 to smithi16579500-49 2024-04-20T01:48:28.754 INFO:tasks.rados.rados.0.smithi165.stdout:370: finishing write tid 3 to smithi16579500-49 2024-04-20T01:48:28.754 INFO:tasks.rados.rados.0.smithi165.stdout:370: oid 49 updating version 0 to 105 2024-04-20T01:48:28.754 INFO:tasks.rados.rados.0.smithi165.stdout:370: oid 49 version 105 is already newer than 104 2024-04-20T01:48:28.754 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 49 v 105 (ObjNum 156 snap 29 seq_num 156) dirty exists 2024-04-20T01:48:28.754 INFO:tasks.rados.rados.0.smithi165.stdout:370: left oid 49 (ObjNum 156 snap 29 seq_num 156) 2024-04-20T01:48:28.761 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-20T01:48:28.769 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:48:28.777 INFO:tasks.rados.rados.0.smithi165.stdout:358: done (4 left) 2024-04-20T01:48:28.777 INFO:tasks.rados.rados.0.smithi165.stdout:370: done (3 left) 2024-04-20T01:48:28.777 INFO:tasks.rados.rados.0.smithi165.stdout:371: done (2 left) 2024-04-20T01:48:28.777 INFO:tasks.rados.rados.0.smithi165.stdout:375: done (1 left) 2024-04-20T01:48:28.778 INFO:tasks.rados.rados.0.smithi165.stdout:376: read oid 40 snap 28 2024-04-20T01:48:28.778 INFO:tasks.rados.rados.0.smithi165.stdout:376: expect (ObjNum 102 snap 16 seq_num 102) 2024-04-20T01:48:28.778 INFO:tasks.rados.rados.0.smithi165.stdout:377: snap_create 2024-04-20T01:48:28.781 INFO:tasks.rados.rados.0.smithi165.stdout:374: expect (ObjNum 143 snap 23 seq_num 143) 2024-04-20T01:48:28.798 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:48:28.819 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:48:28.847 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:28.861+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:28.855 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:48:28.876 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:48:28.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-20T01:48:28.890+0000 7fbff1d9b700 0 -- 172.21.15.44:0/4128580882 >> [v2:172.21.15.123:6824/43110,v1:172.21.15.123:6825/43110] conn(0x7fbff421f040 msgr2=0x7fbff4221430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-20T01:48:28.903 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:28.903 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:28.915 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-20T01:48:28.920 INFO:tasks.rados.rados.0.smithi165.stdout:376: expect (ObjNum 102 snap 16 seq_num 102) 2024-04-20T01:48:28.947 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:28.961+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:29.049 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:29.063+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:29.097 INFO:teuthology.orchestra.run.smithi044.stdout:osd.14: {} 2024-04-20T01:48:29.101 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:29.101 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:29.114 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-20T01:48:29.118 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:48:29.120 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:48:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:48:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:48:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:48:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:48:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:48:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:48:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:48:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:48:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:48:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:48:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:48:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:48:29.121 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:48:29.122 INFO:teuthology.orchestra.run.smithi044.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-20T01:48:29.122 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:48:29.174 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:29.189+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:29.251 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:29.265+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:29.263 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:29.263 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:29.275 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-20T01:48:29.354 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:29.366+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:29.453 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:29.467+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:29.462 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:29.463 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:29.474 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-20T01:48:29.553 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:29.568+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:29.616 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:29.617 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:29.626 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-20T01:48:29.654 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:29.669+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:29.755 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:29.769+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:29.762 INFO:tasks.rados.rados.0.smithi165.stdout:374: done (2 left) 2024-04-20T01:48:29.763 INFO:tasks.rados.rados.0.smithi165.stdout:376: done (1 left) 2024-04-20T01:48:29.763 INFO:tasks.rados.rados.0.smithi165.stdout:377: done (0 left) 2024-04-20T01:48:29.763 INFO:tasks.rados.rados.0.smithi165.stdout:378: read oid 35 snap -1 2024-04-20T01:48:29.763 INFO:tasks.rados.rados.0.smithi165.stdout:378: expect (ObjNum 70 snap 5 seq_num 70) 2024-04-20T01:48:29.763 INFO:tasks.rados.rados.0.smithi165.stdout:379: read oid 4 snap -1 2024-04-20T01:48:29.763 INFO:tasks.rados.rados.0.smithi165.stdout:379: expect (ObjNum 137 snap 23 seq_num 137) 2024-04-20T01:48:29.763 INFO:tasks.rados.rados.0.smithi165.stdout:380: read oid 46 snap -1 2024-04-20T01:48:29.763 INFO:tasks.rados.rados.0.smithi165.stdout:380: expect (ObjNum 152 snap 27 seq_num 152) 2024-04-20T01:48:29.763 INFO:tasks.rados.rados.0.smithi165.stdout:381: copy_from oid 5 from oid 18 current snap is 30 2024-04-20T01:48:29.763 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 14 current snap is 30 2024-04-20T01:48:29.763 INFO:tasks.rados.rados.0.smithi165.stdout:382: seq_num 158 ranges {1720320=262144} 2024-04-20T01:48:29.767 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is set 2024-04-20T01:48:29.773 INFO:tasks.rados.rados.0.smithi165.stdout:382: writing smithi16579500-14 from 1720320 to 1982464 tid 1 2024-04-20T01:48:29.773 INFO:tasks.rados.rados.0.smithi165.stdout:383: read oid 34 snap 29 2024-04-20T01:48:29.773 INFO:tasks.rados.rados.0.smithi165.stdout:383: expect deleted 2024-04-20T01:48:29.774 INFO:tasks.rados.rados.0.smithi165.stdout:384: rollback oid 32 current snap is 30 2024-04-20T01:48:29.774 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 32 to 28 2024-04-20T01:48:29.774 INFO:tasks.rados.rados.0.smithi165.stdout:378: expect (ObjNum 70 snap 5 seq_num 70) 2024-04-20T01:48:29.800 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:29.800 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:29.812 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-20T01:48:29.812 INFO:tasks.rados.rados.0.smithi165.stdout:380: expect (ObjNum 152 snap 27 seq_num 152) 2024-04-20T01:48:29.847 INFO:tasks.rados.rados.0.smithi165.stdout:384: finishing rollback tid 1 to smithi16579500-32 2024-04-20T01:48:29.847 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 32 v 0 (ObjNum 33 snap 0 seq_num 2584647424) dirty dne 2024-04-20T01:48:29.847 INFO:tasks.rados.rados.0.smithi165.stdout:379: expect (ObjNum 137 snap 23 seq_num 137) 2024-04-20T01:48:29.856 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:29.871+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:29.903 INFO:tasks.rados.rados.0.smithi165.stdout:382: finishing write tid 1 to smithi16579500-14 2024-04-20T01:48:29.903 INFO:tasks.rados.rados.0.smithi165.stdout:382: finishing write tid 2 to smithi16579500-14 2024-04-20T01:48:29.903 INFO:tasks.rados.rados.0.smithi165.stdout:382: finishing write tid 3 to smithi16579500-14 2024-04-20T01:48:29.903 INFO:tasks.rados.rados.0.smithi165.stdout:382: oid 14 updating version 0 to 94 2024-04-20T01:48:29.903 INFO:tasks.rados.rados.0.smithi165.stdout:382: oid 14 version 94 is already newer than 93 2024-04-20T01:48:29.903 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 14 v 94 (ObjNum 158 snap 30 seq_num 158) dirty exists 2024-04-20T01:48:29.904 INFO:tasks.rados.rados.0.smithi165.stdout:382: left oid 14 (ObjNum 158 snap 30 seq_num 158) 2024-04-20T01:48:29.904 INFO:tasks.rados.rados.0.smithi165.stdout:381: finishing copy_from racing read to smithi16579500-5 2024-04-20T01:48:29.904 INFO:tasks.rados.rados.0.smithi165.stdout:381: finishing copy_from to smithi16579500-5 2024-04-20T01:48:29.904 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 5 v 46 (ObjNum 133 snap 21 seq_num 133) dirty exists 2024-04-20T01:48:29.904 INFO:tasks.rados.rados.0.smithi165.stdout:378: done (6 left) 2024-04-20T01:48:29.904 INFO:tasks.rados.rados.0.smithi165.stdout:379: done (5 left) 2024-04-20T01:48:29.904 INFO:tasks.rados.rados.0.smithi165.stdout:380: done (4 left) 2024-04-20T01:48:29.904 INFO:tasks.rados.rados.0.smithi165.stdout:381: done (3 left) 2024-04-20T01:48:29.904 INFO:tasks.rados.rados.0.smithi165.stdout:382: done (2 left) 2024-04-20T01:48:29.904 INFO:tasks.rados.rados.0.smithi165.stdout:383: done (1 left) 2024-04-20T01:48:29.904 INFO:tasks.rados.rados.0.smithi165.stdout:384: done (0 left) 2024-04-20T01:48:29.904 INFO:tasks.rados.rados.0.smithi165.stdout:385: copy_from oid 38 from oid 10 current snap is 30 2024-04-20T01:48:29.904 INFO:tasks.rados.rados.0.smithi165.stdout:386: snap_remove snap 20 2024-04-20T01:48:29.942 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:29.942 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:29.952 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-20T01:48:29.995 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:30.010+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:30.058 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:30.072+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:30.079 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:30.079 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:30.089 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-20T01:48:30.158 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:30.173+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:30.257 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:30.257 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:30.259 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:30.274+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:30.267 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-20T01:48:30.361 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:30.374+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:30.394 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:30.395 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:30.405 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-20T01:48:30.461 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:30.475+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:30.532 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:30.533 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:30.543 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-20T01:48:30.562 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:30.576+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:30.663 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:48:30.677+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:48:30.709 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:30.709 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:30.719 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-20T01:48:30.764 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:30.778+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:30.765 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-20T01:48:30.765 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-20T01:48:30.765 INFO:tasks.thrashosds.thrasher:Killing osd 14, live_osds are [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-20T01:48:30.765 DEBUG:tasks.ceph.osd.14:waiting for process to exit 2024-04-20T01:48:30.765 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:48:30.766 INFO:tasks.rados.rados.0.smithi165.stdout:386: done (1 left) 2024-04-20T01:48:30.766 INFO:tasks.rados.rados.0.smithi165.stdout:387: rmattr oid 44 current snap is 30 2024-04-20T01:48:30.766 INFO:tasks.rados.rados.0.smithi165.stdout:385: finishing copy_from racing read to smithi16579500-38 2024-04-20T01:48:30.767 INFO:tasks.rados.rados.0.smithi165.stdout:385: finishing copy_from to smithi16579500-38 2024-04-20T01:48:30.767 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 38 v 143 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-04-20T01:48:30.767 INFO:tasks.rados.rados.0.smithi165.stdout:385: done (1 left) 2024-04-20T01:48:30.767 INFO:tasks.rados.rados.0.smithi165.stdout:388: snap_remove snap 29 2024-04-20T01:48:30.797 INFO:tasks.ceph.osd.14:Stopped 2024-04-20T01:48:30.797 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.14 2024-04-20T01:48:30.797 INFO:tasks.thrashosds.thrasher:remote for osd 14 is ubuntu@smithi123.front.sepia.ceph.com 2024-04-20T01:48:30.797 DEBUG:teuthology.orchestra.run.smithi123:> type ceph-objectstore-tool 2024-04-20T01:48:30.855 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-20T01:48:30.856 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-14 --journal-path /var/lib/ceph/osd/ceph-14/journal --op list-pgs 2024-04-20T01:48:30.859 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:30.859 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:30.865 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:30.879+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:30.869 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-20T01:48:30.966 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:30.980+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:30.998 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:30.998 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:31.009 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-20T01:48:31.066 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:31.081+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:31.134 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-20T01:48:31.136 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:31.136 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:31.147 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-20T01:48:31.167 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:31.182+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:31.268 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:31.282+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:31.274 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:31.274 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:31.285 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-20T01:48:31.304 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:48:31.325 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:48:31.347 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:48:31.368 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:48:31.369 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:31.383+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:31.388 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:48:31.409 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:48:31.422 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:31.422 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:31.423 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:48:31.423 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:31.430 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:48:31.434 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-20T01:48:31.451 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:48:31.469 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:31.484+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:31.471 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:48:31.491 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:48:31.513 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:48:31.534 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:48:31.554 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:48:31.563 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:31.563 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:31.564 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 30, 2024-04-20T01:48:31.564 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-20T01:48:31.564 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:31.570 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:31.584+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:31.573 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-20T01:48:31.574 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:48:31.671 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:31.686+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:31.703 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:31.703 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-20T01:48:31.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-20T01:48:31.703 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-20T01:48:31.703 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.703 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:229 3.fs0 3:fc9748e2:::smithi16579500-32:head [read 0~524288,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:31.703 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.787590+0000", 2024-04-20T01:48:31.703 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 52.928922905999997, 2024-04-20T01:48:31.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047169430000000004, 2024-04-20T01:48:31.704 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:31.704 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:31.704 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:31.704 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:31.704 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:31.704 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 229 2024-04-20T01:48:31.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.704 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:31.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:31.704 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787590+0000", 2024-04-20T01:48:31.704 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:31.705 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787590+0000", 2024-04-20T01:48:31.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.705 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:31.705 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787589+0000", 2024-04-20T01:48:31.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:31.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.705 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:31.705 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787599+0000", 2024-04-20T01:48:31.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0658e-05 2024-04-20T01:48:31.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.706 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.706 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:31.706 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787618+0000", 2024-04-20T01:48:31.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9066999999999999e-05 2024-04-20T01:48:31.706 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.706 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.706 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:31.706 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787629+0000", 2024-04-20T01:48:31.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0604999999999999e-05 2024-04-20T01:48:31.706 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.706 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.706 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.706 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787703+0000", 2024-04-20T01:48:31.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4544999999999999e-05 2024-04-20T01:48:31.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op must wait for map", 2024-04-20T01:48:31.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787713+0000", 2024-04-20T01:48:31.707 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9930000000000007e-06 2024-04-20T01:48:31.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.789878+0000", 2024-04-20T01:48:31.707 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021646809999999999 2024-04-20T01:48:31.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:31.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.789941+0000", 2024-04-20T01:48:31.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2558000000000007e-05 2024-04-20T01:48:31.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:31.708 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.792307+0000", 2024-04-20T01:48:31.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023666920000000001 2024-04-20T01:48:31.708 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.708 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.708 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.708 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:230 3.fs0 3:fc9748e2:::smithi16579500-32:head [read 0~524288,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:31.708 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.787639+0000", 2024-04-20T01:48:31.708 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 52.928874452999999, 2024-04-20T01:48:31.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052033440000000004, 2024-04-20T01:48:31.709 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:31.709 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:31.709 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:31.709 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:31.709 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:31.709 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 230 2024-04-20T01:48:31.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.709 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:31.709 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.709 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:31.709 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787639+0000", 2024-04-20T01:48:31.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.709 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.709 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:31.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787639+0000", 2024-04-20T01:48:31.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:31.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787638+0000", 2024-04-20T01:48:31.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:31.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:31.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787642+0000", 2024-04-20T01:48:31.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4539999999999999e-06 2024-04-20T01:48:31.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:31.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787646+0000", 2024-04-20T01:48:31.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1269999999999996e-06 2024-04-20T01:48:31.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:31.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787650+0000", 2024-04-20T01:48:31.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.467e-06 2024-04-20T01:48:31.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787796+0000", 2024-04-20T01:48:31.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014651000000000001 2024-04-20T01:48:31.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.712 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:31.712 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787801+0000", 2024-04-20T01:48:31.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5059999999999997e-06 2024-04-20T01:48:31.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.712 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.712 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.790292+0000", 2024-04-20T01:48:31.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024917329999999999 2024-04-20T01:48:31.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.712 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:31.712 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.790354+0000", 2024-04-20T01:48:31.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1407000000000003e-05 2024-04-20T01:48:31.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.713 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:31.713 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.792842+0000", 2024-04-20T01:48:31.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024885139999999998 2024-04-20T01:48:31.713 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.713 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.713 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.713 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:231 3.fs0 3:fc9748e2:::smithi16579500-32:head [read 0~524288,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:31.713 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.787695+0000", 2024-04-20T01:48:31.713 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 52.928818221, 2024-04-20T01:48:31.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0078083780000000004, 2024-04-20T01:48:31.713 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:31.714 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:31.714 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:31.714 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:31.714 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:31.714 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 231 2024-04-20T01:48:31.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.714 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:31.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.714 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:31.714 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787695+0000", 2024-04-20T01:48:31.714 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.714 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:31.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787695+0000", 2024-04-20T01:48:31.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:31.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787694+0000", 2024-04-20T01:48:31.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:31.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:31.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787699+0000", 2024-04-20T01:48:31.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8439999999999997e-06 2024-04-20T01:48:31.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:31.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787708+0000", 2024-04-20T01:48:31.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2679999999999994e-06 2024-04-20T01:48:31.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:31.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787715+0000", 2024-04-20T01:48:31.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8290000000000008e-06 2024-04-20T01:48:31.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787839+0000", 2024-04-20T01:48:31.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012379700000000001 2024-04-20T01:48:31.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.717 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.717 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:31.717 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787845+0000", 2024-04-20T01:48:31.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1249999999999998e-06 2024-04-20T01:48:31.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.717 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.717 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.717 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.793431+0000", 2024-04-20T01:48:31.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055859880000000001 2024-04-20T01:48:31.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.717 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.717 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:31.717 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.793524+0000", 2024-04-20T01:48:31.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2943999999999996e-05 2024-04-20T01:48:31.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.718 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:31.718 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.795503+0000", 2024-04-20T01:48:31.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019791520000000001 2024-04-20T01:48:31.718 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.718 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.718 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.718 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(osd.13.22:3 3.fs0 3:f3742432:::smithi16579500-42:head [assert-version v6,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:31.718 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.788849+0000", 2024-04-20T01:48:31.718 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 52.927664601000004, 2024-04-20T01:48:31.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004535295, 2024-04-20T01:48:31.719 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:31.719 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:31.719 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:31.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:31.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788849+0000", 2024-04-20T01:48:31.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:31.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788849+0000", 2024-04-20T01:48:31.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:31.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788847+0000", 2024-04-20T01:48:31.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:31.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:31.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788856+0000", 2024-04-20T01:48:31.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4139999999999994e-06 2024-04-20T01:48:31.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:31.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788868+0000", 2024-04-20T01:48:31.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2235999999999999e-05 2024-04-20T01:48:31.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:31.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788880+0000", 2024-04-20T01:48:31.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2122e-05 2024-04-20T01:48:31.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788930+0000", 2024-04-20T01:48:31.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0160000000000001e-05 2024-04-20T01:48:31.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op must wait for map", 2024-04-20T01:48:31.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788938+0000", 2024-04-20T01:48:31.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8269999999999998e-06 2024-04-20T01:48:31.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.722 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.722 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.791323+0000", 2024-04-20T01:48:31.722 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002384471 2024-04-20T01:48:31.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.722 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:31.722 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.791386+0000", 2024-04-20T01:48:31.722 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3707000000000005e-05 2024-04-20T01:48:31.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.722 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:31.722 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.793384+0000", 2024-04-20T01:48:31.722 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019975800000000001 2024-04-20T01:48:31.723 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.723 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.723 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.723 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.ds1 38/30 ECSubWrite(tid=19, reqid=osd.6.0:18, at_version=0'0, trim_to=0'0, roll_forward_to=38'10))", 2024-04-20T01:48:31.723 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:39.794291+0000", 2024-04-20T01:48:31.723 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 51.922222091999998, 2024-04-20T01:48:31.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068904140000000001, 2024-04-20T01:48:31.723 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:31.723 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:31.723 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:31.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:31.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794291+0000", 2024-04-20T01:48:31.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:31.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794291+0000", 2024-04-20T01:48:31.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:31.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794290+0000", 2024-04-20T01:48:31.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:31.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:31.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794317+0000", 2024-04-20T01:48:31.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7328999999999999e-05 2024-04-20T01:48:31.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:31.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794331+0000", 2024-04-20T01:48:31.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4076999999999999e-05 2024-04-20T01:48:31.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:31.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794339+0000", 2024-04-20T01:48:31.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6629999999999993e-06 2024-04-20T01:48:31.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794383+0000", 2024-04-20T01:48:31.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4106000000000001e-05 2024-04-20T01:48:31.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:31.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794390+0000", 2024-04-20T01:48:31.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5710000000000003e-06 2024-04-20T01:48:31.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:31.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.801146+0000", 2024-04-20T01:48:31.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0067565300000000002 2024-04-20T01:48:31.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.727 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:31.727 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.801182+0000", 2024-04-20T01:48:31.727 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5472999999999997e-05 2024-04-20T01:48:31.727 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.727 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.727 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.727 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 42/30 ECSubWrite(tid=17, reqid=client.4643.0:307, at_version=42'10, trim_to=0'0, roll_forward_to=33'8))", 2024-04-20T01:48:31.727 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.830042+0000", 2024-04-20T01:48:31.727 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.886471041, 2024-04-20T01:48:31.727 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0099127780000000006, 2024-04-20T01:48:31.728 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:31.728 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:31.728 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:31.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.728 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:31.728 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830042+0000", 2024-04-20T01:48:31.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.728 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:31.728 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830042+0000", 2024-04-20T01:48:31.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:31.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830042+0000", 2024-04-20T01:48:31.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:31.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:31.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830306+0000", 2024-04-20T01:48:31.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026456199999999998 2024-04-20T01:48:31.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:31.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830326+0000", 2024-04-20T01:48:31.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9919e-05 2024-04-20T01:48:31.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:31.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830337+0000", 2024-04-20T01:48:31.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0693000000000001e-05 2024-04-20T01:48:31.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830414+0000", 2024-04-20T01:48:31.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6916999999999996e-05 2024-04-20T01:48:31.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:31.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830426+0000", 2024-04-20T01:48:31.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2231e-05 2024-04-20T01:48:31.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:31.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.839923+0000", 2024-04-20T01:48:31.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094965540000000008 2024-04-20T01:48:31.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:31.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.839955+0000", 2024-04-20T01:48:31.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2299999999999999e-05 2024-04-20T01:48:31.731 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.731 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.731 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.732 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 42/30 ECSubWrite(tid=22, reqid=client.4643.0:314, at_version=42'16, trim_to=0'0, roll_forward_to=42'12))", 2024-04-20T01:48:31.732 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.928764+0000", 2024-04-20T01:48:31.732 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.787748808000003, 2024-04-20T01:48:31.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036527479999999999, 2024-04-20T01:48:31.732 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:31.732 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:31.732 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:31.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:31.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.928764+0000", 2024-04-20T01:48:31.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:31.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.928764+0000", 2024-04-20T01:48:31.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:31.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.928764+0000", 2024-04-20T01:48:31.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:31.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:31.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.929079+0000", 2024-04-20T01:48:31.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031541300000000001 2024-04-20T01:48:31.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:31.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.929097+0000", 2024-04-20T01:48:31.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.823e-05 2024-04-20T01:48:31.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:31.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.929108+0000", 2024-04-20T01:48:31.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0139000000000001e-05 2024-04-20T01:48:31.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.929158+0000", 2024-04-20T01:48:31.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.994e-05 2024-04-20T01:48:31.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:31.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.929169+0000", 2024-04-20T01:48:31.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1464000000000001e-05 2024-04-20T01:48:31.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:31.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.932389+0000", 2024-04-20T01:48:31.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032196299999999998 2024-04-20T01:48:31.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:31.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.932417+0000", 2024-04-20T01:48:31.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8620999999999999e-05 2024-04-20T01:48:31.736 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.736 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.736 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.736 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 48/30 ECSubWrite(tid=44, reqid=osd.5.0:44, at_version=48'33, trim_to=0'0, roll_forward_to=47'31))", 2024-04-20T01:48:31.736 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.715966+0000", 2024-04-20T01:48:31.736 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.000547386000001, 2024-04-20T01:48:31.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045136789999999996, 2024-04-20T01:48:31.736 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:31.736 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:31.736 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:31.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:31.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.715966+0000", 2024-04-20T01:48:31.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:31.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.715966+0000", 2024-04-20T01:48:31.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:31.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.715966+0000", 2024-04-20T01:48:31.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:48:31.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:31.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.715986+0000", 2024-04-20T01:48:31.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.016e-05 2024-04-20T01:48:31.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:31.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.715995+0000", 2024-04-20T01:48:31.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5950000000000005e-06 2024-04-20T01:48:31.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:31.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716004+0000", 2024-04-20T01:48:31.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8480000000000007e-06 2024-04-20T01:48:31.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716039+0000", 2024-04-20T01:48:31.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4251000000000001e-05 2024-04-20T01:48:31.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:31.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716049+0000", 2024-04-20T01:48:31.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0276e-05 2024-04-20T01:48:31.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:31.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.720450+0000", 2024-04-20T01:48:31.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044011570000000002 2024-04-20T01:48:31.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:31.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.720480+0000", 2024-04-20T01:48:31.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9567999999999999e-05 2024-04-20T01:48:31.740 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.740 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.740 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.740 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.as6 48/30 ECSubWrite(tid=33, reqid=osd.6.0:33, at_version=48'14, trim_to=0'0, roll_forward_to=47'12))", 2024-04-20T01:48:31.740 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.716709+0000", 2024-04-20T01:48:31.740 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.999804593999997, 2024-04-20T01:48:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060784910000000001, 2024-04-20T01:48:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716709+0000", 2024-04-20T01:48:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716709+0000", 2024-04-20T01:48:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:31.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716709+0000", 2024-04-20T01:48:31.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:48:31.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:31.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716729+0000", 2024-04-20T01:48:31.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0735999999999999e-05 2024-04-20T01:48:31.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:31.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716740+0000", 2024-04-20T01:48:31.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0852e-05 2024-04-20T01:48:31.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:31.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716749+0000", 2024-04-20T01:48:31.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1700000000000003e-06 2024-04-20T01:48:31.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716830+0000", 2024-04-20T01:48:31.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0618999999999995e-05 2024-04-20T01:48:31.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:31.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716841+0000", 2024-04-20T01:48:31.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1421e-05 2024-04-20T01:48:31.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:31.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.722760+0000", 2024-04-20T01:48:31.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059184210000000001 2024-04-20T01:48:31.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:31.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.722787+0000", 2024-04-20T01:48:31.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7432000000000001e-05 2024-04-20T01:48:31.744 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.744 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.745 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.745 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 48/30 ECSubWrite(tid=45, reqid=osd.5.0:44, at_version=0'0, trim_to=0'0, roll_forward_to=48'33))", 2024-04-20T01:48:31.745 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.726177+0000", 2024-04-20T01:48:31.745 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.990336415000002, 2024-04-20T01:48:31.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062928719999999997, 2024-04-20T01:48:31.745 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:31.745 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:31.745 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:31.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:31.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726177+0000", 2024-04-20T01:48:31.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:31.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726177+0000", 2024-04-20T01:48:31.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:31.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726176+0000", 2024-04-20T01:48:31.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:31.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:31.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726187+0000", 2024-04-20T01:48:31.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1216000000000001e-05 2024-04-20T01:48:31.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:31.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726202+0000", 2024-04-20T01:48:31.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4324999999999999e-05 2024-04-20T01:48:31.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:31.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726216+0000", 2024-04-20T01:48:31.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4294000000000001e-05 2024-04-20T01:48:31.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726289+0000", 2024-04-20T01:48:31.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3349000000000005e-05 2024-04-20T01:48:31.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:31.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726300+0000", 2024-04-20T01:48:31.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0190000000000001e-05 2024-04-20T01:48:31.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:31.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.732436+0000", 2024-04-20T01:48:31.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006136028 2024-04-20T01:48:31.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:31.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.732470+0000", 2024-04-20T01:48:31.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4167e-05 2024-04-20T01:48:31.749 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.749 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.749 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.749 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 60/30 ECSubWrite(tid=57, reqid=osd.5.0:56, at_version=0'0, trim_to=0'0, roll_forward_to=60'44))", 2024-04-20T01:48:31.749 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:01.752594+0000", 2024-04-20T01:48:31.749 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 29.963919325999999, 2024-04-20T01:48:31.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048667290000000002, 2024-04-20T01:48:31.749 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:31.749 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:31.749 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:31.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.750 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:31.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.752594+0000", 2024-04-20T01:48:31.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.750 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:31.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.752594+0000", 2024-04-20T01:48:31.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.750 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:31.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.752593+0000", 2024-04-20T01:48:31.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:31.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:31.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.752604+0000", 2024-04-20T01:48:31.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0856000000000001e-05 2024-04-20T01:48:31.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:31.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.752616+0000", 2024-04-20T01:48:31.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1929e-05 2024-04-20T01:48:31.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:31.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.752637+0000", 2024-04-20T01:48:31.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1050999999999999e-05 2024-04-20T01:48:31.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.752686+0000", 2024-04-20T01:48:31.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8772e-05 2024-04-20T01:48:31.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:31.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.752696+0000", 2024-04-20T01:48:31.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0458999999999999e-05 2024-04-20T01:48:31.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:31.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.757429+0000", 2024-04-20T01:48:31.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047330180000000003 2024-04-20T01:48:31.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:31.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.757461+0000", 2024-04-20T01:48:31.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1248999999999998e-05 2024-04-20T01:48:31.753 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.753 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.753 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.753 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:697 3.fs0 3:f3742432:::smithi16579500-42:head [append 0~573440] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:31.753 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.031311+0000", 2024-04-20T01:48:31.753 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 21.685202327999999, 2024-04-20T01:48:31.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007174181, 2024-04-20T01:48:31.754 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:31.754 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:31.754 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:31.754 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:31.754 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:31.754 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 697 2024-04-20T01:48:31.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.754 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:31.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:31.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.031311+0000", 2024-04-20T01:48:31.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:31.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.031311+0000", 2024-04-20T01:48:31.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:31.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.031294+0000", 2024-04-20T01:48:31.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999828 2024-04-20T01:48:31.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:31.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.032139+0000", 2024-04-20T01:48:31.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00084458099999999996 2024-04-20T01:48:31.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:31.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.032156+0000", 2024-04-20T01:48:31.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7090000000000001e-05 2024-04-20T01:48:31.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:31.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.032169+0000", 2024-04-20T01:48:31.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3336000000000001e-05 2024-04-20T01:48:31.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.032222+0000", 2024-04-20T01:48:31.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3338000000000003e-05 2024-04-20T01:48:31.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:31.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.032307+0000", 2024-04-20T01:48:31.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4655999999999997e-05 2024-04-20T01:48:31.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:31.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.034259+0000", 2024-04-20T01:48:31.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019517340000000001 2024-04-20T01:48:31.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:31.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.035704+0000", 2024-04-20T01:48:31.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014451609999999999 2024-04-20T01:48:31.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:31.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.038432+0000", 2024-04-20T01:48:31.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002728285 2024-04-20T01:48:31.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:31.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.038485+0000", 2024-04-20T01:48:31.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3019999999999997e-05 2024-04-20T01:48:31.758 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.758 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.758 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.758 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:698 3.fs0 3:f3742432:::smithi16579500-42:head [append 0~622592] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:31.759 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.035540+0000", 2024-04-20T01:48:31.759 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 21.680973458, 2024-04-20T01:48:31.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060756079999999997, 2024-04-20T01:48:31.759 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:31.759 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:31.759 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:31.759 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:31.759 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:31.759 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 698 2024-04-20T01:48:31.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.759 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:31.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:31.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.035540+0000", 2024-04-20T01:48:31.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:31.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.035540+0000", 2024-04-20T01:48:31.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:31.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.035538+0000", 2024-04-20T01:48:31.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:31.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:31.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.036167+0000", 2024-04-20T01:48:31.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000628924 2024-04-20T01:48:31.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:31.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.036179+0000", 2024-04-20T01:48:31.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1865000000000001e-05 2024-04-20T01:48:31.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:31.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.036185+0000", 2024-04-20T01:48:31.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2570000000000004e-06 2024-04-20T01:48:31.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.036220+0000", 2024-04-20T01:48:31.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4646e-05 2024-04-20T01:48:31.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:31.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.036301+0000", 2024-04-20T01:48:31.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1258e-05 2024-04-20T01:48:31.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:31.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.036852+0000", 2024-04-20T01:48:31.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00055088000000000001 2024-04-20T01:48:31.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:31.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.037828+0000", 2024-04-20T01:48:31.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00097533299999999999 2024-04-20T01:48:31.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:31.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041571+0000", 2024-04-20T01:48:31.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037428990000000001 2024-04-20T01:48:31.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:31.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041615+0000", 2024-04-20T01:48:31.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4934e-05 2024-04-20T01:48:31.763 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.763 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.764 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.764 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:699 3.fs0 3:f3742432:::smithi16579500-42:head [append 0~622592] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:31.764 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.039511+0000", 2024-04-20T01:48:31.764 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 21.677001830000002, 2024-04-20T01:48:31.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048354820000000003, 2024-04-20T01:48:31.764 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:31.764 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:31.764 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:31.764 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:31.764 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:31.764 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 699 2024-04-20T01:48:31.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.765 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:31.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:31.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.039511+0000", 2024-04-20T01:48:31.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:31.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.039511+0000", 2024-04-20T01:48:31.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:31.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.039511+0000", 2024-04-20T01:48:31.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:31.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:31.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.040116+0000", 2024-04-20T01:48:31.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00060539600000000002 2024-04-20T01:48:31.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:31.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.040127+0000", 2024-04-20T01:48:31.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1097000000000001e-05 2024-04-20T01:48:31.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:31.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.040133+0000", 2024-04-20T01:48:31.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6030000000000004e-06 2024-04-20T01:48:31.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.040167+0000", 2024-04-20T01:48:31.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4439e-05 2024-04-20T01:48:31.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:31.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.040234+0000", 2024-04-20T01:48:31.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7099000000000003e-05 2024-04-20T01:48:31.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:31.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.040755+0000", 2024-04-20T01:48:31.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00052112299999999996 2024-04-20T01:48:31.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:31.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041677+0000", 2024-04-20T01:48:31.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00092136200000000001 2024-04-20T01:48:31.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:31.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.044307+0000", 2024-04-20T01:48:31.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026298329999999998 2024-04-20T01:48:31.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:31.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.044347+0000", 2024-04-20T01:48:31.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0370000000000001e-05 2024-04-20T01:48:31.769 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.769 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.769 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.769 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:700 3.fs0 3:f3742432:::smithi16579500-42:head [append 0~311296] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:31.769 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.041315+0000", 2024-04-20T01:48:31.769 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 21.675198203000001, 2024-04-20T01:48:31.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007465185, 2024-04-20T01:48:31.769 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:31.769 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:31.769 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:31.770 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:31.770 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:31.770 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 700 2024-04-20T01:48:31.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.770 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:31.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.770 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:31.770 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041315+0000", 2024-04-20T01:48:31.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.770 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:31.770 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041315+0000", 2024-04-20T01:48:31.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.771 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:31.771 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041314+0000", 2024-04-20T01:48:31.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:31.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.771 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:31.771 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041748+0000", 2024-04-20T01:48:31.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043357900000000002 2024-04-20T01:48:31.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.771 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:31.771 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041762+0000", 2024-04-20T01:48:31.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3529999999999999e-05 2024-04-20T01:48:31.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:31.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041770+0000", 2024-04-20T01:48:31.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.242e-06 2024-04-20T01:48:31.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041797+0000", 2024-04-20T01:48:31.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7515999999999999e-05 2024-04-20T01:48:31.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:31.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041859+0000", 2024-04-20T01:48:31.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1190000000000002e-05 2024-04-20T01:48:31.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:31.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.042286+0000", 2024-04-20T01:48:31.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000427401 2024-04-20T01:48:31.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:31.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.043538+0000", 2024-04-20T01:48:31.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012518799999999999 2024-04-20T01:48:31.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:31.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.048737+0000", 2024-04-20T01:48:31.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051987370000000001 2024-04-20T01:48:31.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:31.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.048780+0000", 2024-04-20T01:48:31.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3735000000000001e-05 2024-04-20T01:48:31.774 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.774 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.774 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.774 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:701 3.fs0 3:f3742432:::smithi16579500-42:head [setxattr _header (57) in=7b] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:31.774 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.041779+0000", 2024-04-20T01:48:31.775 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 21.674734667999999, 2024-04-20T01:48:31.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007335467, 2024-04-20T01:48:31.775 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:31.775 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:31.775 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:31.775 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:31.775 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:31.775 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 701 2024-04-20T01:48:31.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.775 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:31.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:31.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041779+0000", 2024-04-20T01:48:31.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:31.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041779+0000", 2024-04-20T01:48:31.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:31.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041778+0000", 2024-04-20T01:48:31.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:31.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:31.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041784+0000", 2024-04-20T01:48:31.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8649999999999998e-06 2024-04-20T01:48:31.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:31.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041791+0000", 2024-04-20T01:48:31.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8619999999999997e-06 2024-04-20T01:48:31.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:31.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041796+0000", 2024-04-20T01:48:31.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.429e-06 2024-04-20T01:48:31.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.042635+0000", 2024-04-20T01:48:31.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00083878299999999995 2024-04-20T01:48:31.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:31.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.042700+0000", 2024-04-20T01:48:31.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5116999999999993e-05 2024-04-20T01:48:31.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:31.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.043007+0000", 2024-04-20T01:48:31.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00030746499999999998 2024-04-20T01:48:31.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:31.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.043875+0000", 2024-04-20T01:48:31.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00086782099999999996 2024-04-20T01:48:31.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:31.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.049067+0000", 2024-04-20T01:48:31.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051916810000000001 2024-04-20T01:48:31.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:31.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.049114+0000", 2024-04-20T01:48:31.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7163999999999998e-05 2024-04-20T01:48:31.779 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.779 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.779 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.780 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:702 3.fs0 3:f3742432:::smithi16579500-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:31.780 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.041802+0000", 2024-04-20T01:48:31.780 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 21.674710968999999, 2024-04-20T01:48:31.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0093865919999999992, 2024-04-20T01:48:31.780 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:31.780 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:31.780 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:31.780 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:31.780 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:31.780 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 702 2024-04-20T01:48:31.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.780 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:31.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:31.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041802+0000", 2024-04-20T01:48:31.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:31.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041802+0000", 2024-04-20T01:48:31.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:31.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041802+0000", 2024-04-20T01:48:31.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:31.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:31.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041805+0000", 2024-04-20T01:48:31.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.689e-06 2024-04-20T01:48:31.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:31.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041812+0000", 2024-04-20T01:48:31.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3119999999999996e-06 2024-04-20T01:48:31.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:31.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041817+0000", 2024-04-20T01:48:31.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2889999999999997e-06 2024-04-20T01:48:31.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.043248+0000", 2024-04-20T01:48:31.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001430361 2024-04-20T01:48:31.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:31.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.043309+0000", 2024-04-20T01:48:31.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1328999999999995e-05 2024-04-20T01:48:31.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.049322+0000", 2024-04-20T01:48:31.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060133299999999999 2024-04-20T01:48:31.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:31.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.049422+0000", 2024-04-20T01:48:31.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9431000000000004e-05 2024-04-20T01:48:31.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:31.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.051189+0000", 2024-04-20T01:48:31.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001767315 2024-04-20T01:48:31.784 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.784 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.784 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.785 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s1 75/30 ECSubWrite(tid=363, reqid=client.4643.0:790, at_version=75'30, trim_to=0'0, roll_forward_to=75'28))", 2024-04-20T01:48:31.785 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:16.766177+0000", 2024-04-20T01:48:31.785 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.950336071000001, 2024-04-20T01:48:31.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0078257810000000004, 2024-04-20T01:48:31.785 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:31.785 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:31.785 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:31.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:31.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.766177+0000", 2024-04-20T01:48:31.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:31.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.766177+0000", 2024-04-20T01:48:31.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:31.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.766176+0000", 2024-04-20T01:48:31.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:31.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:31.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.767764+0000", 2024-04-20T01:48:31.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015875609999999999 2024-04-20T01:48:31.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:31.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.767781+0000", 2024-04-20T01:48:31.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6869999999999999e-05 2024-04-20T01:48:31.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:31.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.767792+0000", 2024-04-20T01:48:31.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0991e-05 2024-04-20T01:48:31.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.767844+0000", 2024-04-20T01:48:31.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2840999999999998e-05 2024-04-20T01:48:31.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:31.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.767856+0000", 2024-04-20T01:48:31.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1908e-05 2024-04-20T01:48:31.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:31.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.773978+0000", 2024-04-20T01:48:31.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061211390000000003 2024-04-20T01:48:31.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:31.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.774003+0000", 2024-04-20T01:48:31.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5449999999999999e-05 2024-04-20T01:48:31.789 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.789 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.789 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.789 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.as6 80/30 ECSubWrite(tid=43, reqid=client.4643.0:906, at_version=80'22, trim_to=0'0, roll_forward_to=66'20))", 2024-04-20T01:48:31.789 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:21.740963+0000", 2024-04-20T01:48:31.789 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.9755507839999993, 2024-04-20T01:48:31.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035229749999999998, 2024-04-20T01:48:31.789 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:31.789 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:31.789 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:31.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:31.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740963+0000", 2024-04-20T01:48:31.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:31.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740963+0000", 2024-04-20T01:48:31.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:31.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740962+0000", 2024-04-20T01:48:31.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:31.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:31.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740986+0000", 2024-04-20T01:48:31.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4111999999999999e-05 2024-04-20T01:48:31.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:31.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740998+0000", 2024-04-20T01:48:31.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2215000000000001e-05 2024-04-20T01:48:31.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:31.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.741008+0000", 2024-04-20T01:48:31.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6730000000000004e-06 2024-04-20T01:48:31.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.741066+0000", 2024-04-20T01:48:31.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7843000000000002e-05 2024-04-20T01:48:31.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:31.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.741077+0000", 2024-04-20T01:48:31.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.114e-05 2024-04-20T01:48:31.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:31.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.744464+0000", 2024-04-20T01:48:31.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0033866500000000002 2024-04-20T01:48:31.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:31.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.744485+0000", 2024-04-20T01:48:31.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1769000000000001e-05 2024-04-20T01:48:31.793 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.793 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.793 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.793 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s1 86/30 ECSubWrite(tid=394, reqid=client.4643.0:962, at_version=86'43, trim_to=0'0, roll_forward_to=80'39))", 2024-04-20T01:48:31.793 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:27.787296+0000", 2024-04-20T01:48:31.793 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 3.9292175230000002, 2024-04-20T01:48:31.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048431339999999998, 2024-04-20T01:48:31.793 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:31.793 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:31.794 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:31.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:31.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.787296+0000", 2024-04-20T01:48:31.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:31.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.787296+0000", 2024-04-20T01:48:31.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:31.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:31.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.787295+0000", 2024-04-20T01:48:31.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:31.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:31.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.787310+0000", 2024-04-20T01:48:31.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4963999999999999e-05 2024-04-20T01:48:31.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:31.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.787323+0000", 2024-04-20T01:48:31.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2497000000000001e-05 2024-04-20T01:48:31.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:31.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.787332+0000", 2024-04-20T01:48:31.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0380000000000005e-06 2024-04-20T01:48:31.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:31.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.787370+0000", 2024-04-20T01:48:31.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8177999999999999e-05 2024-04-20T01:48:31.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:31.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.787379+0000", 2024-04-20T01:48:31.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5259999999999999e-06 2024-04-20T01:48:31.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:31.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.792116+0000", 2024-04-20T01:48:31.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047370340000000002 2024-04-20T01:48:31.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:31.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:31.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:31.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.792139+0000", 2024-04-20T01:48:31.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2588000000000001e-05 2024-04-20T01:48:31.797 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.797 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.797 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.797 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:31.797 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:31.797 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:31.798 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-20T01:48:31.802 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:31.813+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:31.803 INFO:tasks.rados.rados.0.smithi165.stdout:388: done (1 left) 2024-04-20T01:48:31.803 INFO:tasks.rados.rados.0.smithi165.stdout:389: read oid 42 snap 25 2024-04-20T01:48:31.803 INFO:tasks.rados.rados.0.smithi165.stdout:389: expect (ObjNum 138 snap 23 seq_num 138) 2024-04-20T01:48:31.803 INFO:tasks.rados.rados.0.smithi165.stdout:390: read oid 14 snap -1 2024-04-20T01:48:31.803 INFO:tasks.rados.rados.0.smithi165.stdout:390: expect (ObjNum 158 snap 30 seq_num 158) 2024-04-20T01:48:31.804 INFO:tasks.rados.rados.0.smithi165.stdout:391: read oid 10 snap 27 2024-04-20T01:48:31.804 INFO:tasks.rados.rados.0.smithi165.stdout:391: expect (ObjNum 9 snap 0 seq_num 9) 2024-04-20T01:48:31.804 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 49 current snap is 30 2024-04-20T01:48:31.804 INFO:tasks.rados.rados.0.smithi165.stdout:392: seq_num 159 ranges {2228224=278528} 2024-04-20T01:48:31.804 INFO:tasks.rados.rados.0.smithi165.stdout:392: writing smithi16579500-49 from 2228224 to 2506752 tid 1 2024-04-20T01:48:31.804 INFO:tasks.rados.rados.0.smithi165.stdout:393: read oid 48 snap -1 2024-04-20T01:48:31.804 INFO:tasks.rados.rados.0.smithi165.stdout:393: expect (ObjNum 136 snap 22 seq_num 136) 2024-04-20T01:48:31.804 INFO:tasks.rados.rados.0.smithi165.stdout:394: snap_remove snap 28 2024-04-20T01:48:31.804 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-20T01:48:31.900 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:31.914+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:31.973 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:31.973 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:31.974 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:48:31.974 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:31.984 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-20T01:48:32.001 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:32.015+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:32.063 INFO:teuthology.orchestra.run.smithi123.stdout:3.cs0 2024-04-20T01:48:32.063 INFO:teuthology.orchestra.run.smithi123.stdout:3.as3 2024-04-20T01:48:32.063 INFO:teuthology.orchestra.run.smithi123.stdout:3.9s4 2024-04-20T01:48:32.063 INFO:teuthology.orchestra.run.smithi123.stdout:3.7s6 2024-04-20T01:48:32.063 INFO:teuthology.orchestra.run.smithi123.stdout:3.5s3 2024-04-20T01:48:32.063 INFO:teuthology.orchestra.run.smithi123.stdout:3.3s3 2024-04-20T01:48:32.063 INFO:teuthology.orchestra.run.smithi123.stdout:3.2s4 2024-04-20T01:48:32.063 INFO:teuthology.orchestra.run.smithi123.stdout:3.8s4 2024-04-20T01:48:32.063 INFO:teuthology.orchestra.run.smithi123.stdout:3.1s1 2024-04-20T01:48:32.102 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:32.116+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:32.116 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:32.116 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:32.116 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 30, 2024-04-20T01:48:32.116 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-20T01:48:32.117 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:32.127 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-20T01:48:32.203 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:32.217+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:32.256 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-20T01:48:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-20T01:48:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-20T01:48:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:243 3.ds0 3:b226b74f:::smithi16579500-7:head [delete] snapc 5=[5,2] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.787656+0000", 2024-04-20T01:48:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 53.481764374999997, 2024-04-20T01:48:32.256 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013205184, 2024-04-20T01:48:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 243 2024-04-20T01:48:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787656+0000", 2024-04-20T01:48:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787656+0000", 2024-04-20T01:48:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787654+0000", 2024-04-20T01:48:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787662+0000", 2024-04-20T01:48:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6599999999999995e-06 2024-04-20T01:48:32.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787680+0000", 2024-04-20T01:48:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7672999999999998e-05 2024-04-20T01:48:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787688+0000", 2024-04-20T01:48:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.405e-06 2024-04-20T01:48:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787739+0000", 2024-04-20T01:48:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0652999999999998e-05 2024-04-20T01:48:32.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787851+0000", 2024-04-20T01:48:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112172 2024-04-20T01:48:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788643+0000", 2024-04-20T01:48:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00079174300000000003 2024-04-20T01:48:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.789564+0000", 2024-04-20T01:48:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00092135199999999996 2024-04-20T01:48:32.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.800781+0000", 2024-04-20T01:48:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011217187999999999 2024-04-20T01:48:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.800861+0000", 2024-04-20T01:48:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9796999999999995e-05 2024-04-20T01:48:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.261 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.1s4 48/30 ECSubWrite(tid=119, reqid=osd.4.0:119, at_version=48'14, trim_to=0'0, roll_forward_to=47'12))", 2024-04-20T01:48:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.717339+0000", 2024-04-20T01:48:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.552081688000001, 2024-04-20T01:48:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0079470240000000004, 2024-04-20T01:48:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717339+0000", 2024-04-20T01:48:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.262 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717339+0000", 2024-04-20T01:48:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717338+0000", 2024-04-20T01:48:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717357+0000", 2024-04-20T01:48:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8575000000000001e-05 2024-04-20T01:48:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717368+0000", 2024-04-20T01:48:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.145e-05 2024-04-20T01:48:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717377+0000", 2024-04-20T01:48:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4929999999999998e-06 2024-04-20T01:48:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717424+0000", 2024-04-20T01:48:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7240000000000002e-05 2024-04-20T01:48:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717435+0000", 2024-04-20T01:48:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0533000000000001e-05 2024-04-20T01:48:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.725244+0000", 2024-04-20T01:48:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0078092129999999997 2024-04-20T01:48:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.725286+0000", 2024-04-20T01:48:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1792999999999998e-05 2024-04-20T01:48:32.265 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 60/30 ECSubWrite(tid=280, reqid=client.4643.0:519, at_version=60'66, trim_to=0'0, roll_forward_to=60'63))", 2024-04-20T01:48:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:01.837460+0000", 2024-04-20T01:48:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 30.431960483000001, 2024-04-20T01:48:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071325599999999996, 2024-04-20T01:48:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837460+0000", 2024-04-20T01:48:32.266 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837460+0000", 2024-04-20T01:48:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837459+0000", 2024-04-20T01:48:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837876+0000", 2024-04-20T01:48:32.267 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00041722999999999999 2024-04-20T01:48:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837893+0000", 2024-04-20T01:48:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6803000000000002e-05 2024-04-20T01:48:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837903+0000", 2024-04-20T01:48:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6830000000000003e-06 2024-04-20T01:48:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.268 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837955+0000", 2024-04-20T01:48:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2160000000000002e-05 2024-04-20T01:48:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837967+0000", 2024-04-20T01:48:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2057e-05 2024-04-20T01:48:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.844571+0000", 2024-04-20T01:48:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066042319999999998 2024-04-20T01:48:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.269 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.844592+0000", 2024-04-20T01:48:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1268999999999999e-05 2024-04-20T01:48:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 63/30 ECSubWrite(tid=295, reqid=client.4643.0:574, at_version=63'79, trim_to=0'0, roll_forward_to=62'77))", 2024-04-20T01:48:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:04.795046+0000", 2024-04-20T01:48:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.474374359999999, 2024-04-20T01:48:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076412559999999999, 2024-04-20T01:48:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.270 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795046+0000", 2024-04-20T01:48:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795046+0000", 2024-04-20T01:48:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795044+0000", 2024-04-20T01:48:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.271 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795601+0000", 2024-04-20T01:48:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00055720399999999995 2024-04-20T01:48:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795617+0000", 2024-04-20T01:48:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5341000000000001e-05 2024-04-20T01:48:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795627+0000", 2024-04-20T01:48:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0429e-05 2024-04-20T01:48:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.272 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795692+0000", 2024-04-20T01:48:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4712e-05 2024-04-20T01:48:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795704+0000", 2024-04-20T01:48:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.258e-05 2024-04-20T01:48:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.802650+0000", 2024-04-20T01:48:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069452869999999996 2024-04-20T01:48:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.273 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.802687+0000", 2024-04-20T01:48:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7483e-05 2024-04-20T01:48:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 63/30 ECSubWrite(tid=296, reqid=client.4643.0:575, at_version=63'80, trim_to=0'0, roll_forward_to=62'77))", 2024-04-20T01:48:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:04.798415+0000", 2024-04-20T01:48:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.471005560999998, 2024-04-20T01:48:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071008770000000002, 2024-04-20T01:48:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.274 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798415+0000", 2024-04-20T01:48:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798415+0000", 2024-04-20T01:48:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798414+0000", 2024-04-20T01:48:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:32.275 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798883+0000", 2024-04-20T01:48:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00046914899999999998 2024-04-20T01:48:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798898+0000", 2024-04-20T01:48:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4534e-05 2024-04-20T01:48:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798905+0000", 2024-04-20T01:48:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9149999999999996e-06 2024-04-20T01:48:32.276 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798951+0000", 2024-04-20T01:48:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5776999999999999e-05 2024-04-20T01:48:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798961+0000", 2024-04-20T01:48:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9289999999999993e-06 2024-04-20T01:48:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.805490+0000", 2024-04-20T01:48:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065291029999999996 2024-04-20T01:48:32.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.805516+0000", 2024-04-20T01:48:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5347999999999999e-05 2024-04-20T01:48:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.cs2 68/30 ECSubWrite(tid=68, reqid=client.4643.0:614, at_version=68'59, trim_to=0'0, roll_forward_to=68'56))", 2024-04-20T01:48:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:09.812830+0000", 2024-04-20T01:48:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 22.456590404, 2024-04-20T01:48:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066006179999999999, 2024-04-20T01:48:32.278 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.812830+0000", 2024-04-20T01:48:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.812830+0000", 2024-04-20T01:48:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.279 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.812829+0000", 2024-04-20T01:48:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.812899+0000", 2024-04-20T01:48:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9803999999999998e-05 2024-04-20T01:48:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.812912+0000", 2024-04-20T01:48:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3552e-05 2024-04-20T01:48:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.812921+0000", 2024-04-20T01:48:32.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7469999999999994e-06 2024-04-20T01:48:32.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.281 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.281 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.282 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.813009+0000", 2024-04-20T01:48:32.282 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8298000000000001e-05 2024-04-20T01:48:32.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.282 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.282 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.282 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.813023+0000", 2024-04-20T01:48:32.282 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3701e-05 2024-04-20T01:48:32.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.282 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.282 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.282 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.819402+0000", 2024-04-20T01:48:32.282 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063790799999999996 2024-04-20T01:48:32.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.283 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.283 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.283 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.819431+0000", 2024-04-20T01:48:32.283 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8513999999999998e-05 2024-04-20T01:48:32.283 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.283 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.283 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.283 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.283 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.283 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.cs2 68/30 ECSubWrite(tid=69, reqid=client.4643.0:615, at_version=68'60, trim_to=0'0, roll_forward_to=68'56))", 2024-04-20T01:48:32.283 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:09.814877+0000", 2024-04-20T01:48:32.283 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 22.454543162, 2024-04-20T01:48:32.283 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076095620000000003, 2024-04-20T01:48:32.283 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.284 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.284 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.284 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.284 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.284 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.814877+0000", 2024-04-20T01:48:32.284 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.284 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.284 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.284 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.284 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.814877+0000", 2024-04-20T01:48:32.284 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.284 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.284 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.284 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.284 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.814876+0000", 2024-04-20T01:48:32.285 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:32.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.285 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.285 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.285 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.814983+0000", 2024-04-20T01:48:32.285 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010743699999999999 2024-04-20T01:48:32.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.285 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.285 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.285 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.814996+0000", 2024-04-20T01:48:32.285 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2884000000000001e-05 2024-04-20T01:48:32.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.285 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.285 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.285 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815005+0000", 2024-04-20T01:48:32.286 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5369999999999997e-06 2024-04-20T01:48:32.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.286 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.286 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.286 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815050+0000", 2024-04-20T01:48:32.286 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5364999999999997e-05 2024-04-20T01:48:32.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.286 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.286 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.286 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815059+0000", 2024-04-20T01:48:32.286 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9530000000000005e-06 2024-04-20T01:48:32.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.286 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.286 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.286 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.822460+0000", 2024-04-20T01:48:32.287 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0074009109999999996 2024-04-20T01:48:32.287 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.287 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.287 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.287 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.822487+0000", 2024-04-20T01:48:32.287 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6939000000000001e-05 2024-04-20T01:48:32.287 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.287 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.287 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.287 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.287 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.287 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.cs2 68/30 ECSubWrite(tid=70, reqid=client.4643.0:616, at_version=68'61, trim_to=0'0, roll_forward_to=68'56))", 2024-04-20T01:48:32.287 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:09.815775+0000", 2024-04-20T01:48:32.287 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 22.453645590000001, 2024-04-20T01:48:32.287 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0067272979999999996, 2024-04-20T01:48:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815775+0000", 2024-04-20T01:48:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815775+0000", 2024-04-20T01:48:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.289 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815774+0000", 2024-04-20T01:48:32.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:32.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.289 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.289 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815794+0000", 2024-04-20T01:48:32.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9891999999999999e-05 2024-04-20T01:48:32.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.289 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.289 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815808+0000", 2024-04-20T01:48:32.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4584e-05 2024-04-20T01:48:32.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.289 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815819+0000", 2024-04-20T01:48:32.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0216e-05 2024-04-20T01:48:32.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.290 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.816802+0000", 2024-04-20T01:48:32.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0009835480000000001 2024-04-20T01:48:32.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.290 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.816813+0000", 2024-04-20T01:48:32.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1156e-05 2024-04-20T01:48:32.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.822491+0000", 2024-04-20T01:48:32.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056774030000000001 2024-04-20T01:48:32.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.822502+0000", 2024-04-20T01:48:32.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1297999999999999e-05 2024-04-20T01:48:32.291 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.291 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.291 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.291 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.cs2 70/30 ECSubWrite(tid=103, reqid=client.4643.0:738, at_version=70'83, trim_to=0'0, roll_forward_to=70'80))", 2024-04-20T01:48:32.291 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:11.861902+0000", 2024-04-20T01:48:32.292 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.407518502999999, 2024-04-20T01:48:32.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0073928659999999997, 2024-04-20T01:48:32.292 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.292 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.292 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.292 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.292 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861902+0000", 2024-04-20T01:48:32.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.292 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.292 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861902+0000", 2024-04-20T01:48:32.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861901+0000", 2024-04-20T01:48:32.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:32.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861922+0000", 2024-04-20T01:48:32.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0917e-05 2024-04-20T01:48:32.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861935+0000", 2024-04-20T01:48:32.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3237e-05 2024-04-20T01:48:32.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.294 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.294 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.294 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861945+0000", 2024-04-20T01:48:32.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0326000000000001e-05 2024-04-20T01:48:32.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.294 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.294 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.294 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.862010+0000", 2024-04-20T01:48:32.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4083000000000003e-05 2024-04-20T01:48:32.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.294 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.294 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.294 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.862019+0000", 2024-04-20T01:48:32.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0659999999999993e-06 2024-04-20T01:48:32.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.869223+0000", 2024-04-20T01:48:32.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007204612 2024-04-20T01:48:32.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.869295+0000", 2024-04-20T01:48:32.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1421999999999995e-05 2024-04-20T01:48:32.295 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.295 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.295 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.296 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.296 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 75/30 ECSubWrite(tid=357, reqid=osd.13.0:357, at_version=75'127, trim_to=0'0, roll_forward_to=75'123))", 2024-04-20T01:48:32.296 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:16.739745+0000", 2024-04-20T01:48:32.296 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.529675433, 2024-04-20T01:48:32.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006651985, 2024-04-20T01:48:32.296 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.296 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.296 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.296 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.296 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.296 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739745+0000", 2024-04-20T01:48:32.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739745+0000", 2024-04-20T01:48:32.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739744+0000", 2024-04-20T01:48:32.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:32.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739763+0000", 2024-04-20T01:48:32.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9182e-05 2024-04-20T01:48:32.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739776+0000", 2024-04-20T01:48:32.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2548000000000001e-05 2024-04-20T01:48:32.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739782+0000", 2024-04-20T01:48:32.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0909999999999998e-06 2024-04-20T01:48:32.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739826+0000", 2024-04-20T01:48:32.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3927999999999997e-05 2024-04-20T01:48:32.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.299 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.299 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739836+0000", 2024-04-20T01:48:32.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0368e-05 2024-04-20T01:48:32.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.299 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.299 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.746372+0000", 2024-04-20T01:48:32.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065361309999999997 2024-04-20T01:48:32.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.299 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.299 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.746397+0000", 2024-04-20T01:48:32.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4477999999999998e-05 2024-04-20T01:48:32.300 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.300 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.300 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.300 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 75/30 ECSubWrite(tid=358, reqid=osd.13.0:357, at_version=0'0, trim_to=0'0, roll_forward_to=75'127))", 2024-04-20T01:48:32.300 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:16.750738+0000", 2024-04-20T01:48:32.300 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.518682593999999, 2024-04-20T01:48:32.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066441419999999996, 2024-04-20T01:48:32.300 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.300 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.300 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.300 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.300 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.750738+0000", 2024-04-20T01:48:32.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.750738+0000", 2024-04-20T01:48:32.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.750737+0000", 2024-04-20T01:48:32.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:32.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.750748+0000", 2024-04-20T01:48:32.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0586e-05 2024-04-20T01:48:32.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.750760+0000", 2024-04-20T01:48:32.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2051e-05 2024-04-20T01:48:32.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.750768+0000", 2024-04-20T01:48:32.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0180000000000001e-06 2024-04-20T01:48:32.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.750817+0000", 2024-04-20T01:48:32.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9332000000000001e-05 2024-04-20T01:48:32.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.750827+0000", 2024-04-20T01:48:32.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0071000000000001e-05 2024-04-20T01:48:32.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.757354+0000", 2024-04-20T01:48:32.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065269899999999999 2024-04-20T01:48:32.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.757382+0000", 2024-04-20T01:48:32.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7761000000000001e-05 2024-04-20T01:48:32.304 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.304 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.304 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.304 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.1s4 76/30 ECSubWrite(tid=271, reqid=osd.4.0:271, at_version=76'43, trim_to=0'0, roll_forward_to=75'41))", 2024-04-20T01:48:32.304 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:17.716916+0000", 2024-04-20T01:48:32.304 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.552504025999999, 2024-04-20T01:48:32.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066297489999999999, 2024-04-20T01:48:32.304 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.304 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.304 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:17.716916+0000", 2024-04-20T01:48:32.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:17.716916+0000", 2024-04-20T01:48:32.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:17.716916+0000", 2024-04-20T01:48:32.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:48:32.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:17.716936+0000", 2024-04-20T01:48:32.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9553000000000001e-05 2024-04-20T01:48:32.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:17.716947+0000", 2024-04-20T01:48:32.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.13e-05 2024-04-20T01:48:32.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:17.716961+0000", 2024-04-20T01:48:32.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4402e-05 2024-04-20T01:48:32.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:17.717016+0000", 2024-04-20T01:48:32.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4577999999999998e-05 2024-04-20T01:48:32.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:17.717027+0000", 2024-04-20T01:48:32.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0776e-05 2024-04-20T01:48:32.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:17.723515+0000", 2024-04-20T01:48:32.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064878319999999998 2024-04-20T01:48:32.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:17.723546+0000", 2024-04-20T01:48:32.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1439000000000002e-05 2024-04-20T01:48:32.308 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.308 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.308 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.308 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.1s4 80/30 ECSubWrite(tid=302, reqid=osd.4.0:302, at_version=80'47, trim_to=0'0, roll_forward_to=77'45))", 2024-04-20T01:48:32.308 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:21.728944+0000", 2024-04-20T01:48:32.308 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.540476023, 2024-04-20T01:48:32.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0093705790000000004, 2024-04-20T01:48:32.308 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.308 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.308 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.728944+0000", 2024-04-20T01:48:32.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.728944+0000", 2024-04-20T01:48:32.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.728944+0000", 2024-04-20T01:48:32.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:32.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.728974+0000", 2024-04-20T01:48:32.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9991000000000001e-05 2024-04-20T01:48:32.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.728997+0000", 2024-04-20T01:48:32.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2796999999999999e-05 2024-04-20T01:48:32.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729010+0000", 2024-04-20T01:48:32.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2819999999999999e-05 2024-04-20T01:48:32.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729099+0000", 2024-04-20T01:48:32.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9382999999999994e-05 2024-04-20T01:48:32.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729151+0000", 2024-04-20T01:48:32.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1569e-05 2024-04-20T01:48:32.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.738292+0000", 2024-04-20T01:48:32.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0091419180000000006 2024-04-20T01:48:32.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.738315+0000", 2024-04-20T01:48:32.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2424000000000001e-05 2024-04-20T01:48:32.312 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.313 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.313 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.313 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.cs2 80/30 ECSubWrite(tid=132, reqid=osd.14.0:132, at_version=80'111, trim_to=0'0, roll_forward_to=77'109))", 2024-04-20T01:48:32.313 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:21.729632+0000", 2024-04-20T01:48:32.313 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.539787909999999, 2024-04-20T01:48:32.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0086544840000000005, 2024-04-20T01:48:32.313 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.313 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.313 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729632+0000", 2024-04-20T01:48:32.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729632+0000", 2024-04-20T01:48:32.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729632+0000", 2024-04-20T01:48:32.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:32.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729665+0000", 2024-04-20T01:48:32.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2699e-05 2024-04-20T01:48:32.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729689+0000", 2024-04-20T01:48:32.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4661e-05 2024-04-20T01:48:32.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729701+0000", 2024-04-20T01:48:32.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1432e-05 2024-04-20T01:48:32.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729801+0000", 2024-04-20T01:48:32.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100319 2024-04-20T01:48:32.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.316 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729812+0000", 2024-04-20T01:48:32.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1139e-05 2024-04-20T01:48:32.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.316 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.738257+0000", 2024-04-20T01:48:32.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0084448430000000005 2024-04-20T01:48:32.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.738287+0000", 2024-04-20T01:48:32.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9682000000000001e-05 2024-04-20T01:48:32.317 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.317 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.317 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.317 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:906 3.as0 3:5c6da789:::smithi16579500-40:head [append 0~0] snapc 2f=[2f,2e,28] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-04-20T01:48:32.317 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:21.729776+0000", 2024-04-20T01:48:32.317 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.539644229, 2024-04-20T01:48:32.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.023520553, 2024-04-20T01:48:32.317 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.317 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:32.318 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:32.318 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:32.318 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:32.318 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 906 2024-04-20T01:48:32.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.318 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729776+0000", 2024-04-20T01:48:32.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729776+0000", 2024-04-20T01:48:32.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729774+0000", 2024-04-20T01:48:32.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:32.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729787+0000", 2024-04-20T01:48:32.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3174e-05 2024-04-20T01:48:32.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729805+0000", 2024-04-20T01:48:32.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7558999999999999e-05 2024-04-20T01:48:32.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729816+0000", 2024-04-20T01:48:32.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.111e-05 2024-04-20T01:48:32.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729925+0000", 2024-04-20T01:48:32.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000109251 2024-04-20T01:48:32.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.321 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op must wait for map", 2024-04-20T01:48:32.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729934+0000", 2024-04-20T01:48:32.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0880000000000001e-06 2024-04-20T01:48:32.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.321 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.733659+0000", 2024-04-20T01:48:32.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037253389999999998 2024-04-20T01:48:32.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.321 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:32.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.733787+0000", 2024-04-20T01:48:32.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012743499999999999 2024-04-20T01:48:32.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.741029+0000", 2024-04-20T01:48:32.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0072423269999999998 2024-04-20T01:48:32.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.749061+0000", 2024-04-20T01:48:32.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080322569999999992 2024-04-20T01:48:32.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:32.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.753216+0000", 2024-04-20T01:48:32.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00415468 2024-04-20T01:48:32.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.753297+0000", 2024-04-20T01:48:32.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0539999999999998e-05 2024-04-20T01:48:32.323 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.323 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.323 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.323 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:907 3.as0 3:5c6da789:::smithi16579500-40:head [rollback 28] snapc 2f=[2f,2e,28] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-04-20T01:48:32.323 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:21.729835+0000", 2024-04-20T01:48:32.323 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.539585406, 2024-04-20T01:48:32.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.02475664, 2024-04-20T01:48:32.323 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.323 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:32.324 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:32.324 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:32.324 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:32.324 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 907 2024-04-20T01:48:32.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.324 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.324 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.324 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729835+0000", 2024-04-20T01:48:32.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.324 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.324 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729835+0000", 2024-04-20T01:48:32.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729834+0000", 2024-04-20T01:48:32.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:32.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729838+0000", 2024-04-20T01:48:32.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1729999999999997e-06 2024-04-20T01:48:32.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729846+0000", 2024-04-20T01:48:32.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9379999999999992e-06 2024-04-20T01:48:32.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729853+0000", 2024-04-20T01:48:32.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6590000000000001e-06 2024-04-20T01:48:32.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729995+0000", 2024-04-20T01:48:32.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000142397 2024-04-20T01:48:32.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:32.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.730002+0000", 2024-04-20T01:48:32.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7950000000000002e-06 2024-04-20T01:48:32.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.741764+0000", 2024-04-20T01:48:32.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011761466 2024-04-20T01:48:32.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:32.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.741851+0000", 2024-04-20T01:48:32.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7447999999999994e-05 2024-04-20T01:48:32.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.742330+0000", 2024-04-20T01:48:32.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00047901900000000002 2024-04-20T01:48:32.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.752214+0000", 2024-04-20T01:48:32.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0098841180000000008 2024-04-20T01:48:32.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:32.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.754530+0000", 2024-04-20T01:48:32.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023153750000000002 2024-04-20T01:48:32.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.754592+0000", 2024-04-20T01:48:32.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1982000000000004e-05 2024-04-20T01:48:32.329 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.329 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.329 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.329 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:908 3.as0 3:5c6da789:::smithi16579500-40:head [append 0~0] snapc 2f=[2f,2e,28] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-04-20T01:48:32.329 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:21.729859+0000", 2024-04-20T01:48:32.329 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.539561291, 2024-04-20T01:48:32.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.024873134000000002, 2024-04-20T01:48:32.329 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.329 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:32.330 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:32.330 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:32.330 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:32.330 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 908 2024-04-20T01:48:32.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.330 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729859+0000", 2024-04-20T01:48:32.330 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729859+0000", 2024-04-20T01:48:32.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729858+0000", 2024-04-20T01:48:32.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:32.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729862+0000", 2024-04-20T01:48:32.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3440000000000001e-06 2024-04-20T01:48:32.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729868+0000", 2024-04-20T01:48:32.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1009999999999997e-06 2024-04-20T01:48:32.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729874+0000", 2024-04-20T01:48:32.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8250000000000001e-06 2024-04-20T01:48:32.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.730024+0000", 2024-04-20T01:48:32.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000150233 2024-04-20T01:48:32.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:32.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.730030+0000", 2024-04-20T01:48:32.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0070000000000003e-06 2024-04-20T01:48:32.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.333 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.333 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.744082+0000", 2024-04-20T01:48:32.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014052090999999999 2024-04-20T01:48:32.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.333 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:32.333 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.744147+0000", 2024-04-20T01:48:32.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5204999999999997e-05 2024-04-20T01:48:32.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.744490+0000", 2024-04-20T01:48:32.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000343135 2024-04-20T01:48:32.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.752246+0000", 2024-04-20T01:48:32.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077554449999999997 2024-04-20T01:48:32.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:32.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.754690+0000", 2024-04-20T01:48:32.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024443870000000001 2024-04-20T01:48:32.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.754732+0000", 2024-04-20T01:48:32.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1870999999999999e-05 2024-04-20T01:48:32.335 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.335 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.335 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.335 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 80/30 ECSubWrite(tid=388, reqid=osd.13.0:386, at_version=0'0, trim_to=0'0, roll_forward_to=80'136))", 2024-04-20T01:48:32.335 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:21.735749+0000", 2024-04-20T01:48:32.335 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.533671353000001, 2024-04-20T01:48:32.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016494279000000001, 2024-04-20T01:48:32.335 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.335 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.335 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.735749+0000", 2024-04-20T01:48:32.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.735749+0000", 2024-04-20T01:48:32.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.735748+0000", 2024-04-20T01:48:32.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:32.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.735761+0000", 2024-04-20T01:48:32.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2860000000000001e-05 2024-04-20T01:48:32.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.735771+0000", 2024-04-20T01:48:32.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0312000000000001e-05 2024-04-20T01:48:32.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.735780+0000", 2024-04-20T01:48:32.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1600000000000004e-06 2024-04-20T01:48:32.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.338 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.744021+0000", 2024-04-20T01:48:32.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.008240486 2024-04-20T01:48:32.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.338 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.744026+0000", 2024-04-20T01:48:32.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3190000000000002e-06 2024-04-20T01:48:32.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.338 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.752220+0000", 2024-04-20T01:48:32.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0081939379999999996 2024-04-20T01:48:32.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.752243+0000", 2024-04-20T01:48:32.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3215000000000001e-05 2024-04-20T01:48:32.339 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.339 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.339 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.339 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.0s2 80/30 ECSubWrite(tid=69, reqid=osd.3.0:69, at_version=80'51, trim_to=0'0, roll_forward_to=77'49))", 2024-04-20T01:48:32.339 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:21.740775+0000", 2024-04-20T01:48:32.339 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.528644817, 2024-04-20T01:48:32.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083246099999999996, 2024-04-20T01:48:32.339 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.340 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.340 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740775+0000", 2024-04-20T01:48:32.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740775+0000", 2024-04-20T01:48:32.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740775+0000", 2024-04-20T01:48:32.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:48:32.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740795+0000", 2024-04-20T01:48:32.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9623000000000002e-05 2024-04-20T01:48:32.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740804+0000", 2024-04-20T01:48:32.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3789999999999995e-06 2024-04-20T01:48:32.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740811+0000", 2024-04-20T01:48:32.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0249999999999997e-06 2024-04-20T01:48:32.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740869+0000", 2024-04-20T01:48:32.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8099000000000001e-05 2024-04-20T01:48:32.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740877+0000", 2024-04-20T01:48:32.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4730000000000003e-06 2024-04-20T01:48:32.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.343 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.343 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.749075+0000", 2024-04-20T01:48:32.343 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0081977590000000006 2024-04-20T01:48:32.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.343 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.343 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.749100+0000", 2024-04-20T01:48:32.343 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5443999999999999e-05 2024-04-20T01:48:32.343 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.343 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.343 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.343 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:953 3.ds0 3:b226b74f:::smithi16579500-7:head [delete] snapc 36=[36,35,33,31,28] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e86)", 2024-04-20T01:48:32.344 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:27.764111+0000", 2024-04-20T01:48:32.344 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.505309016, 2024-04-20T01:48:32.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066564759999999997, 2024-04-20T01:48:32.344 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.344 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:32.344 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:32.344 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:32.344 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:32.344 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 953 2024-04-20T01:48:32.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.344 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.344 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.764111+0000", 2024-04-20T01:48:32.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.764111+0000", 2024-04-20T01:48:32.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.764110+0000", 2024-04-20T01:48:32.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:32.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.764117+0000", 2024-04-20T01:48:32.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7309999999999993e-06 2024-04-20T01:48:32.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.764132+0000", 2024-04-20T01:48:32.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4289e-05 2024-04-20T01:48:32.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.764142+0000", 2024-04-20T01:48:32.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0710000000000001e-05 2024-04-20T01:48:32.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.764201+0000", 2024-04-20T01:48:32.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8347999999999998e-05 2024-04-20T01:48:32.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op must wait for map", 2024-04-20T01:48:32.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.764208+0000", 2024-04-20T01:48:32.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4059999999999999e-06 2024-04-20T01:48:32.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.766405+0000", 2024-04-20T01:48:32.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021969889999999999 2024-04-20T01:48:32.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:32.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.766512+0000", 2024-04-20T01:48:32.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000107145 2024-04-20T01:48:32.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.348 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.348 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.770768+0000", 2024-04-20T01:48:32.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042555759999999996 2024-04-20T01:48:32.348 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.348 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.348 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.348 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.348 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.348 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:32.350 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-20T01:48:32.350 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:32.364+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:32.450 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:32.464+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:32.478 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:32.478 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:32.479 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:48:32.479 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:32.489 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-20T01:48:32.551 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:32.566+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:32.597 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-14 --journal-path /var/lib/ceph/osd/ceph-14/journal --op export --pgid 3.5s3 --file /var/log/ceph/exp.3.5s3.14 2024-04-20T01:48:32.624 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:32.624 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:32.624 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 30, 2024-04-20T01:48:32.624 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-20T01:48:32.624 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:32.635 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-20T01:48:32.651 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:32.666+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:32.708 INFO:tasks.rados.rados.0.smithi165.stdout:394: done (6 left) 2024-04-20T01:48:32.708 INFO:tasks.rados.rados.0.smithi165.stdout:395: rollback oid 24 current snap is 30 2024-04-20T01:48:32.708 INFO:tasks.rados.rados.0.smithi165.stdout:rollback oid 24 to 25 2024-04-20T01:48:32.709 INFO:tasks.rados.rados.0.smithi165.stdout:396: setattr oid 46 current snap is 30 2024-04-20T01:48:32.709 INFO:tasks.rados.rados.0.smithi165.stdout:391: expect (ObjNum 9 snap 0 seq_num 9) 2024-04-20T01:48:32.711 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is set 2024-04-20T01:48:32.719 INFO:tasks.rados.rados.0.smithi165.stdout:389: expect (ObjNum 138 snap 23 seq_num 138) 2024-04-20T01:48:32.753 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:32.768+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:32.811 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:32.812 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-20T01:48:32.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-20T01:48:32.812 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-20T01:48:32.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.812 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:10 3.5s0 3:a2e9ac6b:::smithi16579500-4:head [append 0~65536,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-20T01:48:32.812 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.752800+0000", 2024-04-20T01:48:32.812 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 58.072588934999999, 2024-04-20T01:48:32.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011440333, 2024-04-20T01:48:32.812 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.812 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:32.812 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:32.812 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:32.813 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:32.813 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 10 2024-04-20T01:48:32.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.813 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752800+0000", 2024-04-20T01:48:32.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752800+0000", 2024-04-20T01:48:32.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.814 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752799+0000", 2024-04-20T01:48:32.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:32.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.814 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752945+0000", 2024-04-20T01:48:32.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000145761 2024-04-20T01:48:32.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.814 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752955+0000", 2024-04-20T01:48:32.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0229e-05 2024-04-20T01:48:32.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.815 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.815 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752970+0000", 2024-04-20T01:48:32.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5242000000000001e-05 2024-04-20T01:48:32.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.815 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.815 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753046+0000", 2024-04-20T01:48:32.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6218999999999996e-05 2024-04-20T01:48:32.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.815 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:32.815 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753172+0000", 2024-04-20T01:48:32.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012566899999999999 2024-04-20T01:48:32.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753648+0000", 2024-04-20T01:48:32.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00047565500000000001 2024-04-20T01:48:32.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.756671+0000", 2024-04-20T01:48:32.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030231849999999998 2024-04-20T01:48:32.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:32.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764191+0000", 2024-04-20T01:48:32.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007520143 2024-04-20T01:48:32.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764240+0000", 2024-04-20T01:48:32.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9359e-05 2024-04-20T01:48:32.817 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.817 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.817 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.817 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:11 3.5s0 3:a2e9ac6b:::smithi16579500-4:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-20T01:48:32.817 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.752987+0000", 2024-04-20T01:48:32.817 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 58.072402298999997, 2024-04-20T01:48:32.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011858695000000001, 2024-04-20T01:48:32.817 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.817 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:32.818 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:32.818 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:32.818 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:32.818 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 11 2024-04-20T01:48:32.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.818 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752987+0000", 2024-04-20T01:48:32.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752987+0000", 2024-04-20T01:48:32.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752986+0000", 2024-04-20T01:48:32.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:32.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752995+0000", 2024-04-20T01:48:32.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7099999999999996e-06 2024-04-20T01:48:32.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753000+0000", 2024-04-20T01:48:32.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4900000000000002e-06 2024-04-20T01:48:32.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.820 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.820 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753003+0000", 2024-04-20T01:48:32.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4620000000000001e-06 2024-04-20T01:48:32.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.820 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.820 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753974+0000", 2024-04-20T01:48:32.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000971388 2024-04-20T01:48:32.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.820 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:32.820 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754054+0000", 2024-04-20T01:48:32.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9159000000000006e-05 2024-04-20T01:48:32.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.821 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754358+0000", 2024-04-20T01:48:32.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00030434800000000002 2024-04-20T01:48:32.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.821 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.759666+0000", 2024-04-20T01:48:32.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053076169999999997 2024-04-20T01:48:32.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.822 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:32.822 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764756+0000", 2024-04-20T01:48:32.822 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050904139999999997 2024-04-20T01:48:32.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.822 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.822 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764845+0000", 2024-04-20T01:48:32.822 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9488999999999996e-05 2024-04-20T01:48:32.822 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.822 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.822 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.822 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:12 3.5s0 3:a2e9ac6b:::smithi16579500-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-20T01:48:32.823 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.753012+0000", 2024-04-20T01:48:32.823 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 58.072377281999998, 2024-04-20T01:48:32.823 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014201745, 2024-04-20T01:48:32.823 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.823 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:32.823 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:32.823 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:32.823 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:32.823 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 12 2024-04-20T01:48:32.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.823 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.823 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.823 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753012+0000", 2024-04-20T01:48:32.823 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.824 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.824 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753012+0000", 2024-04-20T01:48:32.824 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.824 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.824 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753011+0000", 2024-04-20T01:48:32.824 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:32.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.824 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.824 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753016+0000", 2024-04-20T01:48:32.824 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.989e-06 2024-04-20T01:48:32.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753026+0000", 2024-04-20T01:48:32.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2690000000000005e-06 2024-04-20T01:48:32.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753033+0000", 2024-04-20T01:48:32.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3010000000000001e-06 2024-04-20T01:48:32.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754553+0000", 2024-04-20T01:48:32.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001519947 2024-04-20T01:48:32.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.826 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:32.826 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754638+0000", 2024-04-20T01:48:32.826 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5421999999999996e-05 2024-04-20T01:48:32.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.826 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.826 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.765222+0000", 2024-04-20T01:48:32.826 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010583979 2024-04-20T01:48:32.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.826 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:32.826 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.765288+0000", 2024-04-20T01:48:32.826 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5296999999999995e-05 2024-04-20T01:48:32.827 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.827 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.827 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.767213+0000", 2024-04-20T01:48:32.827 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019259520000000001 2024-04-20T01:48:32.827 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.827 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.827 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.827 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.827 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:64 3.5s0 3:ae9f92b8:::smithi16579500-22:head [append 0~360448,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:32.827 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.781054+0000", 2024-04-20T01:48:32.827 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 58.044334814000003, 2024-04-20T01:48:32.827 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01202491, 2024-04-20T01:48:32.827 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.828 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:32.828 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:32.828 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:32.828 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:32.828 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 64 2024-04-20T01:48:32.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.828 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.828 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781054+0000", 2024-04-20T01:48:32.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781054+0000", 2024-04-20T01:48:32.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781053+0000", 2024-04-20T01:48:32.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:32.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781384+0000", 2024-04-20T01:48:32.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00033060899999999999 2024-04-20T01:48:32.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781394+0000", 2024-04-20T01:48:32.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0016e-05 2024-04-20T01:48:32.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781399+0000", 2024-04-20T01:48:32.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7930000000000003e-06 2024-04-20T01:48:32.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781446+0000", 2024-04-20T01:48:32.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6768999999999999e-05 2024-04-20T01:48:32.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:32.831 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781598+0000", 2024-04-20T01:48:32.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015158500000000001 2024-04-20T01:48:32.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.831 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.831 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.782377+0000", 2024-04-20T01:48:32.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000778975 2024-04-20T01:48:32.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.831 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.831 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784105+0000", 2024-04-20T01:48:32.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017285340000000001 2024-04-20T01:48:32.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:32.832 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793040+0000", 2024-04-20T01:48:32.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0089346440000000003 2024-04-20T01:48:32.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.832 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793079+0000", 2024-04-20T01:48:32.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9239000000000002e-05 2024-04-20T01:48:32.832 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.832 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.832 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.832 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:65 3.5s0 3:ae9f92b8:::smithi16579500-22:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:32.833 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.781409+0000", 2024-04-20T01:48:32.833 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 58.043980214000001, 2024-04-20T01:48:32.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011886404, 2024-04-20T01:48:32.833 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.833 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:32.833 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:32.833 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:32.833 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:32.833 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 65 2024-04-20T01:48:32.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.833 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.833 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.833 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781409+0000", 2024-04-20T01:48:32.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781409+0000", 2024-04-20T01:48:32.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781408+0000", 2024-04-20T01:48:32.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:32.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781411+0000", 2024-04-20T01:48:32.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6309999999999999e-06 2024-04-20T01:48:32.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781416+0000", 2024-04-20T01:48:32.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6550000000000003e-06 2024-04-20T01:48:32.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781423+0000", 2024-04-20T01:48:32.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4540000000000001e-06 2024-04-20T01:48:32.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.782838+0000", 2024-04-20T01:48:32.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001414741 2024-04-20T01:48:32.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.836 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:32.836 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.782921+0000", 2024-04-20T01:48:32.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3018999999999999e-05 2024-04-20T01:48:32.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.836 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.836 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.783284+0000", 2024-04-20T01:48:32.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00036349 2024-04-20T01:48:32.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.836 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.836 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784149+0000", 2024-04-20T01:48:32.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00086487700000000001 2024-04-20T01:48:32.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.837 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:32.837 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793195+0000", 2024-04-20T01:48:32.837 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0090454590000000005 2024-04-20T01:48:32.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.837 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.837 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793295+0000", 2024-04-20T01:48:32.837 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010049 2024-04-20T01:48:32.837 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.837 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.837 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.837 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:66 3.5s0 3:ae9f92b8:::smithi16579500-22:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:32.838 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.781436+0000", 2024-04-20T01:48:32.838 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 58.043952783000002, 2024-04-20T01:48:32.838 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014237204999999999, 2024-04-20T01:48:32.838 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.838 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:32.838 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:32.838 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:32.838 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:32.838 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 66 2024-04-20T01:48:32.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.838 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781436+0000", 2024-04-20T01:48:32.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781436+0000", 2024-04-20T01:48:32.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781436+0000", 2024-04-20T01:48:32.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:32.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781441+0000", 2024-04-20T01:48:32.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3199999999999999e-06 2024-04-20T01:48:32.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.840 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.840 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781446+0000", 2024-04-20T01:48:32.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.544e-06 2024-04-20T01:48:32.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.840 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.840 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781451+0000", 2024-04-20T01:48:32.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5570000000000003e-06 2024-04-20T01:48:32.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.840 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.840 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.783430+0000", 2024-04-20T01:48:32.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019785699999999998 2024-04-20T01:48:32.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:32.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.783482+0000", 2024-04-20T01:48:32.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1780000000000002e-05 2024-04-20T01:48:32.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793826+0000", 2024-04-20T01:48:32.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010344132000000001 2024-04-20T01:48:32.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:32.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793907+0000", 2024-04-20T01:48:32.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1173000000000004e-05 2024-04-20T01:48:32.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.842 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.842 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.795673+0000", 2024-04-20T01:48:32.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001766442 2024-04-20T01:48:32.842 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.842 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.842 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.842 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 47/30 ECSubWrite(tid=247, reqid=client.4643.0:389, at_version=47'39, trim_to=0'0, roll_forward_to=47'35))", 2024-04-20T01:48:32.842 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:48.746964+0000", 2024-04-20T01:48:32.843 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 44.078424589999997, 2024-04-20T01:48:32.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056576650000000001, 2024-04-20T01:48:32.843 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.843 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.843 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746964+0000", 2024-04-20T01:48:32.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746964+0000", 2024-04-20T01:48:32.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.844 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.844 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.746964+0000", 2024-04-20T01:48:32.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:32.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.844 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.844 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747077+0000", 2024-04-20T01:48:32.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113581 2024-04-20T01:48:32.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.844 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.844 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747089+0000", 2024-04-20T01:48:32.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1795e-05 2024-04-20T01:48:32.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.845 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747098+0000", 2024-04-20T01:48:32.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1630000000000002e-06 2024-04-20T01:48:32.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.845 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747143+0000", 2024-04-20T01:48:32.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5286e-05 2024-04-20T01:48:32.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.845 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.747154+0000", 2024-04-20T01:48:32.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0488e-05 2024-04-20T01:48:32.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.846 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.752599+0000", 2024-04-20T01:48:32.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054446670000000003 2024-04-20T01:48:32.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.846 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:48.752622+0000", 2024-04-20T01:48:32.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3521999999999999e-05 2024-04-20T01:48:32.846 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.846 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.846 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.846 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s6 48/30 ECSubWrite(tid=251, reqid=osd.13.0:251, at_version=48'22, trim_to=0'0, roll_forward_to=47'20))", 2024-04-20T01:48:32.847 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.710374+0000", 2024-04-20T01:48:32.847 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 43.115014879999997, 2024-04-20T01:48:32.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069805379999999997, 2024-04-20T01:48:32.847 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.847 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.847 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.847 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710374+0000", 2024-04-20T01:48:32.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.847 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710374+0000", 2024-04-20T01:48:32.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.848 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.848 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710374+0000", 2024-04-20T01:48:32.848 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:32.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.848 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.848 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710395+0000", 2024-04-20T01:48:32.848 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1396e-05 2024-04-20T01:48:32.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.848 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.848 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710417+0000", 2024-04-20T01:48:32.848 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1857000000000001e-05 2024-04-20T01:48:32.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.849 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.849 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.849 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710423+0000", 2024-04-20T01:48:32.849 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1469999999999998e-06 2024-04-20T01:48:32.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.849 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.849 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.849 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710498+0000", 2024-04-20T01:48:32.849 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5072000000000002e-05 2024-04-20T01:48:32.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.849 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.849 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.849 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710509+0000", 2024-04-20T01:48:32.849 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0685e-05 2024-04-20T01:48:32.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.850 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.850 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.850 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717332+0000", 2024-04-20T01:48:32.850 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006823048 2024-04-20T01:48:32.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.850 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.850 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.850 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717355+0000", 2024-04-20T01:48:32.850 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2731e-05 2024-04-20T01:48:32.850 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.850 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.850 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.850 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.851 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 48/30 ECSubWrite(tid=253, reqid=osd.13.0:253, at_version=48'44, trim_to=0'0, roll_forward_to=47'40))", 2024-04-20T01:48:32.851 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.711134+0000", 2024-04-20T01:48:32.851 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 43.114254903000003, 2024-04-20T01:48:32.851 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006623304, 2024-04-20T01:48:32.851 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.851 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.851 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.851 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.851 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711134+0000", 2024-04-20T01:48:32.851 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.851 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711134+0000", 2024-04-20T01:48:32.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.852 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.852 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711134+0000", 2024-04-20T01:48:32.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:32.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.852 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.852 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711155+0000", 2024-04-20T01:48:32.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1673000000000001e-05 2024-04-20T01:48:32.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.852 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.853 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.853 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711166+0000", 2024-04-20T01:48:32.853 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0558e-05 2024-04-20T01:48:32.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.853 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.853 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.853 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711174+0000", 2024-04-20T01:48:32.853 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3359999999999995e-06 2024-04-20T01:48:32.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.853 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.853 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.853 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711257+0000", 2024-04-20T01:48:32.853 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3312999999999993e-05 2024-04-20T01:48:32.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.854 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.854 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711273+0000", 2024-04-20T01:48:32.854 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5701000000000001e-05 2024-04-20T01:48:32.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.854 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.854 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717738+0000", 2024-04-20T01:48:32.854 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064653339999999997 2024-04-20T01:48:32.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.854 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.854 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717757+0000", 2024-04-20T01:48:32.854 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8972999999999999e-05 2024-04-20T01:48:32.855 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.855 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.855 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.855 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 63/30 ECSubWrite(tid=295, reqid=client.4643.0:574, at_version=63'79, trim_to=0'0, roll_forward_to=62'77))", 2024-04-20T01:48:32.855 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:04.795161+0000", 2024-04-20T01:48:32.855 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.030228183999998, 2024-04-20T01:48:32.855 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0075404740000000001, 2024-04-20T01:48:32.855 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.855 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.855 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.855 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795161+0000", 2024-04-20T01:48:32.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795161+0000", 2024-04-20T01:48:32.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795159+0000", 2024-04-20T01:48:32.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:32.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795678+0000", 2024-04-20T01:48:32.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00051905099999999997 2024-04-20T01:48:32.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795694+0000", 2024-04-20T01:48:32.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6076e-05 2024-04-20T01:48:32.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795704+0000", 2024-04-20T01:48:32.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0078000000000001e-05 2024-04-20T01:48:32.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795753+0000", 2024-04-20T01:48:32.858 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9055999999999999e-05 2024-04-20T01:48:32.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.858 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.858 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795763+0000", 2024-04-20T01:48:32.858 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6430000000000007e-06 2024-04-20T01:48:32.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.858 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.858 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.802666+0000", 2024-04-20T01:48:32.858 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069029149999999999 2024-04-20T01:48:32.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.858 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.858 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.802701+0000", 2024-04-20T01:48:32.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5664999999999998e-05 2024-04-20T01:48:32.859 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.859 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.859 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.859 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 63/30 ECSubWrite(tid=296, reqid=client.4643.0:575, at_version=63'80, trim_to=0'0, roll_forward_to=62'77))", 2024-04-20T01:48:32.859 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:04.798493+0000", 2024-04-20T01:48:32.859 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.026895579000001, 2024-04-20T01:48:32.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069235080000000001, 2024-04-20T01:48:32.859 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.859 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.859 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.859 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.860 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798493+0000", 2024-04-20T01:48:32.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.860 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.860 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798493+0000", 2024-04-20T01:48:32.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.860 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.860 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798493+0000", 2024-04-20T01:48:32.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:32.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.860 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798862+0000", 2024-04-20T01:48:32.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00036897900000000002 2024-04-20T01:48:32.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798876+0000", 2024-04-20T01:48:32.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4321999999999999e-05 2024-04-20T01:48:32.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798884+0000", 2024-04-20T01:48:32.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.157e-06 2024-04-20T01:48:32.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798936+0000", 2024-04-20T01:48:32.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1879000000000003e-05 2024-04-20T01:48:32.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798949+0000", 2024-04-20T01:48:32.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3359e-05 2024-04-20T01:48:32.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.805386+0000", 2024-04-20T01:48:32.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064363470000000002 2024-04-20T01:48:32.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.805417+0000", 2024-04-20T01:48:32.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1260999999999998e-05 2024-04-20T01:48:32.863 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.863 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.863 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.863 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:717 3.5s0 3:ae9f92b8:::smithi16579500-22:head [read 0~2129920,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:32.863 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.113707+0000", 2024-04-20T01:48:32.863 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 22.71168175, 2024-04-20T01:48:32.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083649959999999995, 2024-04-20T01:48:32.863 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.863 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:32.864 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:32.864 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:32.864 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:32.864 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 717 2024-04-20T01:48:32.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.864 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113707+0000", 2024-04-20T01:48:32.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113707+0000", 2024-04-20T01:48:32.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113706+0000", 2024-04-20T01:48:32.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:32.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113714+0000", 2024-04-20T01:48:32.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.371e-06 2024-04-20T01:48:32.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113726+0000", 2024-04-20T01:48:32.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1572e-05 2024-04-20T01:48:32.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113732+0000", 2024-04-20T01:48:32.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2870000000000001e-06 2024-04-20T01:48:32.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113785+0000", 2024-04-20T01:48:32.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2648000000000001e-05 2024-04-20T01:48:32.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:32.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113900+0000", 2024-04-20T01:48:32.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011510699999999999 2024-04-20T01:48:32.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.122072+0000", 2024-04-20T01:48:32.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0081722930000000006 2024-04-20T01:48:32.867 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.867 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.867 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.867 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:718 3.5s0 3:ae9f92b8:::smithi16579500-22:head [read 0~2129920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:32.867 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.113742+0000", 2024-04-20T01:48:32.867 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 22.711646679000001, 2024-04-20T01:48:32.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095008990000000001, 2024-04-20T01:48:32.868 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.868 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:32.868 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:32.868 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:32.868 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:32.868 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 718 2024-04-20T01:48:32.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.868 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.868 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.868 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113742+0000", 2024-04-20T01:48:32.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.868 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113742+0000", 2024-04-20T01:48:32.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113742+0000", 2024-04-20T01:48:32.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:32.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113744+0000", 2024-04-20T01:48:32.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4219999999999999e-06 2024-04-20T01:48:32.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113749+0000", 2024-04-20T01:48:32.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.313e-06 2024-04-20T01:48:32.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113752+0000", 2024-04-20T01:48:32.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2449999999999998e-06 2024-04-20T01:48:32.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.117251+0000", 2024-04-20T01:48:32.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034990360000000001 2024-04-20T01:48:32.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:32.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.117402+0000", 2024-04-20T01:48:32.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000151099 2024-04-20T01:48:32.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.123243+0000", 2024-04-20T01:48:32.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058410800000000002 2024-04-20T01:48:32.871 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.871 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.871 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.871 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:719 3.5s0 3:ae9f92b8:::smithi16579500-22:head [read 0~2129920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:32.871 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.113770+0000", 2024-04-20T01:48:32.872 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 22.711619061, 2024-04-20T01:48:32.872 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010243805999999999, 2024-04-20T01:48:32.872 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.872 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:32.872 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:32.872 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:32.872 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:32.872 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 719 2024-04-20T01:48:32.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.872 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.872 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.872 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.872 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113770+0000", 2024-04-20T01:48:32.872 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.873 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.873 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113770+0000", 2024-04-20T01:48:32.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.873 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.873 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113770+0000", 2024-04-20T01:48:32.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:32.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.873 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.873 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113772+0000", 2024-04-20T01:48:32.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1270000000000001e-06 2024-04-20T01:48:32.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113777+0000", 2024-04-20T01:48:32.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4410000000000001e-06 2024-04-20T01:48:32.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113780+0000", 2024-04-20T01:48:32.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.869e-06 2024-04-20T01:48:32.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.120794+0000", 2024-04-20T01:48:32.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070135659999999997 2024-04-20T01:48:32.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:32.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.120909+0000", 2024-04-20T01:48:32.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011497099999999999 2024-04-20T01:48:32.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.124014+0000", 2024-04-20T01:48:32.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00310509 2024-04-20T01:48:32.875 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.875 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.875 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.876 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 75/30 ECSubWrite(tid=357, reqid=osd.13.0:357, at_version=75'127, trim_to=0'0, roll_forward_to=75'123))", 2024-04-20T01:48:32.876 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:16.739742+0000", 2024-04-20T01:48:32.876 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.085647137999999, 2024-04-20T01:48:32.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006963365, 2024-04-20T01:48:32.876 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.876 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.876 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739742+0000", 2024-04-20T01:48:32.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739742+0000", 2024-04-20T01:48:32.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.877 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.877 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739741+0000", 2024-04-20T01:48:32.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:32.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.877 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.877 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739764+0000", 2024-04-20T01:48:32.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2824e-05 2024-04-20T01:48:32.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.877 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.877 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739776+0000", 2024-04-20T01:48:32.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1806e-05 2024-04-20T01:48:32.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739785+0000", 2024-04-20T01:48:32.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1500000000000005e-06 2024-04-20T01:48:32.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739842+0000", 2024-04-20T01:48:32.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7034000000000003e-05 2024-04-20T01:48:32.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739853+0000", 2024-04-20T01:48:32.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0556e-05 2024-04-20T01:48:32.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.746679+0000", 2024-04-20T01:48:32.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068263880000000001 2024-04-20T01:48:32.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.746705+0000", 2024-04-20T01:48:32.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6219000000000001e-05 2024-04-20T01:48:32.879 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.879 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.880 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.880 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s6 75/30 ECSubWrite(tid=363, reqid=client.4643.0:790, at_version=75'30, trim_to=0'0, roll_forward_to=75'28))", 2024-04-20T01:48:32.880 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:16.766298+0000", 2024-04-20T01:48:32.880 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.059091296999998, 2024-04-20T01:48:32.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0078250469999999999, 2024-04-20T01:48:32.880 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.880 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.880 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.766298+0000", 2024-04-20T01:48:32.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.766298+0000", 2024-04-20T01:48:32.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.766297+0000", 2024-04-20T01:48:32.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:32.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.767936+0000", 2024-04-20T01:48:32.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016393880000000001 2024-04-20T01:48:32.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.767947+0000", 2024-04-20T01:48:32.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0061000000000001e-05 2024-04-20T01:48:32.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.767953+0000", 2024-04-20T01:48:32.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.083e-06 2024-04-20T01:48:32.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.767982+0000", 2024-04-20T01:48:32.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9203000000000001e-05 2024-04-20T01:48:32.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.883 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.767988+0000", 2024-04-20T01:48:32.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9669999999999998e-06 2024-04-20T01:48:32.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.883 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.774100+0000", 2024-04-20T01:48:32.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061125729999999996 2024-04-20T01:48:32.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.883 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.884 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.774123+0000", 2024-04-20T01:48:32.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2404000000000001e-05 2024-04-20T01:48:32.884 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.884 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.884 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.884 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.as1 80/30 ECSubWrite(tid=44, reqid=client.4643.0:907, at_version=80'23, trim_to=0'0, roll_forward_to=66'20))", 2024-04-20T01:48:32.884 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:21.742360+0000", 2024-04-20T01:48:32.884 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 11.083028825, 2024-04-20T01:48:32.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068772989999999999, 2024-04-20T01:48:32.884 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.884 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.884 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.742360+0000", 2024-04-20T01:48:32.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.742360+0000", 2024-04-20T01:48:32.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.742359+0000", 2024-04-20T01:48:32.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:32.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.886 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.886 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.742381+0000", 2024-04-20T01:48:32.886 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1467999999999998e-05 2024-04-20T01:48:32.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.886 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.886 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.742395+0000", 2024-04-20T01:48:32.886 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3939e-05 2024-04-20T01:48:32.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.886 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.886 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.742404+0000", 2024-04-20T01:48:32.886 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8810000000000002e-06 2024-04-20T01:48:32.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.742456+0000", 2024-04-20T01:48:32.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2024000000000002e-05 2024-04-20T01:48:32.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.742467+0000", 2024-04-20T01:48:32.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1106e-05 2024-04-20T01:48:32.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.749208+0000", 2024-04-20T01:48:32.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0067413129999999996 2024-04-20T01:48:32.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.888 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.888 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.749237+0000", 2024-04-20T01:48:32.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9439000000000001e-05 2024-04-20T01:48:32.888 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.888 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.888 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.888 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s6 86/30 ECSubWrite(tid=392, reqid=client.4643.0:960, at_version=86'41, trim_to=0'0, roll_forward_to=80'39))", 2024-04-20T01:48:32.888 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:27.784783+0000", 2024-04-20T01:48:32.888 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.0406056010000002, 2024-04-20T01:48:32.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070335490000000001, 2024-04-20T01:48:32.888 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.888 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.889 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.889 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.889 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.784783+0000", 2024-04-20T01:48:32.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.889 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.889 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.784783+0000", 2024-04-20T01:48:32.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.889 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.889 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.784783+0000", 2024-04-20T01:48:32.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:32.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.784839+0000", 2024-04-20T01:48:32.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6625000000000001e-05 2024-04-20T01:48:32.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.784874+0000", 2024-04-20T01:48:32.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4425999999999999e-05 2024-04-20T01:48:32.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.784900+0000", 2024-04-20T01:48:32.891 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6333e-05 2024-04-20T01:48:32.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.891 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.891 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.891 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.785044+0000", 2024-04-20T01:48:32.891 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014331700000000001 2024-04-20T01:48:32.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.891 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.891 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.891 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.785055+0000", 2024-04-20T01:48:32.891 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1483e-05 2024-04-20T01:48:32.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.891 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.891 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.891 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.791786+0000", 2024-04-20T01:48:32.892 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0067313520000000003 2024-04-20T01:48:32.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.892 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.892 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.892 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.791817+0000", 2024-04-20T01:48:32.892 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0608999999999998e-05 2024-04-20T01:48:32.892 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.892 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.892 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.892 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.892 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s6 86/30 ECSubWrite(tid=393, reqid=client.4643.0:961, at_version=86'42, trim_to=0'0, roll_forward_to=80'39))", 2024-04-20T01:48:32.892 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:27.786036+0000", 2024-04-20T01:48:32.892 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.0393527779999996, 2024-04-20T01:48:32.892 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006157226, 2024-04-20T01:48:32.893 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.893 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.893 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.893 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.893 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.893 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.786036+0000", 2024-04-20T01:48:32.893 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.893 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.893 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.893 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.786036+0000", 2024-04-20T01:48:32.893 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.893 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.894 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.894 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.786035+0000", 2024-04-20T01:48:32.894 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:32.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.894 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.894 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.894 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.786057+0000", 2024-04-20T01:48:32.894 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1311999999999999e-05 2024-04-20T01:48:32.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.894 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.894 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.894 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.786073+0000", 2024-04-20T01:48:32.894 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6728e-05 2024-04-20T01:48:32.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.894 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.895 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.895 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.786087+0000", 2024-04-20T01:48:32.895 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4136e-05 2024-04-20T01:48:32.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.895 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.895 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.895 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.786140+0000", 2024-04-20T01:48:32.895 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2845e-05 2024-04-20T01:48:32.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.895 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.895 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.895 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.786151+0000", 2024-04-20T01:48:32.895 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0937e-05 2024-04-20T01:48:32.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.895 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.896 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.896 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.792171+0000", 2024-04-20T01:48:32.896 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060194319999999999 2024-04-20T01:48:32.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.896 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.896 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.896 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.792193+0000", 2024-04-20T01:48:32.896 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2775000000000001e-05 2024-04-20T01:48:32.896 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.896 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.896 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.896 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.896 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.bs5 87/30 ECSubWrite(tid=332, reqid=client.4643.0:967, at_version=0'0, trim_to=0'0, roll_forward_to=87'164))", 2024-04-20T01:48:32.896 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:28.789649+0000", 2024-04-20T01:48:32.897 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.0357400300000004, 2024-04-20T01:48:32.897 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061559609999999997, 2024-04-20T01:48:32.897 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:32.897 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:32.897 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:32.897 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.897 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:32.897 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:28.789649+0000", 2024-04-20T01:48:32.897 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.897 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.897 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:32.897 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:28.789649+0000", 2024-04-20T01:48:32.897 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:32.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.898 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.898 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:32.898 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:28.789648+0000", 2024-04-20T01:48:32.898 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:32.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.898 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.898 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:32.898 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:28.789662+0000", 2024-04-20T01:48:32.898 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3991e-05 2024-04-20T01:48:32.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.898 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.898 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:32.898 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:28.789674+0000", 2024-04-20T01:48:32.898 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2228e-05 2024-04-20T01:48:32.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.899 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.899 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:32.899 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:28.789684+0000", 2024-04-20T01:48:32.899 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2779999999999999e-06 2024-04-20T01:48:32.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.899 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.899 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:32.899 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:28.789732+0000", 2024-04-20T01:48:32.899 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8593e-05 2024-04-20T01:48:32.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.899 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.899 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:32.899 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:28.789743+0000", 2024-04-20T01:48:32.899 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0946e-05 2024-04-20T01:48:32.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.900 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.900 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:32.900 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:28.795783+0000", 2024-04-20T01:48:32.900 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006039928 2024-04-20T01:48:32.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:32.900 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:32.900 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:32.900 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:28.795805+0000", 2024-04-20T01:48:32.900 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1843000000000001e-05 2024-04-20T01:48:32.900 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.900 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.900 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.900 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:32.900 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:32.900 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:32.902 INFO:tasks.rados.rados.0.smithi165.stdout:393: expect (ObjNum 136 snap 22 seq_num 136) 2024-04-20T01:48:32.902 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-20T01:48:32.903 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:32.916+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:33.002 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:33.017+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:33.035 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:33.036 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:33.036 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-04-20T01:48:33.036 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:33.047 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-20T01:48:33.104 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:33.118+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:33.158 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 44 v 106 (ObjNum 57 snap 2 seq_num 57) dirty exists 2024-04-20T01:48:33.158 INFO:tasks.rados.rados.0.smithi165.stdout:392: finishing write tid 1 to smithi16579500-49 2024-04-20T01:48:33.158 INFO:tasks.rados.rados.0.smithi165.stdout:392: finishing write tid 2 to smithi16579500-49 2024-04-20T01:48:33.158 INFO:tasks.rados.rados.0.smithi165.stdout:390: expect (ObjNum 158 snap 30 seq_num 158) 2024-04-20T01:48:33.178 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:33.178 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:33.178 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:48:33.178 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:48:33.178 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:33.189 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-20T01:48:33.232 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:33.246+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:33.241 INFO:tasks.rados.rados.0.smithi165.stdout:395: finishing rollback tid 1 to smithi16579500-24 2024-04-20T01:48:33.241 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 24 v 0 (ObjNum 19809 snap 0 seq_num 2584256256) dirty dne 2024-04-20T01:48:33.241 INFO:tasks.rados.rados.0.smithi165.stdout:392: finishing write tid 3 to smithi16579500-49 2024-04-20T01:48:33.241 INFO:tasks.rados.rados.0.smithi165.stdout:392: oid 49 updating version 0 to 108 2024-04-20T01:48:33.241 INFO:tasks.rados.rados.0.smithi165.stdout:392: oid 49 version 108 is already newer than 107 2024-04-20T01:48:33.241 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 49 v 108 (ObjNum 159 snap 30 seq_num 159) dirty exists 2024-04-20T01:48:33.241 INFO:tasks.rados.rados.0.smithi165.stdout:392: left oid 49 (ObjNum 159 snap 30 seq_num 159) 2024-04-20T01:48:33.241 INFO:tasks.rados.rados.0.smithi165.stdout:387: done (7 left) 2024-04-20T01:48:33.241 INFO:tasks.rados.rados.0.smithi165.stdout:389: done (6 left) 2024-04-20T01:48:33.241 INFO:tasks.rados.rados.0.smithi165.stdout:390: done (5 left) 2024-04-20T01:48:33.241 INFO:tasks.rados.rados.0.smithi165.stdout:391: done (4 left) 2024-04-20T01:48:33.241 INFO:tasks.rados.rados.0.smithi165.stdout:392: done (3 left) 2024-04-20T01:48:33.241 INFO:tasks.rados.rados.0.smithi165.stdout:393: done (2 left) 2024-04-20T01:48:33.241 INFO:tasks.rados.rados.0.smithi165.stdout:395: done (1 left) 2024-04-20T01:48:33.242 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 42 current snap is 30 2024-04-20T01:48:33.242 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 46 v 75 (ObjNum 152 snap 27 seq_num 152) dirty exists 2024-04-20T01:48:33.242 INFO:tasks.rados.rados.0.smithi165.stdout:397: seq_num 160 ranges {5160960=294912} 2024-04-20T01:48:33.270 INFO:tasks.rados.rados.0.smithi165.stdout:397: writing smithi16579500-42 from 5160960 to 5455872 tid 1 2024-04-20T01:48:33.270 INFO:tasks.rados.rados.0.smithi165.stdout:396: done (1 left) 2024-04-20T01:48:33.270 INFO:tasks.rados.rados.0.smithi165.stdout:append oid 12 current snap is 30 2024-04-20T01:48:33.270 INFO:tasks.rados.rados.0.smithi165.stdout:398: seq_num 161 ranges {0=311296} 2024-04-20T01:48:33.273 INFO:tasks.rados.rados.0.smithi165.stdout:398: writing smithi16579500-12 from 0 to 311296 tid 1 2024-04-20T01:48:33.273 INFO:tasks.rados.rados.0.smithi165.stdout:399: rmattr oid 6 current snap is 30 2024-04-20T01:48:33.273 INFO:tasks.rados.rados.0.smithi165.stdout: waiting on 3 2024-04-20T01:48:33.278 INFO:tasks.rados.rados.0.smithi165.stdout:397: finishing write tid 1 to smithi16579500-42 2024-04-20T01:48:33.278 INFO:tasks.rados.rados.0.smithi165.stdout:397: finishing write tid 2 to smithi16579500-42 2024-04-20T01:48:33.279 INFO:tasks.rados.rados.0.smithi165.stdout:397: finishing write tid 3 to smithi16579500-42 2024-04-20T01:48:33.279 INFO:tasks.rados.rados.0.smithi165.stdout:397: oid 42 updating version 0 to 66 2024-04-20T01:48:33.280 INFO:tasks.rados.rados.0.smithi165.stdout:397: oid 42 version 66 is already newer than 65 2024-04-20T01:48:33.280 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 42 v 66 (ObjNum 160 snap 30 seq_num 160) dirty exists 2024-04-20T01:48:33.280 INFO:tasks.rados.rados.0.smithi165.stdout:397: left oid 42 (ObjNum 160 snap 30 seq_num 160) 2024-04-20T01:48:33.280 INFO:tasks.rados.rados.0.smithi165.stdout:397: done (2 left) 2024-04-20T01:48:33.280 INFO:tasks.rados.rados.0.smithi165.stdout: waiting on 2 2024-04-20T01:48:33.280 INFO:tasks.rados.rados.0.smithi165.stdout:398: finishing write tid 1 to smithi16579500-12 2024-04-20T01:48:33.280 INFO:tasks.rados.rados.0.smithi165.stdout:398: finishing write tid 2 to smithi16579500-12 2024-04-20T01:48:33.281 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 6 v 109 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-04-20T01:48:33.281 INFO:tasks.rados.rados.0.smithi165.stdout:399: done (1 left) 2024-04-20T01:48:33.281 INFO:tasks.rados.rados.0.smithi165.stdout: waiting on 1 2024-04-20T01:48:33.284 INFO:tasks.rados.rados.0.smithi165.stdout:398: finishing write tid 3 to smithi16579500-12 2024-04-20T01:48:33.284 INFO:tasks.rados.rados.0.smithi165.stdout:398: oid 12 updating version 0 to 101 2024-04-20T01:48:33.284 INFO:tasks.rados.rados.0.smithi165.stdout:398: oid 12 version 101 is already newer than 100 2024-04-20T01:48:33.284 INFO:tasks.rados.rados.0.smithi165.stdout:update_object_version oid 12 v 101 (ObjNum 161 snap 30 seq_num 161) dirty exists 2024-04-20T01:48:33.284 INFO:tasks.rados.rados.0.smithi165.stdout:398: left oid 12 (ObjNum 161 snap 30 seq_num 161) 2024-04-20T01:48:33.284 INFO:tasks.rados.rados.0.smithi165.stdout:398: done (0 left) 2024-04-20T01:48:33.286 INFO:tasks.rados.rados.0.smithi165.stderr:0 errors. 2024-04-20T01:48:33.286 INFO:tasks.rados.rados.0.smithi165.stderr: 2024-04-20T01:48:33.289 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T01:48:33.324 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:33.324 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:48:33.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:48:33.324 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:48:33.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.324 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:234 3.0s0 3:0c1ed9f1:::smithi16579500-46:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:33.324 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.787453+0000", 2024-04-20T01:48:33.324 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.549841123, 2024-04-20T01:48:33.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016795529, 2024-04-20T01:48:33.325 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.325 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:33.325 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:33.325 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:33.325 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:33.325 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 234 2024-04-20T01:48:33.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.325 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787453+0000", 2024-04-20T01:48:33.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787453+0000", 2024-04-20T01:48:33.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787451+0000", 2024-04-20T01:48:33.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:33.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787462+0000", 2024-04-20T01:48:33.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1083e-05 2024-04-20T01:48:33.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787483+0000", 2024-04-20T01:48:33.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1656999999999999e-05 2024-04-20T01:48:33.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787503+0000", 2024-04-20T01:48:33.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9623000000000002e-05 2024-04-20T01:48:33.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787595+0000", 2024-04-20T01:48:33.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2417000000000006e-05 2024-04-20T01:48:33.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:33.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787604+0000", 2024-04-20T01:48:33.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.952e-06 2024-04-20T01:48:33.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.791584+0000", 2024-04-20T01:48:33.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039792250000000003 2024-04-20T01:48:33.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:33.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.791699+0000", 2024-04-20T01:48:33.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115692 2024-04-20T01:48:33.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.792530+0000", 2024-04-20T01:48:33.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083077900000000002 2024-04-20T01:48:33.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.798777+0000", 2024-04-20T01:48:33.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062472379999999996 2024-04-20T01:48:33.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:33.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.804175+0000", 2024-04-20T01:48:33.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005397158 2024-04-20T01:48:33.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.330 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.804248+0000", 2024-04-20T01:48:33.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3682000000000004e-05 2024-04-20T01:48:33.330 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.330 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.330 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.330 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:235 3.0s0 3:0c1ed9f1:::smithi16579500-46:head [rollback 5] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:33.330 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.787517+0000", 2024-04-20T01:48:33.330 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.549777294000002, 2024-04-20T01:48:33.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019202061999999999, 2024-04-20T01:48:33.330 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.330 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:33.330 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:33.331 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:33.331 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:33.331 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 235 2024-04-20T01:48:33.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.331 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787517+0000", 2024-04-20T01:48:33.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787517+0000", 2024-04-20T01:48:33.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787516+0000", 2024-04-20T01:48:33.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:33.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787521+0000", 2024-04-20T01:48:33.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2599999999999996e-06 2024-04-20T01:48:33.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787534+0000", 2024-04-20T01:48:33.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2717e-05 2024-04-20T01:48:33.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787538+0000", 2024-04-20T01:48:33.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3540000000000002e-06 2024-04-20T01:48:33.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787673+0000", 2024-04-20T01:48:33.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013480799999999999 2024-04-20T01:48:33.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:33.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787678+0000", 2024-04-20T01:48:33.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5229999999999997e-06 2024-04-20T01:48:33.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793075+0000", 2024-04-20T01:48:33.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053977549999999997 2024-04-20T01:48:33.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:33.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793190+0000", 2024-04-20T01:48:33.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114775 2024-04-20T01:48:33.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793760+0000", 2024-04-20T01:48:33.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00056945599999999998 2024-04-20T01:48:33.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799198+0000", 2024-04-20T01:48:33.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0054385589999999999 2024-04-20T01:48:33.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:33.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806653+0000", 2024-04-20T01:48:33.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0074547739999999999 2024-04-20T01:48:33.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806719+0000", 2024-04-20T01:48:33.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5732e-05 2024-04-20T01:48:33.335 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.336 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.336 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.336 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:236 3.0s0 3:0c1ed9f1:::smithi16579500-46:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:33.336 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:48:33.336 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.549717680000001, 2024-04-20T01:48:33.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019273109, 2024-04-20T01:48:33.336 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.336 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:33.336 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:33.336 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:33.336 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:33.336 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 236 2024-04-20T01:48:33.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.336 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:48:33.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:48:33.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:48:33.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:33.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787579+0000", 2024-04-20T01:48:33.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6380000000000002e-06 2024-04-20T01:48:33.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787586+0000", 2024-04-20T01:48:33.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.81e-06 2024-04-20T01:48:33.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787592+0000", 2024-04-20T01:48:33.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5180000000000004e-06 2024-04-20T01:48:33.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787705+0000", 2024-04-20T01:48:33.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011387000000000001 2024-04-20T01:48:33.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:33.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787712+0000", 2024-04-20T01:48:33.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6229999999999999e-06 2024-04-20T01:48:33.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794765+0000", 2024-04-20T01:48:33.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0070530519999999998 2024-04-20T01:48:33.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:33.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794873+0000", 2024-04-20T01:48:33.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107861 2024-04-20T01:48:33.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795365+0000", 2024-04-20T01:48:33.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049192300000000001 2024-04-20T01:48:33.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799201+0000", 2024-04-20T01:48:33.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003836454 2024-04-20T01:48:33.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:33.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806808+0000", 2024-04-20T01:48:33.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076070119999999998 2024-04-20T01:48:33.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806849+0000", 2024-04-20T01:48:33.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0852999999999997e-05 2024-04-20T01:48:33.341 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.341 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.341 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.341 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:443 3.0s0 3:06dd530f:::smithi16579500-30:head [append 0~720896] snapc 15=[15,14,13,b] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:33.342 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.714463+0000", 2024-04-20T01:48:33.342 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 38.622830571999998, 2024-04-20T01:48:33.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014549549, 2024-04-20T01:48:33.342 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.342 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:33.342 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:33.342 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:33.342 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:33.342 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 443 2024-04-20T01:48:33.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.342 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714463+0000", 2024-04-20T01:48:33.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714463+0000", 2024-04-20T01:48:33.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714461+0000", 2024-04-20T01:48:33.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:33.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714916+0000", 2024-04-20T01:48:33.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00045487499999999998 2024-04-20T01:48:33.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714934+0000", 2024-04-20T01:48:33.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7586000000000001e-05 2024-04-20T01:48:33.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714962+0000", 2024-04-20T01:48:33.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7976000000000001e-05 2024-04-20T01:48:33.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.715010+0000", 2024-04-20T01:48:33.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8535999999999997e-05 2024-04-20T01:48:33.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:33.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.715016+0000", 2024-04-20T01:48:33.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8309999999999997e-06 2024-04-20T01:48:33.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717469+0000", 2024-04-20T01:48:33.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024527529999999998 2024-04-20T01:48:33.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:33.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717568+0000", 2024-04-20T01:48:33.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8708000000000004e-05 2024-04-20T01:48:33.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718717+0000", 2024-04-20T01:48:33.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0011493510000000001 2024-04-20T01:48:33.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724443+0000", 2024-04-20T01:48:33.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057254280000000003 2024-04-20T01:48:33.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:33.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.728929+0000", 2024-04-20T01:48:33.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044864919999999999 2024-04-20T01:48:33.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.729013+0000", 2024-04-20T01:48:33.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3770999999999995e-05 2024-04-20T01:48:33.347 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.347 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.347 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.347 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:444 3.0s0 3:06dd530f:::smithi16579500-30:head [append 0~442368] snapc 15=[15,14,13,b] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:33.347 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.716986+0000", 2024-04-20T01:48:33.347 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 38.620308307999998, 2024-04-20T01:48:33.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014745457999999999, 2024-04-20T01:48:33.348 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.348 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:33.348 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:33.348 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:33.348 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:33.348 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 444 2024-04-20T01:48:33.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.348 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.716986+0000", 2024-04-20T01:48:33.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.716986+0000", 2024-04-20T01:48:33.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.716984+0000", 2024-04-20T01:48:33.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:33.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717309+0000", 2024-04-20T01:48:33.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032479500000000001 2024-04-20T01:48:33.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717321+0000", 2024-04-20T01:48:33.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2177e-05 2024-04-20T01:48:33.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717330+0000", 2024-04-20T01:48:33.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0399999999999998e-06 2024-04-20T01:48:33.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.720236+0000", 2024-04-20T01:48:33.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002905992 2024-04-20T01:48:33.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:33.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.720367+0000", 2024-04-20T01:48:33.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000130488 2024-04-20T01:48:33.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721328+0000", 2024-04-20T01:48:33.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00096118699999999996 2024-04-20T01:48:33.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.729125+0000", 2024-04-20T01:48:33.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077972429999999997 2024-04-20T01:48:33.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:33.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731687+0000", 2024-04-20T01:48:33.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00256201 2024-04-20T01:48:33.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731731+0000", 2024-04-20T01:48:33.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4005999999999998e-05 2024-04-20T01:48:33.352 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.352 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.352 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.352 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:445 3.0s0 3:06dd530f:::smithi16579500-30:head [append 0~327680] snapc 15=[15,14,13,b] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:33.352 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.718952+0000", 2024-04-20T01:48:33.352 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 38.618341643999997, 2024-04-20T01:48:33.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012930084, 2024-04-20T01:48:33.353 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.353 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:33.353 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:33.353 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:33.353 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:33.353 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 445 2024-04-20T01:48:33.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.353 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718952+0000", 2024-04-20T01:48:33.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718952+0000", 2024-04-20T01:48:33.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718950+0000", 2024-04-20T01:48:33.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:33.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719225+0000", 2024-04-20T01:48:33.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00027498300000000002 2024-04-20T01:48:33.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719241+0000", 2024-04-20T01:48:33.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6036999999999999e-05 2024-04-20T01:48:33.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719251+0000", 2024-04-20T01:48:33.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.027e-05 2024-04-20T01:48:33.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722012+0000", 2024-04-20T01:48:33.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027604309999999998 2024-04-20T01:48:33.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:33.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722121+0000", 2024-04-20T01:48:33.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109207 2024-04-20T01:48:33.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722804+0000", 2024-04-20T01:48:33.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00068250299999999999 2024-04-20T01:48:33.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.729128+0000", 2024-04-20T01:48:33.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0063243079999999998 2024-04-20T01:48:33.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:33.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731817+0000", 2024-04-20T01:48:33.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0026888239999999998 2024-04-20T01:48:33.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731882+0000", 2024-04-20T01:48:33.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5499000000000004e-05 2024-04-20T01:48:33.357 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.357 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.357 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.357 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:446 3.0s0 3:06dd530f:::smithi16579500-30:head [setxattr _header (57) in=7b] snapc 15=[15,14,13,b] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:33.357 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.719261+0000", 2024-04-20T01:48:33.357 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 38.618033306999997, 2024-04-20T01:48:33.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012992831, 2024-04-20T01:48:33.358 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.358 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:33.358 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:33.358 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:33.358 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:33.358 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 446 2024-04-20T01:48:33.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.358 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719261+0000", 2024-04-20T01:48:33.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719261+0000", 2024-04-20T01:48:33.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719260+0000", 2024-04-20T01:48:33.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:33.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719264+0000", 2024-04-20T01:48:33.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6179999999999997e-06 2024-04-20T01:48:33.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719271+0000", 2024-04-20T01:48:33.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9949999999999999e-06 2024-04-20T01:48:33.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719277+0000", 2024-04-20T01:48:33.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6999999999999996e-06 2024-04-20T01:48:33.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723396+0000", 2024-04-20T01:48:33.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0041186599999999997 2024-04-20T01:48:33.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:33.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723458+0000", 2024-04-20T01:48:33.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2786000000000005e-05 2024-04-20T01:48:33.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723867+0000", 2024-04-20T01:48:33.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040805300000000002 2024-04-20T01:48:33.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731913+0000", 2024-04-20T01:48:33.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080461330000000005 2024-04-20T01:48:33.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:33.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731951+0000", 2024-04-20T01:48:33.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8707000000000001e-05 2024-04-20T01:48:33.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.732253+0000", 2024-04-20T01:48:33.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030198999999999997 2024-04-20T01:48:33.362 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.362 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.362 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.362 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:447 3.0s0 3:06dd530f:::smithi16579500-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:33.362 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.719282+0000", 2024-04-20T01:48:33.363 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 38.61801174, 2024-04-20T01:48:33.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01483317, 2024-04-20T01:48:33.363 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.363 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:33.363 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:33.363 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:33.363 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:33.363 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 447 2024-04-20T01:48:33.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.363 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719282+0000", 2024-04-20T01:48:33.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719282+0000", 2024-04-20T01:48:33.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719282+0000", 2024-04-20T01:48:33.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:33.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719294+0000", 2024-04-20T01:48:33.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2747e-05 2024-04-20T01:48:33.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719302+0000", 2024-04-20T01:48:33.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0960000000000001e-06 2024-04-20T01:48:33.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719308+0000", 2024-04-20T01:48:33.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.066e-06 2024-04-20T01:48:33.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724217+0000", 2024-04-20T01:48:33.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0049083549999999997 2024-04-20T01:48:33.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:33.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724333+0000", 2024-04-20T01:48:33.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011662099999999999 2024-04-20T01:48:33.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.732307+0000", 2024-04-20T01:48:33.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0079733059999999994 2024-04-20T01:48:33.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:33.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.732409+0000", 2024-04-20T01:48:33.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010193599999999999 2024-04-20T01:48:33.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.734115+0000", 2024-04-20T01:48:33.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0017065520000000001 2024-04-20T01:48:33.367 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.367 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.367 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.368 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:781 3.0s0 3:06dd530f:::smithi16579500-30:head [copy-from ver 10] snapc 2a=[2a,28] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-04-20T01:48:33.368 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.717982+0000", 2024-04-20T01:48:33.368 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 16.619311978999999, 2024-04-20T01:48:33.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.036464862000000001, 2024-04-20T01:48:33.368 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.368 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:33.368 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:33.368 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:33.368 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:33.368 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 781 2024-04-20T01:48:33.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.368 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.717982+0000", 2024-04-20T01:48:33.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.717982+0000", 2024-04-20T01:48:33.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.717980+0000", 2024-04-20T01:48:33.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:33.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.717990+0000", 2024-04-20T01:48:33.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9790000000000005e-06 2024-04-20T01:48:33.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718003+0000", 2024-04-20T01:48:33.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.342e-05 2024-04-20T01:48:33.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718014+0000", 2024-04-20T01:48:33.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0258000000000001e-05 2024-04-20T01:48:33.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718122+0000", 2024-04-20T01:48:33.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108208 2024-04-20T01:48:33.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:33.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718130+0000", 2024-04-20T01:48:33.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2120000000000003e-06 2024-04-20T01:48:33.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722371+0000", 2024-04-20T01:48:33.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042406049999999997 2024-04-20T01:48:33.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:33.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722507+0000", 2024-04-20T01:48:33.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000136136 2024-04-20T01:48:33.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.734773+0000", 2024-04-20T01:48:33.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012266325999999999 2024-04-20T01:48:33.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.742440+0000", 2024-04-20T01:48:33.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076668489999999999 2024-04-20T01:48:33.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:33.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.754352+0000", 2024-04-20T01:48:33.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011911880999999999 2024-04-20T01:48:33.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.754447+0000", 2024-04-20T01:48:33.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4827999999999994e-05 2024-04-20T01:48:33.373 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.373 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.373 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.373 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:782 3.0s0 3:06dd530f:::smithi16579500-30:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e75)", 2024-04-20T01:48:33.373 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.718032+0000", 2024-04-20T01:48:33.374 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 16.619261983000001, 2024-04-20T01:48:33.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020311961999999999, 2024-04-20T01:48:33.374 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.374 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:33.374 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:33.374 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:33.374 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:33.374 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 782 2024-04-20T01:48:33.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.374 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718032+0000", 2024-04-20T01:48:33.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718032+0000", 2024-04-20T01:48:33.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718031+0000", 2024-04-20T01:48:33.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:33.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718038+0000", 2024-04-20T01:48:33.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0990000000000001e-06 2024-04-20T01:48:33.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718047+0000", 2024-04-20T01:48:33.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2520000000000005e-06 2024-04-20T01:48:33.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718059+0000", 2024-04-20T01:48:33.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1965e-05 2024-04-20T01:48:33.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718210+0000", 2024-04-20T01:48:33.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150519 2024-04-20T01:48:33.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:33.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718216+0000", 2024-04-20T01:48:33.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3539999999999996e-06 2024-04-20T01:48:33.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722660+0000", 2024-04-20T01:48:33.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.004443687 2024-04-20T01:48:33.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:33.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722692+0000", 2024-04-20T01:48:33.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1761000000000003e-05 2024-04-20T01:48:33.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.738085+0000", 2024-04-20T01:48:33.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015393333 2024-04-20T01:48:33.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:33.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.738203+0000", 2024-04-20T01:48:33.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011827100000000001 2024-04-20T01:48:33.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.738344+0000", 2024-04-20T01:48:33.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014056600000000001 2024-04-20T01:48:33.379 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.379 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.379 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.379 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs1 75/30 ECSubWrite(tid=65, reqid=osd.2.0:65, at_version=75'50, trim_to=0'0, roll_forward_to=73'46))", 2024-04-20T01:48:33.379 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.720810+0000", 2024-04-20T01:48:33.379 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 16.616483643999999, 2024-04-20T01:48:33.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013232499, 2024-04-20T01:48:33.379 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.379 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720810+0000", 2024-04-20T01:48:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720810+0000", 2024-04-20T01:48:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720809+0000", 2024-04-20T01:48:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720840+0000", 2024-04-20T01:48:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0954999999999998e-05 2024-04-20T01:48:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720853+0000", 2024-04-20T01:48:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2748000000000001e-05 2024-04-20T01:48:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720863+0000", 2024-04-20T01:48:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4409999999999999e-06 2024-04-20T01:48:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722832+0000", 2024-04-20T01:48:33.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019689899999999999 2024-04-20T01:48:33.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722841+0000", 2024-04-20T01:48:33.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8320000000000001e-06 2024-04-20T01:48:33.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.734017+0000", 2024-04-20T01:48:33.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011175314 2024-04-20T01:48:33.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.734043+0000", 2024-04-20T01:48:33.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5962999999999999e-05 2024-04-20T01:48:33.383 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.383 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.383 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.383 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s1 75/30 ECSubWrite(tid=351, reqid=osd.13.0:351, at_version=75'121, trim_to=0'0, roll_forward_to=73'119))", 2024-04-20T01:48:33.383 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.721483+0000", 2024-04-20T01:48:33.383 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 16.615811134000001, 2024-04-20T01:48:33.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012358233, 2024-04-20T01:48:33.383 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.383 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.384 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721483+0000", 2024-04-20T01:48:33.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721483+0000", 2024-04-20T01:48:33.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721482+0000", 2024-04-20T01:48:33.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:33.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721517+0000", 2024-04-20T01:48:33.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4520999999999997e-05 2024-04-20T01:48:33.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721530+0000", 2024-04-20T01:48:33.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2928000000000001e-05 2024-04-20T01:48:33.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721540+0000", 2024-04-20T01:48:33.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0276e-05 2024-04-20T01:48:33.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721654+0000", 2024-04-20T01:48:33.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113496 2024-04-20T01:48:33.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721665+0000", 2024-04-20T01:48:33.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1796e-05 2024-04-20T01:48:33.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.733806+0000", 2024-04-20T01:48:33.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012140656999999999 2024-04-20T01:48:33.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.733841+0000", 2024-04-20T01:48:33.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4820999999999997e-05 2024-04-20T01:48:33.387 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.387 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.387 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.387 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s1 75/30 ECSubWrite(tid=352, reqid=osd.13.0:352, at_version=75'123, trim_to=0'0, roll_forward_to=73'119))", 2024-04-20T01:48:33.387 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.723743+0000", 2024-04-20T01:48:33.387 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 16.613550473, 2024-04-20T01:48:33.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013134636999999999, 2024-04-20T01:48:33.388 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.388 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.388 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723743+0000", 2024-04-20T01:48:33.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723743+0000", 2024-04-20T01:48:33.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723742+0000", 2024-04-20T01:48:33.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:33.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723769+0000", 2024-04-20T01:48:33.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6086999999999999e-05 2024-04-20T01:48:33.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723782+0000", 2024-04-20T01:48:33.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3675e-05 2024-04-20T01:48:33.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723794+0000", 2024-04-20T01:48:33.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1965e-05 2024-04-20T01:48:33.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725734+0000", 2024-04-20T01:48:33.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019396439999999999 2024-04-20T01:48:33.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725743+0000", 2024-04-20T01:48:33.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8659999999999995e-06 2024-04-20T01:48:33.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736860+0000", 2024-04-20T01:48:33.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011117311 2024-04-20T01:48:33.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736878+0000", 2024-04-20T01:48:33.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7938999999999999e-05 2024-04-20T01:48:33.391 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.391 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.391 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.391 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs3 75/30 ECSubWrite(tid=112, reqid=osd.14.0:112, at_version=75'91, trim_to=0'0, roll_forward_to=74'89))", 2024-04-20T01:48:33.391 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.724406+0000", 2024-04-20T01:48:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 16.612887785000002, 2024-04-20T01:48:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012721514999999999, 2024-04-20T01:48:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724406+0000", 2024-04-20T01:48:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724406+0000", 2024-04-20T01:48:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724406+0000", 2024-04-20T01:48:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724444+0000", 2024-04-20T01:48:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8659999999999999e-05 2024-04-20T01:48:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724455+0000", 2024-04-20T01:48:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0468999999999999e-05 2024-04-20T01:48:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724469+0000", 2024-04-20T01:48:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4498000000000001e-05 2024-04-20T01:48:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724894+0000", 2024-04-20T01:48:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00042428799999999999 2024-04-20T01:48:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724905+0000", 2024-04-20T01:48:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1415e-05 2024-04-20T01:48:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.737103+0000", 2024-04-20T01:48:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012198293000000001 2024-04-20T01:48:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.737128+0000", 2024-04-20T01:48:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4178000000000001e-05 2024-04-20T01:48:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs3 75/30 ECSubWrite(tid=113, reqid=osd.14.0:113, at_version=75'93, trim_to=0'0, roll_forward_to=74'89))", 2024-04-20T01:48:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.724486+0000", 2024-04-20T01:48:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 16.612808092000002, 2024-04-20T01:48:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012679546999999999, 2024-04-20T01:48:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724486+0000", 2024-04-20T01:48:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724486+0000", 2024-04-20T01:48:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724485+0000", 2024-04-20T01:48:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724545+0000", 2024-04-20T01:48:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9892000000000002e-05 2024-04-20T01:48:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724556+0000", 2024-04-20T01:48:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0706999999999999e-05 2024-04-20T01:48:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724564+0000", 2024-04-20T01:48:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5129999999999996e-06 2024-04-20T01:48:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725568+0000", 2024-04-20T01:48:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001003547 2024-04-20T01:48:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725593+0000", 2024-04-20T01:48:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4604e-05 2024-04-20T01:48:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.737155+0000", 2024-04-20T01:48:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011562915 2024-04-20T01:48:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.737165+0000", 2024-04-20T01:48:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7909999999999999e-06 2024-04-20T01:48:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s3 75/30 ECSubWrite(tid=101, reqid=osd.5.0:101, at_version=75'82, trim_to=0'0, roll_forward_to=68'78))", 2024-04-20T01:48:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.724669+0000", 2024-04-20T01:48:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 16.612625158, 2024-04-20T01:48:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012107346, 2024-04-20T01:48:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724669+0000", 2024-04-20T01:48:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724669+0000", 2024-04-20T01:48:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724668+0000", 2024-04-20T01:48:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724693+0000", 2024-04-20T01:48:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4289000000000001e-05 2024-04-20T01:48:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724707+0000", 2024-04-20T01:48:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4489e-05 2024-04-20T01:48:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724715+0000", 2024-04-20T01:48:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.174e-06 2024-04-20T01:48:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725729+0000", 2024-04-20T01:48:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0010136870000000001 2024-04-20T01:48:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725739+0000", 2024-04-20T01:48:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0064000000000001e-05 2024-04-20T01:48:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736749+0000", 2024-04-20T01:48:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011009778 2024-04-20T01:48:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736776+0000", 2024-04-20T01:48:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7220999999999999e-05 2024-04-20T01:48:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs3 80/30 ECSubWrite(tid=132, reqid=osd.14.0:132, at_version=80'111, trim_to=0'0, roll_forward_to=77'109))", 2024-04-20T01:48:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:21.729572+0000", 2024-04-20T01:48:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.607721567, 2024-04-20T01:48:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020941090999999998, 2024-04-20T01:48:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729572+0000", 2024-04-20T01:48:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729572+0000", 2024-04-20T01:48:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729572+0000", 2024-04-20T01:48:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729602+0000", 2024-04-20T01:48:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0369e-05 2024-04-20T01:48:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729615+0000", 2024-04-20T01:48:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3158e-05 2024-04-20T01:48:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729626+0000", 2024-04-20T01:48:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0056e-05 2024-04-20T01:48:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729699+0000", 2024-04-20T01:48:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3857999999999997e-05 2024-04-20T01:48:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729712+0000", 2024-04-20T01:48:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3088000000000001e-05 2024-04-20T01:48:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.750484+0000", 2024-04-20T01:48:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020771511999999999 2024-04-20T01:48:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.750513+0000", 2024-04-20T01:48:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9363e-05 2024-04-20T01:48:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.es5 80/30 ECSubWrite(tid=112, reqid=osd.11.0:112, at_version=80'88, trim_to=0'0, roll_forward_to=77'86))", 2024-04-20T01:48:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:21.733917+0000", 2024-04-20T01:48:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.603376663000001, 2024-04-20T01:48:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015930045, 2024-04-20T01:48:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.733917+0000", 2024-04-20T01:48:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.733917+0000", 2024-04-20T01:48:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.733917+0000", 2024-04-20T01:48:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.733943+0000", 2024-04-20T01:48:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6764e-05 2024-04-20T01:48:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.733956+0000", 2024-04-20T01:48:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2537999999999999e-05 2024-04-20T01:48:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.733966+0000", 2024-04-20T01:48:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0145e-05 2024-04-20T01:48:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.734023+0000", 2024-04-20T01:48:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7231000000000001e-05 2024-04-20T01:48:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.734035+0000", 2024-04-20T01:48:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1163e-05 2024-04-20T01:48:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.749818+0000", 2024-04-20T01:48:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015783452 2024-04-20T01:48:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.749847+0000", 2024-04-20T01:48:33.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9235e-05 2024-04-20T01:48:33.413 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.413 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.413 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.413 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s5 80/57 ECSubWrite(tid=205, reqid=osd.8.0:205, at_version=80'147, trim_to=0'0, roll_forward_to=79'145))", 2024-04-20T01:48:33.413 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:21.737407+0000", 2024-04-20T01:48:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.599887048999999, 2024-04-20T01:48:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013126919000000001, 2024-04-20T01:48:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737407+0000", 2024-04-20T01:48:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737407+0000", 2024-04-20T01:48:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737406+0000", 2024-04-20T01:48:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737434+0000", 2024-04-20T01:48:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7623999999999998e-05 2024-04-20T01:48:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737448+0000", 2024-04-20T01:48:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3811e-05 2024-04-20T01:48:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737458+0000", 2024-04-20T01:48:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0426e-05 2024-04-20T01:48:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737568+0000", 2024-04-20T01:48:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109221 2024-04-20T01:48:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737579+0000", 2024-04-20T01:48:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1895e-05 2024-04-20T01:48:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.750521+0000", 2024-04-20T01:48:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012941998 2024-04-20T01:48:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.750534+0000", 2024-04-20T01:48:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2267e-05 2024-04-20T01:48:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s5 80/57 ECSubWrite(tid=206, reqid=osd.8.0:206, at_version=80'149, trim_to=0'0, roll_forward_to=79'145))", 2024-04-20T01:48:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:21.738507+0000", 2024-04-20T01:48:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.598786848, 2024-04-20T01:48:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014548627999999999, 2024-04-20T01:48:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.738507+0000", 2024-04-20T01:48:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.738507+0000", 2024-04-20T01:48:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.738506+0000", 2024-04-20T01:48:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.738540+0000", 2024-04-20T01:48:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4316000000000001e-05 2024-04-20T01:48:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.738553+0000", 2024-04-20T01:48:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2755000000000001e-05 2024-04-20T01:48:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.738564+0000", 2024-04-20T01:48:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.066e-05 2024-04-20T01:48:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.739602+0000", 2024-04-20T01:48:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001038186 2024-04-20T01:48:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.739612+0000", 2024-04-20T01:48:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0145e-05 2024-04-20T01:48:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.753027+0000", 2024-04-20T01:48:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013415152 2024-04-20T01:48:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.753056+0000", 2024-04-20T01:48:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8393000000000001e-05 2024-04-20T01:48:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.422 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:33.422 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:33.337+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:33.423 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-20T01:48:33.424 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:33.438+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:33.525 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:33.539+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:33.593 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-04-20T01:48:33.593 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:33.603 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-20T01:48:33.646 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:33.660+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:33.665 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:48:33.665 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":91,"fsid":"9c2f64c1-aa35-4337-9f5c-081b9e3a2938","created":"2024-04-20T01:47:02.934241+0000","modified":"2024-04-20T01:48:32.709860+0000","last_up_change":"2024-04-20T01:48:31.779231+0000","last_in_change":"2024-04-20T01:47:07.894719+0000","flags":"noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5806080,"flags_set":["nodeep-scrub","noscrub","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T01:47:15.195181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.95162487030029297,"average_primary_affinity":0.94999998807907104,"average_primary_affinity_weighted":0.94999998807907104}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-20T01:47:18.448004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":0.68999999761581421,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":0.95162487030029297,"average_primary_affinity":0.94999998807907104,"average_primary_affinity_weighted":0.94999998807907104}},{"pool":3,"pool_name":"unique_pool_0","create_time":"2024-04-20T01:47:30.734962+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":3,"size":7,"min_size":5,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":59,"snap_epoch":91,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"shecprofile","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":16384,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"3903b88f-ff2b-4252-9480-92b701b1e59b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":43518},{"type":"v1","addr":"172.21.15.44:6825","nonce":43518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":43518},{"type":"v1","addr":"172.21.15.44:6827","nonce":43518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":43518},{"type":"v1","addr":"172.21.15.44:6831","nonce":43518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":43518},{"type":"v1","addr":"172.21.15.44:6829","nonce":43518}]},"public_addr":"172.21.15.44:6825/43518","cluster_addr":"172.21.15.44:6827/43518","heartbeat_back_addr":"172.21.15.44:6831/43518","heartbeat_front_addr":"172.21.15.44:6829/43518","state":["exists","up"]},{"osd":1,"uuid":"c5ff0175-b4f0-40a5-86c7-7a0a351bf070","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":74361},{"type":"v1","addr":"172.21.15.121:6817","nonce":74361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":74361},{"type":"v1","addr":"172.21.15.121:6819","nonce":74361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":74361},{"type":"v1","addr":"172.21.15.121:6823","nonce":74361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":74361},{"type":"v1","addr":"172.21.15.121:6821","nonce":74361}]},"public_addr":"172.21.15.121:6817/74361","cluster_addr":"172.21.15.121:6819/74361","heartbeat_back_addr":"172.21.15.121:6823/74361","heartbeat_front_addr":"172.21.15.121:6821/74361","state":["exists","up"]},{"osd":2,"uuid":"7af325c1-36b5-420c-a2a9-44c00a002977","up":1,"in":1,"weight":1,"primary_affinity":0.2259979248046875,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6809","nonce":43108},{"type":"v1","addr":"172.21.15.123:6811","nonce":43108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":43108},{"type":"v1","addr":"172.21.15.123:6816","nonce":43108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":43108},{"type":"v1","addr":"172.21.15.123:6823","nonce":43108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":43108},{"type":"v1","addr":"172.21.15.123:6821","nonce":43108}]},"public_addr":"172.21.15.123:6811/43108","cluster_addr":"172.21.15.123:6816/43108","heartbeat_back_addr":"172.21.15.123:6823/43108","heartbeat_front_addr":"172.21.15.123:6821/43108","state":["exists","up"]},{"osd":3,"uuid":"99334f43-83ea-4c3f-b91d-ae72c8361b8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":74545},{"type":"v1","addr":"172.21.15.165:6825","nonce":74545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":74545},{"type":"v1","addr":"172.21.15.165:6827","nonce":74545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":74545},{"type":"v1","addr":"172.21.15.165:6831","nonce":74545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":74545},{"type":"v1","addr":"172.21.15.165:6829","nonce":74545}]},"public_addr":"172.21.15.165:6825/74545","cluster_addr":"172.21.15.165:6827/74545","heartbeat_back_addr":"172.21.15.165:6831/74545","heartbeat_front_addr":"172.21.15.165:6829/74545","state":["exists","up"]},{"osd":4,"uuid":"8764d799-af67-4e6c-889f-9fdd91201cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":43517},{"type":"v1","addr":"172.21.15.44:6811","nonce":43517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":43517},{"type":"v1","addr":"172.21.15.44:6815","nonce":43517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":43517},{"type":"v1","addr":"172.21.15.44:6823","nonce":43517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6817","nonce":43517},{"type":"v1","addr":"172.21.15.44:6819","nonce":43517}]},"public_addr":"172.21.15.44:6811/43517","cluster_addr":"172.21.15.44:6815/43517","heartbeat_back_addr":"172.21.15.44:6823/43517","heartbeat_front_addr":"172.21.15.44:6819/43517","state":["exists","up"]},{"osd":5,"uuid":"2556b2fa-4dcc-479c-a379-6f9fe96e9c77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":74354},{"type":"v1","addr":"172.21.15.121:6801","nonce":74354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":74354},{"type":"v1","addr":"172.21.15.121:6803","nonce":74354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":74354},{"type":"v1","addr":"172.21.15.121:6807","nonce":74354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":74354},{"type":"v1","addr":"172.21.15.121:6805","nonce":74354}]},"public_addr":"172.21.15.121:6801/74354","cluster_addr":"172.21.15.121:6803/74354","heartbeat_back_addr":"172.21.15.121:6807/74354","heartbeat_front_addr":"172.21.15.121:6805/74354","state":["exists","up"]},{"osd":6,"uuid":"baeaae3a-fff4-4ccc-a08a-ff32db4531ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":43107},{"type":"v1","addr":"172.21.15.123:6801","nonce":43107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":43107},{"type":"v1","addr":"172.21.15.123:6803","nonce":43107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":43107},{"type":"v1","addr":"172.21.15.123:6807","nonce":43107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":43107},{"type":"v1","addr":"172.21.15.123:6805","nonce":43107}]},"public_addr":"172.21.15.123:6801/43107","cluster_addr":"172.21.15.123:6803/43107","heartbeat_back_addr":"172.21.15.123:6807/43107","heartbeat_front_addr":"172.21.15.123:6805/43107","state":["exists","up"]},{"osd":7,"uuid":"0019f005-3e62-4d9d-b334-f1d347a484ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":74543},{"type":"v1","addr":"172.21.15.165:6801","nonce":74543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":74543},{"type":"v1","addr":"172.21.15.165:6803","nonce":74543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":74543},{"type":"v1","addr":"172.21.15.165:6807","nonce":74543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":74543},{"type":"v1","addr":"172.21.15.165:6805","nonce":74543}]},"public_addr":"172.21.15.165:6801/74543","cluster_addr":"172.21.15.165:6803/74543","heartbeat_back_addr":"172.21.15.165:6807/74543","heartbeat_front_addr":"172.21.15.165:6805/74543","state":["exists","up"]},{"osd":8,"uuid":"6f5200a3-8980-4241-a6fc-ef604fa612b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":43515},{"type":"v1","addr":"172.21.15.44:6810","nonce":43515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":43515},{"type":"v1","addr":"172.21.15.44:6814","nonce":43515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":43515},{"type":"v1","addr":"172.21.15.44:6822","nonce":43515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":43515},{"type":"v1","addr":"172.21.15.44:6818","nonce":43515}]},"public_addr":"172.21.15.44:6810/43515","cluster_addr":"172.21.15.44:6814/43515","heartbeat_back_addr":"172.21.15.44:6822/43515","heartbeat_front_addr":"172.21.15.44:6818/43515","state":["exists","up"]},{"osd":9,"uuid":"29cf9aaa-3a99-4605-92f4-64f1f6a1728d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":74379},{"type":"v1","addr":"172.21.15.121:6825","nonce":74379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":74379},{"type":"v1","addr":"172.21.15.121:6827","nonce":74379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":74379},{"type":"v1","addr":"172.21.15.121:6831","nonce":74379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":74379},{"type":"v1","addr":"172.21.15.121:6829","nonce":74379}]},"public_addr":"172.21.15.121:6825/74379","cluster_addr":"172.21.15.121:6827/74379","heartbeat_back_addr":"172.21.15.121:6831/74379","heartbeat_front_addr":"172.21.15.121:6829/74379","state":["exists","up"]},{"osd":10,"uuid":"705906c4-1514-4754-ac40-45226a837180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":43109},{"type":"v1","addr":"172.21.15.123:6810","nonce":43109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":43109},{"type":"v1","addr":"172.21.15.123:6813","nonce":43109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":43109},{"type":"v1","addr":"172.21.15.123:6820","nonce":43109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":43109},{"type":"v1","addr":"172.21.15.123:6817","nonce":43109}]},"public_addr":"172.21.15.123:6810/43109","cluster_addr":"172.21.15.123:6813/43109","heartbeat_back_addr":"172.21.15.123:6820/43109","heartbeat_front_addr":"172.21.15.123:6817/43109","state":["exists","up"]},{"osd":11,"uuid":"b45f2645-72d1-4d72-89e9-2212f5aa074c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":74544},{"type":"v1","addr":"172.21.15.165:6809","nonce":74544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":74544},{"type":"v1","addr":"172.21.15.165:6811","nonce":74544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":74544},{"type":"v1","addr":"172.21.15.165:6815","nonce":74544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":74544},{"type":"v1","addr":"172.21.15.165:6813","nonce":74544}]},"public_addr":"172.21.15.165:6809/74544","cluster_addr":"172.21.15.165:6811/74544","heartbeat_back_addr":"172.21.15.165:6815/74544","heartbeat_front_addr":"172.21.15.165:6813/74544","state":["exists","up"]},{"osd":12,"uuid":"3ea410b9-a368-467b-a179-2e3aba562d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":43516},{"type":"v1","addr":"172.21.15.44:6801","nonce":43516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":43516},{"type":"v1","addr":"172.21.15.44:6803","nonce":43516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":43516},{"type":"v1","addr":"172.21.15.44:6807","nonce":43516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":43516},{"type":"v1","addr":"172.21.15.44:6805","nonce":43516}]},"public_addr":"172.21.15.44:6801/43516","cluster_addr":"172.21.15.44:6803/43516","heartbeat_back_addr":"172.21.15.44:6807/43516","heartbeat_front_addr":"172.21.15.44:6805/43516","state":["exists","up"]},{"osd":13,"uuid":"15a5154b-e7cb-4b57-8468-dcb048d93f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":74355},{"type":"v1","addr":"172.21.15.121:6809","nonce":74355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":74355},{"type":"v1","addr":"172.21.15.121:6811","nonce":74355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":74355},{"type":"v1","addr":"172.21.15.121:6815","nonce":74355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":74355},{"type":"v1","addr":"172.21.15.121:6813","nonce":74355}]},"public_addr":"172.21.15.121:6809/74355","cluster_addr":"172.21.15.121:6811/74355","heartbeat_back_addr":"172.21.15.121:6815/74355","heartbeat_front_addr":"172.21.15.121:6813/74355","state":["exists","up"]},{"osd":14,"uuid":"23b53da2-f848-4e76-9632-4d9effd5b430","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":90,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":43110},{"type":"v1","addr":"172.21.15.123:6825","nonce":43110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":43110},{"type":"v1","addr":"172.21.15.123:6827","nonce":43110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":43110},{"type":"v1","addr":"172.21.15.123:6831","nonce":43110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":43110},{"type":"v1","addr":"172.21.15.123:6829","nonce":43110}]},"public_addr":"172.21.15.123:6825/43110","cluster_addr":"172.21.15.123:6827/43110","heartbeat_back_addr":"172.21.15.123:6831/43110","heartbeat_front_addr":"172.21.15.123:6829/43110","state":["exists"]},{"osd":15,"uuid":"546efc0f-82a3-47d2-aeec-2d849eb1faa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":74562},{"type":"v1","addr":"172.21.15.165:6817","nonce":74562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":74562},{"type":"v1","addr":"172.21.15.165:6819","nonce":74562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":74562},{"type":"v1","addr":"172.21.15.165:6823","nonce":74562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":74562},{"type":"v1","addr":"172.21.15.165:6821","nonce":74562}]},"public_addr":"172.21.15.165:6817/74562","cluster_addr":"172.21.15.165:6819/74562","heartbeat_back_addr":"172.21.15.165:6823/74562","heartbeat_front_addr":"172.21.15.165:6821/74562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.124928+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:19.984157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.008326+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:15.114260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:25.217793+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:25.078524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.032367+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:26.485365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:28.281278+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:23.704764+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:28.498783+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:14.370592+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:28.932481+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:29.119303+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:47:13.128995+0000","dead_epoch":90},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:17.320410+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.4","mappings":[{"from":3,"to":6},{"from":15,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":40,"length":1},{"begin":51,"length":1},{"begin":54,"length":1},{"begin":56,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":54,"length":1}]}],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":42,"length":1},{"begin":45,"length":3}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T01:48:33.681 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T01:48:33.689 INFO:teuthology.orchestra.run.smithi123.stderr:Exporting 3.5s3 info 3.5s3( v 77'33 (0'0,77'33] local-lis/les=30/31 n=3 ec=30/30 lis/c=30/30 les/c/f=31/31/0 sis=30) 2024-04-20T01:48:33.695 INFO:teuthology.orchestra.run.smithi123.stderr:Read 3#3:a2e9ac6b:::smithi16579500-4:head# 2024-04-20T01:48:33.697 INFO:teuthology.orchestra.run.smithi123.stderr:Read 3#3:ae9f92b8:::smithi16579500-22:28# 2024-04-20T01:48:33.698 INFO:teuthology.orchestra.run.smithi123.stderr:Read 3#3:ae9f92b8:::smithi16579500-22:head# 2024-04-20T01:48:33.698 INFO:teuthology.orchestra.run.smithi123.stderr:Export successful 2024-04-20T01:48:33.731 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:33.731 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:33.731 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:48:33.731 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:48:33.732 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:33.741 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-20T01:48:33.764 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:33.779+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:33.828 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:33.843+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:33.913 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:33.913 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:48:33.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:48:33.914 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:48:33.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.914 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:289 3.9s0 3:94399c02:::smithi16579500-3:head [copy-from ver 6] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:33.914 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:43.807269+0000", 2024-04-20T01:48:33.914 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 50.119696871000002, 2024-04-20T01:48:33.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016170202000000002, 2024-04-20T01:48:33.914 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.914 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:33.914 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:33.914 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:33.914 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:33.914 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 289 2024-04-20T01:48:33.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.915 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807269+0000", 2024-04-20T01:48:33.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807269+0000", 2024-04-20T01:48:33.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807267+0000", 2024-04-20T01:48:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807280+0000", 2024-04-20T01:48:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3227999999999999e-05 2024-04-20T01:48:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807303+0000", 2024-04-20T01:48:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2991000000000001e-05 2024-04-20T01:48:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807318+0000", 2024-04-20T01:48:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4321999999999999e-05 2024-04-20T01:48:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807385+0000", 2024-04-20T01:48:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7403000000000005e-05 2024-04-20T01:48:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807394+0000", 2024-04-20T01:48:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7010000000000002e-06 2024-04-20T01:48:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.809246+0000", 2024-04-20T01:48:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0018521 2024-04-20T01:48:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.809314+0000", 2024-04-20T01:48:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7818999999999996e-05 2024-04-20T01:48:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.817661+0000", 2024-04-20T01:48:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0083471710000000005 2024-04-20T01:48:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.819085+0000", 2024-04-20T01:48:33.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0014239739999999999 2024-04-20T01:48:33.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:33.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.823350+0000", 2024-04-20T01:48:33.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042652990000000002 2024-04-20T01:48:33.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.823439+0000", 2024-04-20T01:48:33.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9424000000000003e-05 2024-04-20T01:48:33.919 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.919 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.920 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.920 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:290 3.9s0 3:94399c02:::smithi16579500-3:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:33.920 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:43.807329+0000", 2024-04-20T01:48:33.920 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 50.11963695, 2024-04-20T01:48:33.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011459461000000001, 2024-04-20T01:48:33.920 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.920 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:33.920 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:33.920 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:33.920 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:33.920 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 290 2024-04-20T01:48:33.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.921 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807329+0000", 2024-04-20T01:48:33.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807329+0000", 2024-04-20T01:48:33.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807329+0000", 2024-04-20T01:48:33.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:33.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807332+0000", 2024-04-20T01:48:33.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.349e-06 2024-04-20T01:48:33.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807343+0000", 2024-04-20T01:48:33.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1270999999999999e-05 2024-04-20T01:48:33.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807347+0000", 2024-04-20T01:48:33.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-04-20T01:48:33.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807453+0000", 2024-04-20T01:48:33.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106162 2024-04-20T01:48:33.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:33.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807460+0000", 2024-04-20T01:48:33.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9809999999999997e-06 2024-04-20T01:48:33.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.809441+0000", 2024-04-20T01:48:33.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019806060000000002 2024-04-20T01:48:33.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:33.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.809484+0000", 2024-04-20T01:48:33.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3019000000000003e-05 2024-04-20T01:48:33.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.818614+0000", 2024-04-20T01:48:33.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091302699999999994 2024-04-20T01:48:33.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:33.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.818713+0000", 2024-04-20T01:48:33.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8776000000000004e-05 2024-04-20T01:48:33.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.818789+0000", 2024-04-20T01:48:33.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5944000000000003e-05 2024-04-20T01:48:33.925 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.925 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.925 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 47/30 ECSubWrite(tid=38, reqid=osd.14.0:38, at_version=47'31, trim_to=0'0, roll_forward_to=46'29))", 2024-04-20T01:48:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.711869+0000", 2024-04-20T01:48:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 45.215097061999998, 2024-04-20T01:48:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016896644999999998, 2024-04-20T01:48:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711869+0000", 2024-04-20T01:48:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711869+0000", 2024-04-20T01:48:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711868+0000", 2024-04-20T01:48:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711907+0000", 2024-04-20T01:48:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8828000000000002e-05 2024-04-20T01:48:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711927+0000", 2024-04-20T01:48:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9656e-05 2024-04-20T01:48:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711937+0000", 2024-04-20T01:48:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0277e-05 2024-04-20T01:48:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711991+0000", 2024-04-20T01:48:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3980000000000002e-05 2024-04-20T01:48:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.712002+0000", 2024-04-20T01:48:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719e-05 2024-04-20T01:48:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.728747+0000", 2024-04-20T01:48:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016745466000000001 2024-04-20T01:48:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.728766+0000", 2024-04-20T01:48:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8607999999999999e-05 2024-04-20T01:48:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 47/30 ECSubWrite(tid=39, reqid=osd.14.0:39, at_version=47'33, trim_to=0'0, roll_forward_to=46'29))", 2024-04-20T01:48:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.713021+0000", 2024-04-20T01:48:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 45.213944855000001, 2024-04-20T01:48:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020945089, 2024-04-20T01:48:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713021+0000", 2024-04-20T01:48:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713021+0000", 2024-04-20T01:48:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713020+0000", 2024-04-20T01:48:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713046+0000", 2024-04-20T01:48:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5554e-05 2024-04-20T01:48:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713064+0000", 2024-04-20T01:48:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7635999999999999e-05 2024-04-20T01:48:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713074+0000", 2024-04-20T01:48:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9529999999999994e-06 2024-04-20T01:48:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713140+0000", 2024-04-20T01:48:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6235999999999994e-05 2024-04-20T01:48:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713155+0000", 2024-04-20T01:48:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5647000000000001e-05 2024-04-20T01:48:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733953+0000", 2024-04-20T01:48:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020797936 2024-04-20T01:48:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733966+0000", 2024-04-20T01:48:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3067999999999999e-05 2024-04-20T01:48:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s1 47/30 ECSubWrite(tid=30, reqid=osd.3.0:30, at_version=47'19, trim_to=0'0, roll_forward_to=44'17))", 2024-04-20T01:48:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.713663+0000", 2024-04-20T01:48:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 45.213303095000001, 2024-04-20T01:48:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0202712, 2024-04-20T01:48:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713663+0000", 2024-04-20T01:48:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713663+0000", 2024-04-20T01:48:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713662+0000", 2024-04-20T01:48:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713693+0000", 2024-04-20T01:48:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1031999999999997e-05 2024-04-20T01:48:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713712+0000", 2024-04-20T01:48:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8615999999999999e-05 2024-04-20T01:48:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713726+0000", 2024-04-20T01:48:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4477e-05 2024-04-20T01:48:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713779+0000", 2024-04-20T01:48:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3090999999999997e-05 2024-04-20T01:48:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713791+0000", 2024-04-20T01:48:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1921e-05 2024-04-20T01:48:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733906+0000", 2024-04-20T01:48:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020114522999999999 2024-04-20T01:48:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733934+0000", 2024-04-20T01:48:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8357999999999999e-05 2024-04-20T01:48:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 47/30 ECSubWrite(tid=39, reqid=osd.5.0:39, at_version=47'27, trim_to=0'0, roll_forward_to=46'25))", 2024-04-20T01:48:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.718432+0000", 2024-04-20T01:48:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 45.208534438000001, 2024-04-20T01:48:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015468671, 2024-04-20T01:48:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718432+0000", 2024-04-20T01:48:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718432+0000", 2024-04-20T01:48:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718431+0000", 2024-04-20T01:48:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718455+0000", 2024-04-20T01:48:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3085e-05 2024-04-20T01:48:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718471+0000", 2024-04-20T01:48:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6172999999999999e-05 2024-04-20T01:48:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718480+0000", 2024-04-20T01:48:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5719999999999992e-06 2024-04-20T01:48:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718592+0000", 2024-04-20T01:48:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111517 2024-04-20T01:48:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718602+0000", 2024-04-20T01:48:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.064e-05 2024-04-20T01:48:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733869+0000", 2024-04-20T01:48:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015266798 2024-04-20T01:48:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733900+0000", 2024-04-20T01:48:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.116e-05 2024-04-20T01:48:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 47/30 ECSubWrite(tid=40, reqid=osd.5.0:40, at_version=47'29, trim_to=0'0, roll_forward_to=46'25))", 2024-04-20T01:48:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.719277+0000", 2024-04-20T01:48:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 45.207688742000002, 2024-04-20T01:48:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017279452000000001, 2024-04-20T01:48:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719277+0000", 2024-04-20T01:48:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719277+0000", 2024-04-20T01:48:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719277+0000", 2024-04-20T01:48:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719306+0000", 2024-04-20T01:48:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9771999999999999e-05 2024-04-20T01:48:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719322+0000", 2024-04-20T01:48:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6035000000000001e-05 2024-04-20T01:48:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719331+0000", 2024-04-20T01:48:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1130000000000007e-06 2024-04-20T01:48:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.728810+0000", 2024-04-20T01:48:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094789090000000006 2024-04-20T01:48:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.728819+0000", 2024-04-20T01:48:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8890000000000008e-06 2024-04-20T01:48:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.736530+0000", 2024-04-20T01:48:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077106129999999998 2024-04-20T01:48:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.736557+0000", 2024-04-20T01:48:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6974999999999998e-05 2024-04-20T01:48:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 47/30 ECSubWrite(tid=240, reqid=osd.13.0:240, at_version=47'33, trim_to=0'0, roll_forward_to=44'31))", 2024-04-20T01:48:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.722777+0000", 2024-04-20T01:48:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 45.204189028000002, 2024-04-20T01:48:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008649894, 2024-04-20T01:48:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722777+0000", 2024-04-20T01:48:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722777+0000", 2024-04-20T01:48:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722777+0000", 2024-04-20T01:48:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722805+0000", 2024-04-20T01:48:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7881e-05 2024-04-20T01:48:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722820+0000", 2024-04-20T01:48:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5387e-05 2024-04-20T01:48:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722831+0000", 2024-04-20T01:48:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813e-05 2024-04-20T01:48:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722915+0000", 2024-04-20T01:48:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3578000000000005e-05 2024-04-20T01:48:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722929+0000", 2024-04-20T01:48:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4256999999999999e-05 2024-04-20T01:48:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.731402+0000", 2024-04-20T01:48:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008472785 2024-04-20T01:48:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.731427+0000", 2024-04-20T01:48:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5355e-05 2024-04-20T01:48:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 47/30 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=47'35, trim_to=0'0, roll_forward_to=44'31))", 2024-04-20T01:48:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.723066+0000", 2024-04-20T01:48:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 45.203900281999999, 2024-04-20T01:48:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010874728, 2024-04-20T01:48:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723066+0000", 2024-04-20T01:48:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723066+0000", 2024-04-20T01:48:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723065+0000", 2024-04-20T01:48:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723089+0000", 2024-04-20T01:48:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3529000000000001e-05 2024-04-20T01:48:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723101+0000", 2024-04-20T01:48:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2143e-05 2024-04-20T01:48:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723110+0000", 2024-04-20T01:48:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9809999999999992e-06 2024-04-20T01:48:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.725642+0000", 2024-04-20T01:48:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002532337 2024-04-20T01:48:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.725655+0000", 2024-04-20T01:48:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2707e-05 2024-04-20T01:48:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733911+0000", 2024-04-20T01:48:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082559369999999997 2024-04-20T01:48:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733941+0000", 2024-04-20T01:48:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9798000000000002e-05 2024-04-20T01:48:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:396 3.9s0 3:9a2fbd15:::smithi16579500-1:head [copy-from ver 32] snapc d=[c,b] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-20T01:48:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.746088+0000", 2024-04-20T01:48:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 45.180878104000001, 2024-04-20T01:48:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01355685, 2024-04-20T01:48:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 396 2024-04-20T01:48:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746088+0000", 2024-04-20T01:48:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746088+0000", 2024-04-20T01:48:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746086+0000", 2024-04-20T01:48:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746094+0000", 2024-04-20T01:48:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.272e-06 2024-04-20T01:48:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746109+0000", 2024-04-20T01:48:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5563999999999999e-05 2024-04-20T01:48:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746131+0000", 2024-04-20T01:48:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1515999999999999e-05 2024-04-20T01:48:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746178+0000", 2024-04-20T01:48:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7472000000000002e-05 2024-04-20T01:48:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746261+0000", 2024-04-20T01:48:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2412999999999999e-05 2024-04-20T01:48:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.752312+0000", 2024-04-20T01:48:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0060515120000000002 2024-04-20T01:48:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.754299+0000", 2024-04-20T01:48:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019870690000000002 2024-04-20T01:48:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.759594+0000", 2024-04-20T01:48:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0052944070000000001 2024-04-20T01:48:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.759645+0000", 2024-04-20T01:48:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1236000000000002e-05 2024-04-20T01:48:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 48/30 ECSubWrite(tid=44, reqid=osd.5.0:44, at_version=48'33, trim_to=0'0, roll_forward_to=47'31))", 2024-04-20T01:48:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.716311+0000", 2024-04-20T01:48:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 44.210654699000003, 2024-04-20T01:48:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093701210000000004, 2024-04-20T01:48:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716311+0000", 2024-04-20T01:48:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716311+0000", 2024-04-20T01:48:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716311+0000", 2024-04-20T01:48:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716336+0000", 2024-04-20T01:48:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5216e-05 2024-04-20T01:48:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716348+0000", 2024-04-20T01:48:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1369000000000001e-05 2024-04-20T01:48:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716358+0000", 2024-04-20T01:48:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9019999999999994e-06 2024-04-20T01:48:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716410+0000", 2024-04-20T01:48:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1974000000000001e-05 2024-04-20T01:48:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716419+0000", 2024-04-20T01:48:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4439999999999997e-06 2024-04-20T01:48:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.725652+0000", 2024-04-20T01:48:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092323909999999995 2024-04-20T01:48:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.725682+0000", 2024-04-20T01:48:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0049999999999999e-05 2024-04-20T01:48:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 48/30 ECSubWrite(tid=256, reqid=client.4643.0:403, at_version=48'46, trim_to=0'0, roll_forward_to=48'44))", 2024-04-20T01:48:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.789372+0000", 2024-04-20T01:48:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 44.137594123, 2024-04-20T01:48:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086922490000000008, 2024-04-20T01:48:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789372+0000", 2024-04-20T01:48:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789372+0000", 2024-04-20T01:48:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789371+0000", 2024-04-20T01:48:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789968+0000", 2024-04-20T01:48:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00059678300000000004 2024-04-20T01:48:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789979+0000", 2024-04-20T01:48:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1499e-05 2024-04-20T01:48:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789986+0000", 2024-04-20T01:48:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2179999999999996e-06 2024-04-20T01:48:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.790029+0000", 2024-04-20T01:48:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2876000000000001e-05 2024-04-20T01:48:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.790036+0000", 2024-04-20T01:48:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7870000000000002e-06 2024-04-20T01:48:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.798040+0000", 2024-04-20T01:48:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080038720000000004 2024-04-20T01:48:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.798064+0000", 2024-04-20T01:48:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4037999999999999e-05 2024-04-20T01:48:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 48/30 ECSubWrite(tid=257, reqid=client.4643.0:404, at_version=48'47, trim_to=0'0, roll_forward_to=48'44))", 2024-04-20T01:48:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.791622+0000", 2024-04-20T01:48:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 44.135344136999997, 2024-04-20T01:48:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0088453630000000002, 2024-04-20T01:48:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.791622+0000", 2024-04-20T01:48:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.791622+0000", 2024-04-20T01:48:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.791621+0000", 2024-04-20T01:48:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792215+0000", 2024-04-20T01:48:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000593689 2024-04-20T01:48:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792227+0000", 2024-04-20T01:48:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2037e-05 2024-04-20T01:48:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792236+0000", 2024-04-20T01:48:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8240000000000006e-06 2024-04-20T01:48:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792538+0000", 2024-04-20T01:48:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000301668 2024-04-20T01:48:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792549+0000", 2024-04-20T01:48:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0915e-05 2024-04-20T01:48:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.800443+0000", 2024-04-20T01:48:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0078945300000000003 2024-04-20T01:48:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.800467+0000", 2024-04-20T01:48:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4309000000000001e-05 2024-04-20T01:48:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s1 53/30 ECSubWrite(tid=33, reqid=client.4643.0:444, at_version=53'22, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.721415+0000", 2024-04-20T01:48:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.205551360000001, 2024-04-20T01:48:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097484879999999996, 2024-04-20T01:48:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721415+0000", 2024-04-20T01:48:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721415+0000", 2024-04-20T01:48:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721413+0000", 2024-04-20T01:48:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721522+0000", 2024-04-20T01:48:33.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108445 2024-04-20T01:48:33.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721540+0000", 2024-04-20T01:48:33.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8128e-05 2024-04-20T01:48:33.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721555+0000", 2024-04-20T01:48:33.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5228999999999999e-05 2024-04-20T01:48:33.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721639+0000", 2024-04-20T01:48:33.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3803999999999999e-05 2024-04-20T01:48:33.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721650+0000", 2024-04-20T01:48:33.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0702e-05 2024-04-20T01:48:33.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731131+0000", 2024-04-20T01:48:33.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094816929999999994 2024-04-20T01:48:33.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731163+0000", 2024-04-20T01:48:33.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1946999999999998e-05 2024-04-20T01:48:33.975 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.975 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.975 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.975 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 58/30 ECSubWrite(tid=52, reqid=client.4643.0:489, at_version=58'41, trim_to=0'0, roll_forward_to=55'37))", 2024-04-20T01:48:33.975 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:59.769343+0000", 2024-04-20T01:48:33.975 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 34.157623022000003, 2024-04-20T01:48:33.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094380780000000008, 2024-04-20T01:48:33.975 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.975 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.975 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769343+0000", 2024-04-20T01:48:33.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769343+0000", 2024-04-20T01:48:33.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769343+0000", 2024-04-20T01:48:33.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:33.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769462+0000", 2024-04-20T01:48:33.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011929 2024-04-20T01:48:33.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769474+0000", 2024-04-20T01:48:33.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1857e-05 2024-04-20T01:48:33.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769482+0000", 2024-04-20T01:48:33.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0809999999999994e-06 2024-04-20T01:48:33.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769531+0000", 2024-04-20T01:48:33.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9168000000000001e-05 2024-04-20T01:48:33.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769538+0000", 2024-04-20T01:48:33.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5930000000000002e-06 2024-04-20T01:48:33.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.778749+0000", 2024-04-20T01:48:33.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092115670000000004 2024-04-20T01:48:33.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.778781+0000", 2024-04-20T01:48:33.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2140000000000001e-05 2024-04-20T01:48:33.979 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.979 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.979 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.979 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:733 3.9s0 3:94399c02:::smithi16579500-3:head [rmxattr Pp[amim]nQkdfb][ca]Rhonh_mr`cpapUabYUfb\\ in=40b,rmxattr QP`igcZgYZ`X]j_`ObrUNcR_gQZW^mQs_T`XhhQp in=40b,rmxattr QXqRngrniZSrdNdQaffOsm_g_pYQoedrXoNqT[mn in=40b,rmxattr V[thp]nUrl^Xata^al[oh[d`ddg`p[siljQWZ^\\X in=40b,rmxattr XgcaedbaNamUmoa^d[dhg[arrjeikfUlToSjsbY` in=40b,rmxattr ZcXlR_O_^fXfTQrt`cZfOgVlmhepod\\^\\`socsW` in=40b,rmxattr ][ifNPWgaZd[^RnfpRhWbfS\\^a]m[\\[ienetlUUZ in=40b,rmxattr `U`PSXddYml`lYXX[sSNnPfg`VgWbXsYnftUQ`it in=40b,rmxattr aR in=2b,rmxattr eNsVcjeRcj\\m]Q[VhYdSqNcmPhhR[UPR^ORSk_Vd in=40b,rmxattr eRqQZNtlPRVjcgdjmbSmOsUSjSlU]tSoVjr`^oRN in=40b,rmxattr fVRh^qlag^rjpdaqVk`^r]eR`P[XUlr]]agZjWq[ in=40b,rmxattr frZXQX\\isoq]gmjs`rk]PiQ\\fV[RTNPNYN]YV[mq in=40b,rmxattr h\\bUdlZ]kknhkaaYeTcclaXatghTgbVrhNQblto_ in=40b,rmxattr hgq\\iqsggNPTRPQok in=17b,rmxattr jecnmafmfg__r^\\S_sU]bi]kYaplTcttSSX`\\prR in=40b,rmxattr jmliR in=5b,rmxattr mnhUacnNajpOrNecmPZT[OUj[XaWmU in=30b,rmxattr peUYXiNb_UsOaXnhlX_s_[QTW_SoUaN]Wal`rkao in=40b,rmxattr rZrXsckW]Y`iWQiRn in=17b] snapc 26=[26,25,1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-20T01:48:33.979 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:11.823678+0000", 2024-04-20T01:48:33.979 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 22.103288656, 2024-04-20T01:48:33.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0098521919999999992, 2024-04-20T01:48:33.979 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.979 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:33.979 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:33.979 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:33.980 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:33.980 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 733 2024-04-20T01:48:33.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.980 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823678+0000", 2024-04-20T01:48:33.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823678+0000", 2024-04-20T01:48:33.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823675+0000", 2024-04-20T01:48:33.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:33.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823691+0000", 2024-04-20T01:48:33.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5954999999999999e-05 2024-04-20T01:48:33.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823715+0000", 2024-04-20T01:48:33.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4113000000000001e-05 2024-04-20T01:48:33.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823730+0000", 2024-04-20T01:48:33.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4694e-05 2024-04-20T01:48:33.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823786+0000", 2024-04-20T01:48:33.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6419000000000003e-05 2024-04-20T01:48:33.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:33.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823913+0000", 2024-04-20T01:48:33.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012677500000000001 2024-04-20T01:48:33.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.824989+0000", 2024-04-20T01:48:33.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0010762199999999999 2024-04-20T01:48:33.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.828014+0000", 2024-04-20T01:48:33.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0030246190000000001 2024-04-20T01:48:33.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:33.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.833442+0000", 2024-04-20T01:48:33.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0054285490000000004 2024-04-20T01:48:33.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.833530+0000", 2024-04-20T01:48:33.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7462999999999999e-05 2024-04-20T01:48:33.984 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.984 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.984 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.984 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 75/30 ECSubWrite(tid=100, reqid=osd.5.0:100, at_version=75'80, trim_to=0'0, roll_forward_to=68'78))", 2024-04-20T01:48:33.984 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.720432+0000", 2024-04-20T01:48:33.984 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 17.206533910000001, 2024-04-20T01:48:33.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010779983, 2024-04-20T01:48:33.984 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.984 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.985 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720432+0000", 2024-04-20T01:48:33.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720432+0000", 2024-04-20T01:48:33.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720432+0000", 2024-04-20T01:48:33.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:33.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720457+0000", 2024-04-20T01:48:33.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5272000000000001e-05 2024-04-20T01:48:33.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720469+0000", 2024-04-20T01:48:33.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2193e-05 2024-04-20T01:48:33.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720480+0000", 2024-04-20T01:48:33.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0291e-05 2024-04-20T01:48:33.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723263+0000", 2024-04-20T01:48:33.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002783003 2024-04-20T01:48:33.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723273+0000", 2024-04-20T01:48:33.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.01e-05 2024-04-20T01:48:33.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731180+0000", 2024-04-20T01:48:33.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0079069929999999993 2024-04-20T01:48:33.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731212+0000", 2024-04-20T01:48:33.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2421e-05 2024-04-20T01:48:33.988 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.988 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.988 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.988 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 75/30 ECSubWrite(tid=112, reqid=osd.14.0:112, at_version=75'91, trim_to=0'0, roll_forward_to=74'89))", 2024-04-20T01:48:33.988 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.720882+0000", 2024-04-20T01:48:33.988 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 17.206084381, 2024-04-20T01:48:33.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010362481999999999, 2024-04-20T01:48:33.988 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.988 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.989 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720882+0000", 2024-04-20T01:48:33.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720882+0000", 2024-04-20T01:48:33.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720881+0000", 2024-04-20T01:48:33.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:33.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720905+0000", 2024-04-20T01:48:33.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3683e-05 2024-04-20T01:48:33.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720917+0000", 2024-04-20T01:48:33.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1651000000000001e-05 2024-04-20T01:48:33.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720926+0000", 2024-04-20T01:48:33.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2040000000000003e-06 2024-04-20T01:48:33.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724280+0000", 2024-04-20T01:48:33.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003354363 2024-04-20T01:48:33.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724291+0000", 2024-04-20T01:48:33.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1004e-05 2024-04-20T01:48:33.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731213+0000", 2024-04-20T01:48:33.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069213670000000003 2024-04-20T01:48:33.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731244+0000", 2024-04-20T01:48:33.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1563000000000003e-05 2024-04-20T01:48:33.992 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.992 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.992 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.992 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 75/30 ECSubWrite(tid=101, reqid=osd.5.0:101, at_version=75'82, trim_to=0'0, roll_forward_to=68'78))", 2024-04-20T01:48:33.992 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.723236+0000", 2024-04-20T01:48:33.992 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 17.203730009000001, 2024-04-20T01:48:33.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010789544, 2024-04-20T01:48:33.992 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.992 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.993 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723236+0000", 2024-04-20T01:48:33.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723236+0000", 2024-04-20T01:48:33.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723235+0000", 2024-04-20T01:48:33.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:33.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723261+0000", 2024-04-20T01:48:33.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5381e-05 2024-04-20T01:48:33.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723279+0000", 2024-04-20T01:48:33.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7796e-05 2024-04-20T01:48:33.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723290+0000", 2024-04-20T01:48:33.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1739000000000001e-05 2024-04-20T01:48:33.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726374+0000", 2024-04-20T01:48:33.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003084023 2024-04-20T01:48:33.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726389+0000", 2024-04-20T01:48:33.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5041e-05 2024-04-20T01:48:33.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:33.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.733997+0000", 2024-04-20T01:48:33.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076079329999999999 2024-04-20T01:48:33.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:33.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.734026+0000", 2024-04-20T01:48:33.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8394e-05 2024-04-20T01:48:33.996 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.996 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:33.996 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:33.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.996 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 75/30 ECSubWrite(tid=113, reqid=osd.14.0:113, at_version=75'93, trim_to=0'0, roll_forward_to=74'89))", 2024-04-20T01:48:33.996 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.724246+0000", 2024-04-20T01:48:33.996 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 17.202720164999999, 2024-04-20T01:48:33.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097217950000000001, 2024-04-20T01:48:33.996 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:33.996 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:33.997 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:33.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:33.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724246+0000", 2024-04-20T01:48:33.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:33.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724246+0000", 2024-04-20T01:48:33.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:33.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:33.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724245+0000", 2024-04-20T01:48:33.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:33.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:33.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724270+0000", 2024-04-20T01:48:33.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4536999999999999e-05 2024-04-20T01:48:33.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:33.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724284+0000", 2024-04-20T01:48:33.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4421e-05 2024-04-20T01:48:33.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:33.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724292+0000", 2024-04-20T01:48:33.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6009999999999997e-06 2024-04-20T01:48:33.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:33.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726433+0000", 2024-04-20T01:48:33.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021418399999999999 2024-04-20T01:48:33.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:33.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726444+0000", 2024-04-20T01:48:33.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0671e-05 2024-04-20T01:48:33.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:33.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:33.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:34.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.733939+0000", 2024-04-20T01:48:34.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0074953249999999997 2024-04-20T01:48:34.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.733968+0000", 2024-04-20T01:48:34.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8368e-05 2024-04-20T01:48:34.000 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.000 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.000 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.000 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.000 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.000 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:34.002 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-20T01:48:34.002 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:34.016+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:34.091 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:48:34.093 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T01:48:34.102 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:34.117+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:34.105 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-04-20T01:48:33.792729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":125829664,"num_objects":71,"num_object_clones":24,"num_object_copies":487,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":2,"num_read":598,"num_read_kb":212551,"num_write":1191,"num_write_kb":170420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":45056000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1363,"ondisk_log_size":1363,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":700904,"kb_used_data":247988,"kb_used_omap":24,"kb_used_meta":452583,"kb_avail":1509248536,"statfs":{"total":1546188226560,"available":1545470500864,"internally_reserved":0,"allocated":253939712,"data_stored":359854650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25507,"internal_metadata":463445085},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9453568,"num_objects":-9,"num_object_clones":-8,"num_object_copies":-63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-9,"num_whiteouts":0,"num_read":30,"num_read_kb":33370,"num_write":20,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":20856832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020446"},"pg_stats":[{"pgid":"3.f","version":"89'63","reported_seq":256,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T01:48:30.798062+0000","last_change":"2024-04-20T01:48:30.799178+0000","last_active":"2024-04-20T01:48:30.798062+0000","last_peered":"2024-04-20T01:48:30.798062+0000","last_clean":"2024-04-20T01:48:30.798062+0000","last_became_active":"2024-04-20T01:47:32.867228+0000","last_became_peered":"2024-04-20T01:47:32.867228+0000","last_unstale":"2024-04-20T01:48:30.798062+0000","last_undegraded":"2024-04-20T01:48:30.798062+0000","last_fullsized":"2024-04-20T01:48:30.798062+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.328818+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.013492588,"stat_sum":{"num_bytes":8863744,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":16058,"num_write":69,"num_write_kb":9456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,15,13,8,11,0],"acting":[2,3,15,13,8,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2d","length":"3"},{"start":"33","length":"1"}]},{"pgid":"3.e","version":"89'99","reported_seq":299,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T01:48:30.805505+0000","last_change":"2024-04-20T01:48:30.806232+0000","last_active":"2024-04-20T01:48:30.805505+0000","last_peered":"2024-04-20T01:48:30.805505+0000","last_clean":"2024-04-20T01:48:30.805505+0000","last_became_active":"2024-04-20T01:47:32.867409+0000","last_became_peered":"2024-04-20T01:47:32.867409+0000","last_unstale":"2024-04-20T01:48:30.805505+0000","last_undegraded":"2024-04-20T01:48:30.805505+0000","last_fullsized":"2024-04-20T01:48:30.805505+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:42.581217+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.020773026999999999,"stat_sum":{"num_bytes":8175616,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":43,"num_read_kb":7598,"num_write":40,"num_write_kb":11936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,1,10,8,3,12],"acting":[11,9,1,10,8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2d","length":"3"},{"start":"33","length":"1"}]},{"pgid":"3.d","version":"86'13","reported_seq":196,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T01:48:30.784971+0000","last_change":"2024-04-20T01:48:30.785595+0000","last_active":"2024-04-20T01:48:30.784971+0000","last_peered":"2024-04-20T01:48:30.784971+0000","last_clean":"2024-04-20T01:48:30.784971+0000","last_became_active":"2024-04-20T01:47:32.867954+0000","last_became_peered":"2024-04-20T01:47:32.867954+0000","last_unstale":"2024-04-20T01:48:30.784971+0000","last_undegraded":"2024-04-20T01:48:30.784971+0000","last_fullsized":"2024-04-20T01:48:30.784971+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.083554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075594499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,11,15,7,8,4],"acting":[6,2,11,15,7,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2d","length":"3"},{"start":"33","length":"1"}]},{"pgid":"3.c","version":"84'117","reported_seq":288,"reported_epoch":84,"state":"stale+active+clean","last_fresh":"2024-04-20T01:48:25.757240+0000","last_change":"2024-04-20T01:48:25.757788+0000","last_active":"2024-04-20T01:48:25.757240+0000","last_peered":"2024-04-20T01:48:25.757240+0000","last_clean":"2024-04-20T01:48:25.757240+0000","last_became_active":"2024-04-20T01:47:32.867926+0000","last_became_peered":"2024-04-20T01:47:32.867926+0000","last_unstale":"2024-04-20T01:48:31.784728+0000","last_undegraded":"2024-04-20T01:48:25.757240+0000","last_fullsized":"2024-04-20T01:48:25.757240+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:54.374528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006835336,"stat_sum":{"num_bytes":24707072,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":23,"num_read_kb":8938,"num_write":82,"num_write_kb":22768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,8,6,3,7,12,5],"acting":[14,8,6,3,7,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"3"}]},{"pgid":"3.b","version":"89'172","reported_seq":381,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T01:48:30.809781+0000","last_change":"2024-04-20T01:48:30.811377+0000","last_active":"2024-04-20T01:48:30.809781+0000","last_peered":"2024-04-20T01:48:30.809781+0000","last_clean":"2024-04-20T01:48:30.809781+0000","last_became_active":"2024-04-20T01:47:32.867309+0000","last_became_peered":"2024-04-20T01:47:32.867309+0000","last_unstale":"2024-04-20T01:48:30.809781+0000","last_undegraded":"2024-04-20T01:48:30.809781+0000","last_fullsized":"2024-04-20T01:48:30.809781+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:01.227735+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.025073073000000001,"stat_sum":{"num_bytes":24674304,"num_objects":7,"num_object_clones":1,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":72,"num_read_kb":68190,"num_write":184,"num_write_kb":26320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,11,10,9],"acting":[4,0,1,12,11,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2d","length":"3"},{"start":"33","length":"1"}]},{"pgid":"3.a","version":"84'28","reported_seq":215,"reported_epoch":89,"state":"active+clean+snaptrim","last_fresh":"2024-04-20T01:48:30.784716+0000","last_change":"2024-04-20T01:48:30.784754+0000","last_active":"2024-04-20T01:48:30.784716+0000","last_peered":"2024-04-20T01:48:30.784716+0000","last_clean":"2024-04-20T01:48:30.784716+0000","last_became_active":"2024-04-20T01:47:32.868002+0000","last_became_peered":"2024-04-20T01:47:32.868002+0000","last_unstale":"2024-04-20T01:48:30.784716+0000","last_undegraded":"2024-04-20T01:48:30.784716+0000","last_fullsized":"2024-04-20T01:48:30.784716+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:35.716198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068918899999999999,"stat_sum":{"num_bytes":1720320,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":7013,"num_write":12,"num_write_kb":3376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,1,14,3,15,2],"acting":[6,10,1,14,3,15,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"3"},{"start":"33","length":"1"}]},{"pgid":"3.9","version":"87'37","reported_seq":223,"reported_epoch":89,"state":"active+clean+snaptrim","last_fresh":"2024-04-20T01:48:30.794846+0000","last_change":"2024-04-20T01:48:30.794897+0000","last_active":"2024-04-20T01:48:30.794846+0000","last_peered":"2024-04-20T01:48:30.794846+0000","last_clean":"2024-04-20T01:48:30.794846+0000","last_became_active":"2024-04-20T01:47:32.866706+0000","last_became_peered":"2024-04-20T01:47:32.866706+0000","last_unstale":"2024-04-20T01:48:30.794846+0000","last_undegraded":"2024-04-20T01:48:30.794846+0000","last_fullsized":"2024-04-20T01:48:30.794846+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:42.776206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012668992,"stat_sum":{"num_bytes":4620288,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":3124,"num_write":50,"num_write_kb":5008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0,11,14,8,3],"acting":[7,1,0,11,14,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"3"},{"start":"33","length":"1"}]},{"pgid":"3.8","version":"88'94","reported_seq":289,"reported_epoch":89,"state":"active+clean+snaptrim","last_fresh":"2024-04-20T01:48:30.796818+0000","last_change":"2024-04-20T01:48:30.796855+0000","last_active":"2024-04-20T01:48:30.796818+0000","last_peered":"2024-04-20T01:48:30.796818+0000","last_clean":"2024-04-20T01:48:30.796818+0000","last_became_active":"2024-04-20T01:47:32.718069+0000","last_became_peered":"2024-04-20T01:47:32.718069+0000","last_unstale":"2024-04-20T01:48:30.796818+0000","last_undegraded":"2024-04-20T01:48:30.796818+0000","last_fullsized":"2024-04-20T01:48:30.796818+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:58.489885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043849600000000002,"stat_sum":{"num_bytes":7716864,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":20032,"num_write":45,"num_write_kb":13232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,11,3,14,2,7],"acting":[5,0,11,3,14,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"3"},{"start":"33","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T01:48:29.774946+0000","last_change":"2024-04-20T01:47:20.071988+0000","last_active":"2024-04-20T01:48:29.774946+0000","last_peered":"2024-04-20T01:48:29.774946+0000","last_clean":"2024-04-20T01:48:29.774946+0000","last_became_active":"2024-04-20T01:47:20.071615+0000","last_became_peered":"2024-04-20T01:47:20.071615+0000","last_unstale":"2024-04-20T01:48:29.774946+0000","last_undegraded":"2024-04-20T01:48:29.774946+0000","last_fullsized":"2024-04-20T01:48:29.774946+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:40.106777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"77'33","reported_seq":224,"reported_epoch":89,"state":"active+clean+snaptrim","last_fresh":"2024-04-20T01:48:30.781193+0000","last_change":"2024-04-20T01:48:30.781219+0000","last_active":"2024-04-20T01:48:30.781193+0000","last_peered":"2024-04-20T01:48:30.781193+0000","last_clean":"2024-04-20T01:48:30.781193+0000","last_became_active":"2024-04-20T01:47:32.867180+0000","last_became_peered":"2024-04-20T01:47:32.867180+0000","last_unstale":"2024-04-20T01:48:30.781193+0000","last_undegraded":"2024-04-20T01:48:30.781193+0000","last_fullsized":"2024-04-20T01:48:30.781193+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:43.358783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049871900000000001,"stat_sum":{"num_bytes":4440064,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":14071,"num_write":18,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,4,14,12,0,11],"acting":[10,8,4,14,12,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"3"},{"start":"33","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T01:48:29.789988+0000","last_change":"2024-04-20T01:47:20.075613+0000","last_active":"2024-04-20T01:48:29.789988+0000","last_peered":"2024-04-20T01:48:29.789988+0000","last_clean":"2024-04-20T01:48:29.789988+0000","last_became_active":"2024-04-20T01:47:20.075314+0000","last_became_peered":"2024-04-20T01:47:20.075314+0000","last_unstale":"2024-04-20T01:48:29.789988+0000","last_undegraded":"2024-04-20T01:48:29.789988+0000","last_fullsized":"2024-04-20T01:48:29.789988+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:35.956173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,12],"acting":[2,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"89'49","reported_seq":236,"reported_epoch":90,"state":"peering","last_fresh":"2024-04-20T01:48:31.786581+0000","last_change":"2024-04-20T01:48:31.786821+0000","last_active":"2024-04-20T01:48:31.786472+0000","last_peered":"2024-04-20T01:48:30.786632+0000","last_clean":"2024-04-20T01:48:30.786632+0000","last_became_active":"2024-04-20T01:47:32.719136+0000","last_became_peered":"2024-04-20T01:47:32.719136+0000","last_unstale":"2024-04-20T01:48:31.786581+0000","last_undegraded":"2024-04-20T01:48:31.786581+0000","last_fullsized":"2024-04-20T01:48:31.786581+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:43.099853+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.015135822,"stat_sum":{"num_bytes":1540096,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":3173,"num_write":61,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,4,2147483647,11,5,10],"acting":[13,2,4,2147483647,11,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,5,10,11],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"3"},{"start":"33","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":83,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T01:48:30.786389+0000","last_change":"2024-04-20T01:48:26.145238+0000","last_active":"2024-04-20T01:48:30.786389+0000","last_peered":"2024-04-20T01:48:30.786389+0000","last_clean":"2024-04-20T01:48:30.786389+0000","last_became_active":"2024-04-20T01:47:20.038972+0000","last_became_peered":"2024-04-20T01:47:20.038972+0000","last_unstale":"2024-04-20T01:48:30.786389+0000","last_undegraded":"2024-04-20T01:48:30.786389+0000","last_fullsized":"2024-04-20T01:48:30.786389+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:26.145095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:26.145095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:54.362058+0000","scrub_duration":0.0015014830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"89'69","reported_seq":268,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T01:48:30.808270+0000","last_change":"2024-04-20T01:48:30.809117+0000","last_active":"2024-04-20T01:48:30.808270+0000","last_peered":"2024-04-20T01:48:30.808270+0000","last_clean":"2024-04-20T01:48:30.808270+0000","last_became_active":"2024-04-20T01:47:32.868365+0000","last_became_peered":"2024-04-20T01:47:32.868365+0000","last_unstale":"2024-04-20T01:48:30.808270+0000","last_undegraded":"2024-04-20T01:48:30.808270+0000","last_fullsized":"2024-04-20T01:48:30.808270+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:41.198793+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.023149652,"stat_sum":{"num_bytes":9338880,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":9566,"num_write":48,"num_write_kb":11056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,1,5,15,4],"acting":[3,7,6,1,5,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2d","length":"3"},{"start":"33","length":"1"}]},{"pgid":"2.3","version":"0'0","reported_seq":83,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T01:48:29.773565+0000","last_change":"2024-04-20T01:48:26.810391+0000","last_active":"2024-04-20T01:48:29.773565+0000","last_peered":"2024-04-20T01:48:29.773565+0000","last_clean":"2024-04-20T01:48:29.773565+0000","last_became_active":"2024-04-20T01:47:20.038764+0000","last_became_peered":"2024-04-20T01:47:20.038764+0000","last_unstale":"2024-04-20T01:48:29.773565+0000","last_undegraded":"2024-04-20T01:48:29.773565+0000","last_fullsized":"2024-04-20T01:48:29.773565+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:26.810276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:26.810276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:27.490462+0000","scrub_duration":0.0038071630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"25'157","reported_seq":270,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T01:48:30.786390+0000","last_change":"2024-04-20T01:47:17.303029+0000","last_active":"2024-04-20T01:48:30.786390+0000","last_peered":"2024-04-20T01:48:30.786390+0000","last_clean":"2024-04-20T01:48:30.786390+0000","last_became_active":"2024-04-20T01:47:17.302105+0000","last_became_peered":"2024-04-20T01:47:17.302105+0000","last_unstale":"2024-04-20T01:48:30.786390+0000","last_undegraded":"2024-04-20T01:48:30.786390+0000","last_fullsized":"2024-04-20T01:48:30.786390+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_clean_scrub_stamp":"2024-04-20T01:47:15.986754+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,15],"acting":[13,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.2","version":"89'146","reported_seq":346,"reported_epoch":90,"state":"peering","last_fresh":"2024-04-20T01:48:31.786366+0000","last_change":"2024-04-20T01:48:31.786553+0000","last_active":"2024-04-20T01:48:31.786270+0000","last_peered":"2024-04-20T01:48:30.787189+0000","last_clean":"2024-04-20T01:48:30.787189+0000","last_became_active":"2024-04-20T01:47:32.866509+0000","last_became_peered":"2024-04-20T01:47:32.866509+0000","last_unstale":"2024-04-20T01:48:31.786366+0000","last_undegraded":"2024-04-20T01:48:31.786366+0000","last_fullsized":"2024-04-20T01:48:31.786366+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:46.649300+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.011785924,"stat_sum":{"num_bytes":13025280,"num_objects":8,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":62,"num_read_kb":21576,"num_write":105,"num_write_kb":23152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,6,10,2147483647,1,7],"acting":[13,3,6,10,2147483647,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3,6,7,10],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"3"},{"start":"33","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T01:48:29.779552+0000","last_change":"2024-04-20T01:48:19.877116+0000","last_active":"2024-04-20T01:48:29.779552+0000","last_peered":"2024-04-20T01:48:29.779552+0000","last_clean":"2024-04-20T01:48:29.779552+0000","last_became_active":"2024-04-20T01:48:19.876908+0000","last_became_peered":"2024-04-20T01:48:19.876908+0000","last_unstale":"2024-04-20T01:48:29.779552+0000","last_undegraded":"2024-04-20T01:48:29.779552+0000","last_fullsized":"2024-04-20T01:48:29.779552+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"80'47","reported_seq":231,"reported_epoch":89,"state":"active+clean+snaptrim","last_fresh":"2024-04-20T01:48:30.786264+0000","last_change":"2024-04-20T01:48:30.786309+0000","last_active":"2024-04-20T01:48:30.786264+0000","last_peered":"2024-04-20T01:48:30.786264+0000","last_clean":"2024-04-20T01:48:30.786264+0000","last_became_active":"2024-04-20T01:47:32.722189+0000","last_became_peered":"2024-04-20T01:47:32.722189+0000","last_unstale":"2024-04-20T01:48:30.786264+0000","last_undegraded":"2024-04-20T01:48:30.786264+0000","last_fullsized":"2024-04-20T01:48:30.786264+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:43.443403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096742699999999998,"stat_sum":{"num_bytes":2588672,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":1012,"num_write":36,"num_write_kb":3424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,14,9,5,6,3,13],"acting":[4,14,9,5,6,3,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"3"},{"start":"33","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T01:48:29.783407+0000","last_change":"2024-04-20T01:47:20.038409+0000","last_active":"2024-04-20T01:48:29.783407+0000","last_peered":"2024-04-20T01:48:29.783407+0000","last_clean":"2024-04-20T01:48:29.783407+0000","last_became_active":"2024-04-20T01:47:20.038107+0000","last_became_peered":"2024-04-20T01:47:20.038107+0000","last_unstale":"2024-04-20T01:48:29.783407+0000","last_undegraded":"2024-04-20T01:48:29.783407+0000","last_fullsized":"2024-04-20T01:48:29.783407+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:44.125577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"89'174","reported_seq":652,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T01:48:30.800639+0000","last_change":"2024-04-20T01:48:30.801500+0000","last_active":"2024-04-20T01:48:30.800639+0000","last_peered":"2024-04-20T01:48:30.800639+0000","last_clean":"2024-04-20T01:48:30.800639+0000","last_became_active":"2024-04-20T01:48:29.782732+0000","last_became_peered":"2024-04-20T01:48:29.782732+0000","last_unstale":"2024-04-20T01:48:30.800639+0000","last_undegraded":"2024-04-20T01:48:30.800639+0000","last_fullsized":"2024-04-20T01:48:30.800639+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:01.314314+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0204043,"stat_sum":{"num_bytes":9256960,"num_objects":9,"num_object_clones":2,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":16502,"num_write":126,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":45056000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,15,4,1,0,6,13],"acting":[8,15,4,1,0,6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2d","length":"3"},{"start":"33","length":"1"}]},{"pgid":"3.7","version":"89'43","reported_seq":229,"reported_epoch":89,"state":"active+clean+snaptrim","last_fresh":"2024-04-20T01:48:30.811463+0000","last_change":"2024-04-20T01:48:30.811499+0000","last_active":"2024-04-20T01:48:30.811463+0000","last_peered":"2024-04-20T01:48:30.811463+0000","last_clean":"2024-04-20T01:48:30.811463+0000","last_became_active":"2024-04-20T01:47:32.722002+0000","last_became_peered":"2024-04-20T01:47:32.722002+0000","last_unstale":"2024-04-20T01:48:30.811463+0000","last_undegraded":"2024-04-20T01:48:30.811463+0000","last_fullsized":"2024-04-20T01:48:30.811463+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:43.990113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015133989,"stat_sum":{"num_bytes":3932160,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":20,"num_read_kb":13318,"num_write":54,"num_write_kb":4640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,0,8,5,14],"acting":[4,11,9,0,8,5,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2d","length":"3"},{"start":"33","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T01:48:29.776904+0000","last_change":"2024-04-20T01:47:20.110600+0000","last_active":"2024-04-20T01:48:29.776904+0000","last_peered":"2024-04-20T01:48:29.776904+0000","last_clean":"2024-04-20T01:48:29.776904+0000","last_became_active":"2024-04-20T01:47:20.110260+0000","last_became_peered":"2024-04-20T01:47:20.110260+0000","last_unstale":"2024-04-20T01:48:29.776904+0000","last_undegraded":"2024-04-20T01:48:29.776904+0000","last_fullsized":"2024-04-20T01:48:29.776904+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:47.718290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T01:48:29.779514+0000","last_change":"2024-04-20T01:47:20.036634+0000","last_active":"2024-04-20T01:48:29.779514+0000","last_peered":"2024-04-20T01:48:29.779514+0000","last_clean":"2024-04-20T01:48:29.779514+0000","last_became_active":"2024-04-20T01:47:20.036328+0000","last_became_peered":"2024-04-20T01:47:20.036328+0000","last_unstale":"2024-04-20T01:48:29.779514+0000","last_undegraded":"2024-04-20T01:48:29.779514+0000","last_fullsized":"2024-04-20T01:48:29.779514+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:36.385250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,13],"acting":[6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"89'22","reported_seq":213,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T01:48:30.790151+0000","last_change":"2024-04-20T01:48:30.791390+0000","last_active":"2024-04-20T01:48:30.790151+0000","last_peered":"2024-04-20T01:48:30.790151+0000","last_clean":"2024-04-20T01:48:30.790151+0000","last_became_active":"2024-04-20T01:47:32.866834+0000","last_became_peered":"2024-04-20T01:47:32.866834+0000","last_unstale":"2024-04-20T01:48:30.790151+0000","last_undegraded":"2024-04-20T01:48:30.790151+0000","last_fullsized":"2024-04-20T01:48:30.790151+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:54.424527+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0097936429999999994,"stat_sum":{"num_bytes":638976,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":2260,"num_write":52,"num_write_kb":2720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,3,6,2,9],"acting":[0,13,1,3,6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"28","length":"1"},{"start":"2a","length":"1"},{"start":"2d","length":"3"},{"start":"33","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":125239296,"num_objects":69,"num_object_clones":24,"num_object_copies":483,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":2,"num_read":462,"num_read_kb":212433,"num_write":988,"num_write_kb":166960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":45056000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":219475968,"data_stored":291270656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1206,"ondisk_log_size":1206,"up":112,"acting":112,"num_store_stats":16},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280530,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43060,"kb_used_data":11944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94328780,"statfs":{"total":96636764160,"available":96592670720,"internally_reserved":0,"allocated":12230656,"data_stored":15308983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.38}]},{"osd":2,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.503}]},{"osd":3,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.442},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.32900000000000001}]},{"osd":4,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":9.0129999999999999,"5min":9.0129999999999999,"15min":9.0129999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.48399999999999999}]},{"osd":5,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.34599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":7,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.55000000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":12,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":9.1389999999999993,"5min":9.1389999999999993,"15min":9.1389999999999993},"last":0.81599999999999995}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.52900000000000003}]},{"osd":15,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.8719999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.877},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.57599999999999996}]}]},{"osd":15,"up_from":21,"seq":90194313234,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37496,"kb_used_data":10468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334344,"statfs":{"total":96636764160,"available":96598368256,"internally_reserved":0,"allocated":10719232,"data_stored":14926476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":4.5259999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":0.36499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":2,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.47499999999999998}]},{"osd":3,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.26500000000000001}]},{"osd":4,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.496}]},{"osd":5,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.50900000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.46899999999999997}]},{"osd":7,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.41499999999999998}]},{"osd":8,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":4.5979999999999999,"5min":4.5979999999999999,"15min":4.5979999999999999},"last":0.43099999999999999}]},{"osd":9,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.57599999999999996}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.53600000000000003}]}]},{"osd":14,"up_from":21,"seq":90194313234,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43664,"kb_used_data":16640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328176,"statfs":{"total":96636764160,"available":96592052224,"internally_reserved":0,"allocated":17039360,"data_stored":23276351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,4,5,6,7,8,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.49199999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.44},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.56799999999999995}]},{"osd":5,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.629},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.61699999999999999}]},{"osd":6,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.42599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":9,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.65700000000000003}]},{"osd":12,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.67400000000000004}]},{"osd":13,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.75}]},{"osd":15,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.70999999999999996}]}]},{"osd":12,"up_from":21,"seq":90194313234,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42264,"kb_used_data":15236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94329576,"statfs":{"total":96636764160,"available":96593485824,"internally_reserved":0,"allocated":15601664,"data_stored":22786156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.68700000000000006}]},{"osd":2,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.93400000000000005}]},{"osd":3,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.79800000000000004}]},{"osd":4,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.82399999999999995}]},{"osd":5,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":1.0800000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":1.0349999999999999}]},{"osd":11,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":6.6779999999999999,"5min":6.6779999999999999,"15min":6.6779999999999999},"last":1.012}]},{"osd":13,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":1.1739999999999999}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":1.135},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.99299999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":1.21}]}]},{"osd":11,"up_from":21,"seq":90194313233,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48316,"kb_used_data":17188,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94323524,"statfs":{"total":96636764160,"available":96587288576,"internally_reserved":0,"allocated":17600512,"data_stored":25858156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,3,4,8,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.45300000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":3,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.28799999999999998}]},{"osd":4,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":8.5969999999999995,"5min":8.5969999999999995,"15min":8.5969999999999995},"last":0.376},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.44400000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":9,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.42999999999999999}]},{"osd":10,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.65800000000000003}]},{"osd":12,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":0.46300000000000002},{"interface":"front","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":12.385,"5min":12.385,"15min":12.385},"last":0.504}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.61899999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.40899999999999997}]}]},{"osd":10,"up_from":21,"seq":90194313235,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47168,"kb_used_data":16040,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94324672,"statfs":{"total":96636764160,"available":96588464128,"internally_reserved":0,"allocated":16424960,"data_stored":24178796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,4,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":7,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.67800000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":9,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.90000000000000002}]},{"osd":11,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":6.0410000000000004,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":0.78800000000000003}]},{"osd":12,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.331,"15min":1.331},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":9.7509999999999994,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":9.7759999999999998,"5min":9.7759999999999998,"15min":9.7759999999999998},"last":0.623}]},{"osd":13,"last update":"Sat Apr 20 01:48:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.69999999999999996}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.64200000000000002}]},{"osd":15,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.83199999999999996}]}]},{"osd":9,"up_from":21,"seq":90194313235,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38368,"kb_used_data":11340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333472,"statfs":{"total":96636764160,"available":96597475328,"internally_reserved":0,"allocated":11612160,"data_stored":17715308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,4,6,7,8,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.54900000000000004}]},{"osd":4,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.61099999999999999}]},{"osd":6,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.67300000000000004}]},{"osd":7,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.315}]},{"osd":8,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.3319999999999999,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.502}]},{"osd":10,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.625},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.57299999999999995}]},{"osd":11,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.42099999999999999}]},{"osd":12,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.433}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.56000000000000005}]},{"osd":15,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.39000000000000001}]}]},{"osd":8,"up_from":21,"seq":90194313234,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43956,"kb_used_data":16928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94327884,"statfs":{"total":96636764160,"available":96591753216,"internally_reserved":0,"allocated":17334272,"data_stored":25186412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,6,7,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":1,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.44}]},{"osd":3,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9630000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":0.48099999999999998}]},{"osd":7,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.65600000000000003}]},{"osd":9,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.63}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":11,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.5110000000000001,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.77300000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":15,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.70199999999999996}]}]},{"osd":7,"up_from":21,"seq":90194313235,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43068,"kb_used_data":16040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328772,"statfs":{"total":96636764160,"available":96592662528,"internally_reserved":0,"allocated":16424960,"data_stored":22454380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,3,4,6,8,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":1,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.55500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":4,"last update":"Sat Apr 20 01:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":4.1630000000000003,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":4.1920000000000002,"5min":4.1920000000000002,"15min":4.1920000000000002},"last":0.76300000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.71099999999999997}]},{"osd":8,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":5.7679999999999998,"5min":5.7679999999999998,"15min":5.7679999999999998},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":9.952,"5min":9.952,"15min":9.952},"last":0.82099999999999995}]},{"osd":9,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.61899999999999999}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":11,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.57399999999999995}]},{"osd":12,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":4.2720000000000002,"5min":4.2720000000000002,"15min":4.2720000000000002},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.83699999999999997}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.85999999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":6,"up_from":21,"seq":90194313233,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46356,"kb_used_data":19328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94325484,"statfs":{"total":96636764160,"available":96589295616,"internally_reserved":0,"allocated":19791872,"data_stored":25972844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,7,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":5,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.48099999999999998}]},{"osd":7,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.64500000000000002}]},{"osd":8,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.83599999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":10,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.59099999999999997}]},{"osd":11,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.67500000000000004}]},{"osd":12,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.81599999999999995}]},{"osd":13,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.746},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.63400000000000001}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.65700000000000003}]},{"osd":15,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.76100000000000001}]}]},{"osd":1,"up_from":21,"seq":90194313234,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53096,"kb_used_data":21972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94318744,"statfs":{"total":96636764160,"available":96582393856,"internally_reserved":0,"allocated":22499328,"data_stored":33022060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.442},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.56599999999999995}]},{"osd":2,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.46600000000000003}]},{"osd":3,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.503}]},{"osd":4,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.40999999999999998}]},{"osd":5,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.371},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.35599999999999998}]},{"osd":6,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.56000000000000005}]},{"osd":7,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51300000000000001}]},{"osd":8,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.58299999999999996}]},{"osd":9,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.47499999999999998}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.57699999999999996}]}]},{"osd":0,"up_from":21,"seq":90194313235,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45144,"kb_used_data":18116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94326696,"statfs":{"total":96636764160,"available":96590536704,"internally_reserved":0,"allocated":18550784,"data_stored":27074082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.58199999999999996}]},{"osd":2,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":8.0619999999999994,"5min":8.0619999999999994,"15min":8.0619999999999994},"last":0.625},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.69199999999999995}]},{"osd":3,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":3.363,"5min":3.363,"15min":3.363},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":7.9710000000000001,"5min":7.9710000000000001,"15min":7.9710000000000001},"last":0.86899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":3.5339999999999998,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":0.60699999999999998}]},{"osd":6,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":8.0239999999999991,"5min":8.0239999999999991,"15min":8.0239999999999991},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.754}]},{"osd":7,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":3.4020000000000001,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":7.8630000000000004,"5min":7.8630000000000004,"15min":7.8630000000000004},"last":0.90400000000000003}]},{"osd":8,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":8.1189999999999998,"5min":8.1189999999999998,"15min":8.1189999999999998},"last":0.93400000000000005}]},{"osd":9,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":3.5699999999999998,"5min":3.5699999999999998,"15min":3.5699999999999998},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":0.89500000000000002}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":15,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":3.4369999999999998,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":0.94499999999999995}]}]},{"osd":2,"up_from":21,"seq":90194313233,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33292,"kb_used_data":6272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338548,"statfs":{"total":96636764160,"available":96602673152,"internally_reserved":0,"allocated":6422528,"data_stored":7002354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":1,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.59099999999999997}]},{"osd":3,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.72699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":7,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.67300000000000004}]},{"osd":8,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.52500000000000002}]},{"osd":9,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.77500000000000002}]},{"osd":11,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.86799999999999999}]},{"osd":12,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":3.1190000000000002,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":3.492,"5min":3.492,"15min":3.492},"last":0.69399999999999995}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]}]},{"osd":3,"up_from":21,"seq":90194313236,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52848,"kb_used_data":21724,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94318992,"statfs":{"total":96636764160,"available":96582647808,"internally_reserved":0,"allocated":22245376,"data_stored":31383660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,4,5,6,7,8,9,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":2,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.629}]},{"osd":4,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":9.3650000000000002,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":0.56999999999999995}]},{"osd":5,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.44800000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.46000000000000002}]},{"osd":7,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.502}]},{"osd":8,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":9.4700000000000006,"5min":9.4700000000000006,"15min":9.4700000000000006},"last":0.62}]},{"osd":9,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.59499999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":12,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":10.959,"5min":10.959,"15min":10.959},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.69199999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]}]},{"osd":4,"up_from":21,"seq":90194313235,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43468,"kb_used_data":16440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328372,"statfs":{"total":96636764160,"available":96592252928,"internally_reserved":0,"allocated":16834560,"data_stored":25538668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,5,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":5.0679999999999996,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":0.47899999999999998},{"interface":"front","average":{"1min":1.331,"5min":1.331,"15min":1.331},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":16.27,"5min":16.27,"15min":16.27},"last":0.42899999999999999}]},{"osd":1,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":10.507999999999999,"5min":10.507999999999999,"15min":10.507999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":11.019,"5min":11.019,"15min":11.019},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":4.9180000000000001,"5min":4.9180000000000001,"15min":4.9180000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":10.619,"5min":10.619,"15min":10.619},"last":0.59899999999999998}]},{"osd":5,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":10.895,"5min":10.895,"15min":10.895},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":4.9589999999999996,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":0.61499999999999999}]},{"osd":6,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":11.051,"5min":11.051,"15min":11.051},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":10.589,"5min":10.589,"15min":10.589},"last":0.71699999999999997}]},{"osd":7,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":10.957000000000001,"5min":10.957000000000001,"15min":10.957000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":10.656000000000001,"5min":10.656000000000001,"15min":10.656000000000001},"last":0.66100000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":9,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.385,"15min":1.385},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":10.929,"5min":10.929,"15min":10.929},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":10.555999999999999,"5min":10.555999999999999,"15min":10.555999999999999},"last":0.79800000000000004}]},{"osd":10,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":5.0229999999999997,"5min":5.0229999999999997,"15min":5.0229999999999997},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":10.69,"5min":10.69,"15min":10.69},"last":0.84199999999999997}]},{"osd":11,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":10.987,"5min":10.987,"15min":10.987},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":4.9900000000000002,"5min":4.9900000000000002,"15min":4.9900000000000002},"last":0.81699999999999995}]},{"osd":12,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":10.720000000000001,"5min":10.720000000000001,"15min":10.720000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.772,"5min":1.772,"15min":1.772},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":16.617000000000001,"5min":16.617000000000001,"15min":16.617000000000001},"last":0.86599999999999999}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]},{"osd":5,"up_from":21,"seq":90194313234,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39340,"kb_used_data":12312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94332500,"statfs":{"total":96636764160,"available":96596480000,"internally_reserved":0,"allocated":12607488,"data_stored":18169964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":10.423999999999999,"5min":10.423999999999999,"15min":10.423999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":10.465,"5min":10.465,"15min":10.465},"last":0.67400000000000004}]},{"osd":2,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.622},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":4,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":8.5269999999999992,"5min":8.5269999999999992,"15min":8.5269999999999992},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.79,"5min":2.79,"15min":2.79},"last":0.65800000000000003}]},{"osd":6,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.745},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.77000000000000002}]},{"osd":7,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.80300000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":10,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.83299999999999996}]},{"osd":11,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.874}]},{"osd":12,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":10.616,"5min":10.616,"15min":10.616},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":10.629,"5min":10.629,"15min":10.629},"last":0.86299999999999999}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.84699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16035840,"data_stored":20062208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19283968,"data_stored":24420352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5120000,"data_stored":6250496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20262912,"data_stored":27172864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":13942784,"data_stored":18989056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10948608,"data_stored":17084416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16306176,"data_stored":22474752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":14766080,"data_stored":20942848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":14491648,"data_stored":20504576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":10002432,"data_stored":12136448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":14815232,"data_stored":18333696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":15990784,"data_stored":19365888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":13991936,"data_stored":16941056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":10399744,"data_stored":14028800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":15822848,"data_stored":22577152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":7294976,"data_stored":9986048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T01:48:34.109 INFO:tasks.ceph.ceph_manager.ceph:pg 3.a in trimming, state: active+clean+snaptrim 2024-04-20T01:48:34.109 INFO:tasks.ceph.ceph_manager.ceph:pg 3.9 in trimming, state: active+clean+snaptrim 2024-04-20T01:48:34.109 INFO:tasks.ceph.ceph_manager.ceph:pg 3.8 in trimming, state: active+clean+snaptrim 2024-04-20T01:48:34.109 INFO:tasks.ceph.ceph_manager.ceph:pg 3.5 in trimming, state: active+clean+snaptrim 2024-04-20T01:48:34.109 INFO:tasks.ceph.ceph_manager.ceph:pg 3.1 in trimming, state: active+clean+snaptrim 2024-04-20T01:48:34.109 INFO:tasks.ceph.ceph_manager.ceph:pg 3.7 in trimming, state: active+clean+snaptrim 2024-04-20T01:48:34.110 INFO:tasks.ceph.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-04-20T01:48:34.132 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:34.132 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:34.132 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-04-20T01:48:34.132 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:34.142 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-20T01:48:34.203 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:34.218+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:34.225 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-14 --journal-path /var/lib/ceph/osd/ceph-14/journal --force --op remove --pgid 3.5s3 2024-04-20T01:48:34.271 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:34.271 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:34.271 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:48:34.271 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:48:34.271 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:34.281 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-20T01:48:34.309 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:34.324+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:34.404 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:34.419+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:34.411 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:34.412 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:48:34.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:48:34.412 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:48:34.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.412 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:104 3.es0 3:78f9000f:::smithi16579500-35:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:34.412 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:48:34.412 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.462734335999997, 2024-04-20T01:48:34.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012120811, 2024-04-20T01:48:34.412 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:34.412 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:34.412 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:34.412 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:34.412 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:34.412 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 104 2024-04-20T01:48:34.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.413 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:34.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:34.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:48:34.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:34.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:48:34.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:34.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:48:34.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:34.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:34.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962445+0000", 2024-04-20T01:48:34.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3210000000000001e-06 2024-04-20T01:48:34.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:34.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962451+0000", 2024-04-20T01:48:34.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.31e-06 2024-04-20T01:48:34.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:34.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962457+0000", 2024-04-20T01:48:34.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4819999999999999e-06 2024-04-20T01:48:34.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966434+0000", 2024-04-20T01:48:34.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039772239999999997 2024-04-20T01:48:34.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:34.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966501+0000", 2024-04-20T01:48:34.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6947999999999997e-05 2024-04-20T01:48:34.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:34.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966766+0000", 2024-04-20T01:48:34.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00026468999999999999 2024-04-20T01:48:34.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:34.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970614+0000", 2024-04-20T01:48:34.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0038481330000000001 2024-04-20T01:48:34.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:34.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974486+0000", 2024-04-20T01:48:34.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003871799 2024-04-20T01:48:34.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974563+0000", 2024-04-20T01:48:34.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7416999999999994e-05 2024-04-20T01:48:34.417 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.418 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.418 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.418 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:105 3.es0 3:78f9000f:::smithi16579500-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:34.418 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962461+0000", 2024-04-20T01:48:34.418 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.462716125999997, 2024-04-20T01:48:34.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014134643000000001, 2024-04-20T01:48:34.418 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:34.418 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:34.418 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:34.418 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:34.418 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:34.418 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 105 2024-04-20T01:48:34.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.419 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:34.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:34.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962461+0000", 2024-04-20T01:48:34.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:34.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962461+0000", 2024-04-20T01:48:34.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:34.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962460+0000", 2024-04-20T01:48:34.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:34.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:34.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962462+0000", 2024-04-20T01:48:34.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.897e-06 2024-04-20T01:48:34.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:34.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962466+0000", 2024-04-20T01:48:34.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4520000000000002e-06 2024-04-20T01:48:34.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:34.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962468+0000", 2024-04-20T01:48:34.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6570000000000001e-06 2024-04-20T01:48:34.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966956+0000", 2024-04-20T01:48:34.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044879730000000001 2024-04-20T01:48:34.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:34.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.967029+0000", 2024-04-20T01:48:34.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2657999999999995e-05 2024-04-20T01:48:34.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974602+0000", 2024-04-20T01:48:34.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075729370000000001 2024-04-20T01:48:34.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:34.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974650+0000", 2024-04-20T01:48:34.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8000000000000001e-05 2024-04-20T01:48:34.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976595+0000", 2024-04-20T01:48:34.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019453789999999999 2024-04-20T01:48:34.423 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.423 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.423 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.423 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:109 3.es0 3:7f925225:::smithi16579500-37:head [append 0~606208,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:34.423 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:48:34.423 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.462705694999997, 2024-04-20T01:48:34.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012453573000000001, 2024-04-20T01:48:34.423 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:34.423 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:34.423 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:34.423 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:34.423 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:34.424 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 109 2024-04-20T01:48:34.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.424 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:34.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:34.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:48:34.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:34.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:48:34.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:34.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:48:34.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:34.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:34.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962732+0000", 2024-04-20T01:48:34.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00026073200000000001 2024-04-20T01:48:34.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:34.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962742+0000", 2024-04-20T01:48:34.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0735e-05 2024-04-20T01:48:34.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:34.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962749+0000", 2024-04-20T01:48:34.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.365e-06 2024-04-20T01:48:34.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.967197+0000", 2024-04-20T01:48:34.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044487119999999996 2024-04-20T01:48:34.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:34.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.967324+0000", 2024-04-20T01:48:34.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126575 2024-04-20T01:48:34.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:34.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968044+0000", 2024-04-20T01:48:34.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00072015399999999998 2024-04-20T01:48:34.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:34.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970795+0000", 2024-04-20T01:48:34.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027507650000000001 2024-04-20T01:48:34.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:34.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974884+0000", 2024-04-20T01:48:34.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040886469999999999 2024-04-20T01:48:34.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974925+0000", 2024-04-20T01:48:34.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1094000000000003e-05 2024-04-20T01:48:34.429 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.429 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.429 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.430 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:110 3.es0 3:7f925225:::smithi16579500-37:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:34.430 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962755+0000", 2024-04-20T01:48:34.430 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.462421825, 2024-04-20T01:48:34.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012356723999999999, 2024-04-20T01:48:34.430 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:34.430 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:34.430 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:34.430 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:34.430 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:34.430 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 110 2024-04-20T01:48:34.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.430 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:34.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:34.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962755+0000", 2024-04-20T01:48:34.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:34.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962755+0000", 2024-04-20T01:48:34.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:34.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962754+0000", 2024-04-20T01:48:34.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:34.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:34.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962758+0000", 2024-04-20T01:48:34.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8750000000000002e-06 2024-04-20T01:48:34.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:34.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962765+0000", 2024-04-20T01:48:34.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2370000000000003e-06 2024-04-20T01:48:34.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:34.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962770+0000", 2024-04-20T01:48:34.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9030000000000003e-06 2024-04-20T01:48:34.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968614+0000", 2024-04-20T01:48:34.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0058437619999999997 2024-04-20T01:48:34.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:34.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968676+0000", 2024-04-20T01:48:34.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1592000000000003e-05 2024-04-20T01:48:34.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:34.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968935+0000", 2024-04-20T01:48:34.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025953599999999999 2024-04-20T01:48:34.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:34.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970828+0000", 2024-04-20T01:48:34.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0018928510000000001 2024-04-20T01:48:34.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:34.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975036+0000", 2024-04-20T01:48:34.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042079680000000003 2024-04-20T01:48:34.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975112+0000", 2024-04-20T01:48:34.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5558999999999999e-05 2024-04-20T01:48:34.436 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.436 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.436 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.437 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:111 3.es0 3:7f925225:::smithi16579500-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:34.437 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962775+0000", 2024-04-20T01:48:34.437 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.462402070000003, 2024-04-20T01:48:34.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014337756, 2024-04-20T01:48:34.437 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:34.437 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:34.437 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:34.437 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:34.437 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:34.437 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 111 2024-04-20T01:48:34.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.438 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:34.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:34.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962775+0000", 2024-04-20T01:48:34.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:34.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962775+0000", 2024-04-20T01:48:34.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:34.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962774+0000", 2024-04-20T01:48:34.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:34.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:34.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962777+0000", 2024-04-20T01:48:34.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7839999999999999e-06 2024-04-20T01:48:34.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:34.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962784+0000", 2024-04-20T01:48:34.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9809999999999997e-06 2024-04-20T01:48:34.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:34.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962790+0000", 2024-04-20T01:48:34.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5509999999999999e-06 2024-04-20T01:48:34.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970564+0000", 2024-04-20T01:48:34.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077740680000000003 2024-04-20T01:48:34.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:34.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970609+0000", 2024-04-20T01:48:34.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5059999999999999e-05 2024-04-20T01:48:34.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975165+0000", 2024-04-20T01:48:34.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0045564999999999998 2024-04-20T01:48:34.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:34.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975263+0000", 2024-04-20T01:48:34.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7689000000000006e-05 2024-04-20T01:48:34.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977112+0000", 2024-04-20T01:48:34.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001849552 2024-04-20T01:48:34.443 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.443 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.443 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.443 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:142 3.es0 3:797661d0:::smithi16579500-48:head [append 0~425984,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:34.444 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962794+0000", 2024-04-20T01:48:34.444 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.462382869000002, 2024-04-20T01:48:34.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013969364, 2024-04-20T01:48:34.444 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:34.444 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:34.444 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:34.444 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:34.444 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:34.444 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 142 2024-04-20T01:48:34.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.444 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:34.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:34.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962794+0000", 2024-04-20T01:48:34.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:34.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962794+0000", 2024-04-20T01:48:34.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:34.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962793+0000", 2024-04-20T01:48:34.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:34.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:34.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963014+0000", 2024-04-20T01:48:34.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000221015 2024-04-20T01:48:34.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:34.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963025+0000", 2024-04-20T01:48:34.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0739e-05 2024-04-20T01:48:34.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:34.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963032+0000", 2024-04-20T01:48:34.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7580000000000003e-06 2024-04-20T01:48:34.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.971338+0000", 2024-04-20T01:48:34.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008305659 2024-04-20T01:48:34.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:34.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.971477+0000", 2024-04-20T01:48:34.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013920100000000001 2024-04-20T01:48:34.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:34.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.972169+0000", 2024-04-20T01:48:34.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00069257800000000001 2024-04-20T01:48:34.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:34.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974577+0000", 2024-04-20T01:48:34.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024072820000000002 2024-04-20T01:48:34.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:34.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976694+0000", 2024-04-20T01:48:34.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021172629999999999 2024-04-20T01:48:34.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976763+0000", 2024-04-20T01:48:34.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9259999999999998e-05 2024-04-20T01:48:34.448 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.448 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.449 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.449 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:143 3.es0 3:797661d0:::smithi16579500-48:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:34.449 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.963039+0000", 2024-04-20T01:48:34.449 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.462138113999998, 2024-04-20T01:48:34.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013923471999999999, 2024-04-20T01:48:34.449 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:34.449 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:34.449 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:34.449 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:34.449 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:34.449 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 143 2024-04-20T01:48:34.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.449 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:34.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:34.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963039+0000", 2024-04-20T01:48:34.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:34.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963039+0000", 2024-04-20T01:48:34.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:34.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963038+0000", 2024-04-20T01:48:34.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:34.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:34.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963210+0000", 2024-04-20T01:48:34.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017155800000000001 2024-04-20T01:48:34.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:34.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963217+0000", 2024-04-20T01:48:34.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7309999999999993e-06 2024-04-20T01:48:34.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:34.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963230+0000", 2024-04-20T01:48:34.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.295e-05 2024-04-20T01:48:34.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.972654+0000", 2024-04-20T01:48:34.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094237369999999997 2024-04-20T01:48:34.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:34.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.972720+0000", 2024-04-20T01:48:34.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5958999999999996e-05 2024-04-20T01:48:34.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:34.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.973225+0000", 2024-04-20T01:48:34.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00050500399999999998 2024-04-20T01:48:34.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:34.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974578+0000", 2024-04-20T01:48:34.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001353295 2024-04-20T01:48:34.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976919+0000", 2024-04-20T01:48:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023405840000000002 2024-04-20T01:48:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976962+0000", 2024-04-20T01:48:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3226000000000003e-05 2024-04-20T01:48:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:144 3.es0 3:797661d0:::smithi16579500-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:48:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.461939479999998, 2024-04-20T01:48:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014456992, 2024-04-20T01:48:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 144 2024-04-20T01:48:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:48:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:48:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:48:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963240+0000", 2024-04-20T01:48:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6459999999999999e-06 2024-04-20T01:48:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963247+0000", 2024-04-20T01:48:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9650000000000002e-06 2024-04-20T01:48:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963253+0000", 2024-04-20T01:48:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1900000000000003e-06 2024-04-20T01:48:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.973649+0000", 2024-04-20T01:48:34.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010396865999999999 2024-04-20T01:48:34.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:34.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.973745+0000", 2024-04-20T01:48:34.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5345000000000003e-05 2024-04-20T01:48:34.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:34.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974151+0000", 2024-04-20T01:48:34.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000405893 2024-04-20T01:48:34.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976606+0000", 2024-04-20T01:48:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024551730000000002 2024-04-20T01:48:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977622+0000", 2024-04-20T01:48:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0010159570000000001 2024-04-20T01:48:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977694+0000", 2024-04-20T01:48:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2469e-05 2024-04-20T01:48:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:145 3.es0 3:797661d0:::smithi16579500-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:48:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.461917659000001, 2024-04-20T01:48:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016339408999999999, 2024-04-20T01:48:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 145 2024-04-20T01:48:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:48:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:48:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:48:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963267+0000", 2024-04-20T01:48:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2490000000000001e-06 2024-04-20T01:48:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963274+0000", 2024-04-20T01:48:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7599999999999997e-06 2024-04-20T01:48:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963279+0000", 2024-04-20T01:48:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5400000000000003e-06 2024-04-20T01:48:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977300+0000", 2024-04-20T01:48:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014020608 2024-04-20T01:48:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977365+0000", 2024-04-20T01:48:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5617999999999994e-05 2024-04-20T01:48:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.978057+0000", 2024-04-20T01:48:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00069179099999999998 2024-04-20T01:48:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.978136+0000", 2024-04-20T01:48:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9132000000000001e-05 2024-04-20T01:48:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.979598+0000", 2024-04-20T01:48:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0014621599999999999 2024-04-20T01:48:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:245 3.es0 3:78f9000f:::smithi16579500-35:head [copy-from ver 10] snapc 5=[2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785668+0000", 2024-04-20T01:48:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.639508472999999, 2024-04-20T01:48:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025155522, 2024-04-20T01:48:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:34.466 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:34.466 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:34.466 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 245 2024-04-20T01:48:34.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.466 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:34.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:34.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785668+0000", 2024-04-20T01:48:34.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:34.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785668+0000", 2024-04-20T01:48:34.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:34.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785666+0000", 2024-04-20T01:48:34.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:34.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:34.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785676+0000", 2024-04-20T01:48:34.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0139999999999999e-05 2024-04-20T01:48:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785692+0000", 2024-04-20T01:48:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5894999999999999e-05 2024-04-20T01:48:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785715+0000", 2024-04-20T01:48:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2654999999999999e-05 2024-04-20T01:48:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785771+0000", 2024-04-20T01:48:34.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6331e-05 2024-04-20T01:48:34.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:34.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785779+0000", 2024-04-20T01:48:34.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4789999999999999e-06 2024-04-20T01:48:34.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791776+0000", 2024-04-20T01:48:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059978840000000002 2024-04-20T01:48:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791905+0000", 2024-04-20T01:48:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000128147 2024-04-20T01:48:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.805603+0000", 2024-04-20T01:48:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013698824 2024-04-20T01:48:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.808612+0000", 2024-04-20T01:48:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003008541 2024-04-20T01:48:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.810754+0000", 2024-04-20T01:48:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021421840000000001 2024-04-20T01:48:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.810824+0000", 2024-04-20T01:48:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9627000000000003e-05 2024-04-20T01:48:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:246 3.es0 3:78f9000f:::smithi16579500-35:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:48:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.639448293999997, 2024-04-20T01:48:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020980167000000001, 2024-04-20T01:48:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 246 2024-04-20T01:48:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:48:34.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:34.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:48:34.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:34.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:48:34.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:34.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:34.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785731+0000", 2024-04-20T01:48:34.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3550000000000001e-06 2024-04-20T01:48:34.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:34.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785738+0000", 2024-04-20T01:48:34.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2080000000000003e-06 2024-04-20T01:48:34.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:34.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785747+0000", 2024-04-20T01:48:34.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.388e-06 2024-04-20T01:48:34.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785859+0000", 2024-04-20T01:48:34.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112432 2024-04-20T01:48:34.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785866+0000", 2024-04-20T01:48:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0890000000000002e-06 2024-04-20T01:48:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792104+0000", 2024-04-20T01:48:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062376970000000004 2024-04-20T01:48:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792157+0000", 2024-04-20T01:48:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3269000000000001e-05 2024-04-20T01:48:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.806521+0000", 2024-04-20T01:48:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014363651999999999 2024-04-20T01:48:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.806639+0000", 2024-04-20T01:48:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011755400000000001 2024-04-20T01:48:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.806709+0000", 2024-04-20T01:48:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0037999999999996e-05 2024-04-20T01:48:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:248 3.es0 3:79bd6b86:::smithi16579500-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:48:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.639392534000002, 2024-04-20T01:48:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011602988, 2024-04-20T01:48:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 248 2024-04-20T01:48:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:48:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:48:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:48:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785787+0000", 2024-04-20T01:48:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4079999999999998e-06 2024-04-20T01:48:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785791+0000", 2024-04-20T01:48:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9929999999999997e-06 2024-04-20T01:48:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785795+0000", 2024-04-20T01:48:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1559999999999997e-06 2024-04-20T01:48:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785936+0000", 2024-04-20T01:48:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014018 2024-04-20T01:48:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785939+0000", 2024-04-20T01:48:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-04-20T01:48:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794183+0000", 2024-04-20T01:48:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082434339999999991 2024-04-20T01:48:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794272+0000", 2024-04-20T01:48:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9246000000000006e-05 2024-04-20T01:48:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.797387+0000", 2024-04-20T01:48:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0031150269999999998 2024-04-20T01:48:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:249 3.es0 3:79bd6b86:::smithi16579500-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:48:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.639378237000003, 2024-04-20T01:48:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011999312, 2024-04-20T01:48:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 249 2024-04-20T01:48:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:48:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:48:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:48:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785800+0000", 2024-04-20T01:48:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.995e-06 2024-04-20T01:48:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785804+0000", 2024-04-20T01:48:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2990000000000001e-06 2024-04-20T01:48:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785807+0000", 2024-04-20T01:48:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.715e-06 2024-04-20T01:48:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785955+0000", 2024-04-20T01:48:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000147923 2024-04-20T01:48:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785958+0000", 2024-04-20T01:48:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8940000000000002e-06 2024-04-20T01:48:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794609+0000", 2024-04-20T01:48:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086509289999999999 2024-04-20T01:48:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.795125+0000", 2024-04-20T01:48:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00051564999999999998 2024-04-20T01:48:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.797798+0000", 2024-04-20T01:48:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0026731250000000002 2024-04-20T01:48:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:586 3.es0 3:79bd6b86:::smithi16579500-12:head [delete] snapc 20=[20,1f,1a,b] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e65)", 2024-04-20T01:48:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:06.762289+0000", 2024-04-20T01:48:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.662888158000001, 2024-04-20T01:48:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011713963000000001, 2024-04-20T01:48:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 586 2024-04-20T01:48:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762289+0000", 2024-04-20T01:48:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762289+0000", 2024-04-20T01:48:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762287+0000", 2024-04-20T01:48:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762294+0000", 2024-04-20T01:48:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6450000000000005e-06 2024-04-20T01:48:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762308+0000", 2024-04-20T01:48:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3652000000000001e-05 2024-04-20T01:48:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762318+0000", 2024-04-20T01:48:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6679999999999996e-06 2024-04-20T01:48:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762391+0000", 2024-04-20T01:48:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3823000000000001e-05 2024-04-20T01:48:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762400+0000", 2024-04-20T01:48:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6750000000000008e-06 2024-04-20T01:48:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.767267+0000", 2024-04-20T01:48:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0048666300000000003 2024-04-20T01:48:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.767389+0000", 2024-04-20T01:48:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012267699999999999 2024-04-20T01:48:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.774003+0000", 2024-04-20T01:48:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066131640000000004 2024-04-20T01:48:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:727 3.es0 3:797661d0:::smithi16579500-48:head [append 0~622592] snapc 26=[26,25,1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-20T01:48:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:11.814331+0000", 2024-04-20T01:48:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 22.610846125999998, 2024-04-20T01:48:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013760674000000001, 2024-04-20T01:48:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 727 2024-04-20T01:48:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814331+0000", 2024-04-20T01:48:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814331+0000", 2024-04-20T01:48:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814329+0000", 2024-04-20T01:48:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814688+0000", 2024-04-20T01:48:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00035975800000000001 2024-04-20T01:48:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814706+0000", 2024-04-20T01:48:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7248000000000002e-05 2024-04-20T01:48:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814716+0000", 2024-04-20T01:48:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9620000000000005e-06 2024-04-20T01:48:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814789+0000", 2024-04-20T01:48:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3734999999999998e-05 2024-04-20T01:48:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814800+0000", 2024-04-20T01:48:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0898e-05 2024-04-20T01:48:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.816951+0000", 2024-04-20T01:48:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021503379999999999 2024-04-20T01:48:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.817082+0000", 2024-04-20T01:48:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013159599999999999 2024-04-20T01:48:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.818406+0000", 2024-04-20T01:48:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013237749999999999 2024-04-20T01:48:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.822671+0000", 2024-04-20T01:48:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042655460000000003 2024-04-20T01:48:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.828006+0000", 2024-04-20T01:48:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005334999 2024-04-20T01:48:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.828091+0000", 2024-04-20T01:48:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4857000000000004e-05 2024-04-20T01:48:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:732 3.es0 3:797661d0:::smithi16579500-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e70)", 2024-04-20T01:48:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:11.823728+0000", 2024-04-20T01:48:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 22.601448975, 2024-04-20T01:48:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012035161000000001, 2024-04-20T01:48:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 732 2024-04-20T01:48:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823728+0000", 2024-04-20T01:48:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823728+0000", 2024-04-20T01:48:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823727+0000", 2024-04-20T01:48:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823731+0000", 2024-04-20T01:48:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8349999999999997e-06 2024-04-20T01:48:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823745+0000", 2024-04-20T01:48:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4211e-05 2024-04-20T01:48:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823756+0000", 2024-04-20T01:48:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0705e-05 2024-04-20T01:48:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.827650+0000", 2024-04-20T01:48:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003894163 2024-04-20T01:48:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.827758+0000", 2024-04-20T01:48:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107759 2024-04-20T01:48:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.834824+0000", 2024-04-20T01:48:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.007066587 2024-04-20T01:48:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.834920+0000", 2024-04-20T01:48:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5639999999999999e-05 2024-04-20T01:48:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.835763+0000", 2024-04-20T01:48:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000842835 2024-04-20T01:48:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:1007 3.es0 3:797661d0:::smithi16579500-48:head [read 0~6258688,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:31.807916+0000", 2024-04-20T01:48:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 2.6172612210000001, 2024-04-20T01:48:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011898517000000001, 2024-04-20T01:48:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1007 2024-04-20T01:48:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807916+0000", 2024-04-20T01:48:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807916+0000", 2024-04-20T01:48:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807913+0000", 2024-04-20T01:48:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807935+0000", 2024-04-20T01:48:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1943e-05 2024-04-20T01:48:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807962+0000", 2024-04-20T01:48:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6199000000000001e-05 2024-04-20T01:48:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807976+0000", 2024-04-20T01:48:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4029000000000001e-05 2024-04-20T01:48:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808105+0000", 2024-04-20T01:48:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012951299999999999 2024-04-20T01:48:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808234+0000", 2024-04-20T01:48:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000129068 2024-04-20T01:48:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.819814+0000", 2024-04-20T01:48:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011579789 2024-04-20T01:48:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:1008 3.es0 3:797661d0:::smithi16579500-48:head [read 0~6258688,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:31.807999+0000", 2024-04-20T01:48:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 2.6171781730000001, 2024-04-20T01:48:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020846726999999999, 2024-04-20T01:48:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1008 2024-04-20T01:48:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807999+0000", 2024-04-20T01:48:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807999+0000", 2024-04-20T01:48:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807998+0000", 2024-04-20T01:48:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808003+0000", 2024-04-20T01:48:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0919999999999997e-06 2024-04-20T01:48:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808014+0000", 2024-04-20T01:48:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1331999999999999e-05 2024-04-20T01:48:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808021+0000", 2024-04-20T01:48:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0559999999999999e-06 2024-04-20T01:48:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808626+0000", 2024-04-20T01:48:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000604493 2024-04-20T01:48:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808697+0000", 2024-04-20T01:48:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1711000000000005e-05 2024-04-20T01:48:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.828845+0000", 2024-04-20T01:48:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020147805000000001 2024-04-20T01:48:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:1009 3.es0 3:797661d0:::smithi16579500-48:head [read 0~6258688,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:31.808028+0000", 2024-04-20T01:48:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 2.6171487020000002, 2024-04-20T01:48:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.026288720000000002, 2024-04-20T01:48:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1009 2024-04-20T01:48:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808028+0000", 2024-04-20T01:48:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808028+0000", 2024-04-20T01:48:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808027+0000", 2024-04-20T01:48:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808032+0000", 2024-04-20T01:48:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1459999999999998e-06 2024-04-20T01:48:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808038+0000", 2024-04-20T01:48:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.494e-06 2024-04-20T01:48:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808044+0000", 2024-04-20T01:48:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2770000000000002e-06 2024-04-20T01:48:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808958+0000", 2024-04-20T01:48:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00091398300000000005 2024-04-20T01:48:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.809027+0000", 2024-04-20T01:48:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8675000000000002e-05 2024-04-20T01:48:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.834317+0000", 2024-04-20T01:48:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025289696 2024-04-20T01:48:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpReadReply(3.es0 90/30 ECSubReadReply(tid=129, attrs_read=0))", 2024-04-20T01:48:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:31.816185+0000", 2024-04-20T01:48:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 2.6089914790000002, 2024-04-20T01:48:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012799764, 2024-04-20T01:48:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.816185+0000", 2024-04-20T01:48:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.816185+0000", 2024-04-20T01:48:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.816185+0000", 2024-04-20T01:48:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.824675+0000", 2024-04-20T01:48:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0084907639999999996 2024-04-20T01:48:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.824690+0000", 2024-04-20T01:48:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4503e-05 2024-04-20T01:48:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.824700+0000", 2024-04-20T01:48:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7820000000000005e-06 2024-04-20T01:48:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.824734+0000", 2024-04-20T01:48:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4072000000000002e-05 2024-04-20T01:48:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.828985+0000", 2024-04-20T01:48:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042512629999999999 2024-04-20T01:48:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:34.524 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:34.526 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-20T01:48:34.526 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:34.539+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:34.627 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:34.641+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:34.696 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:34.696 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:34.706 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-20T01:48:34.732 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-20T01:48:34.747 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:34.761+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:34.829 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:34.842+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:34.875 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:34.875 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:34.885 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-20T01:48:34.969 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:34.983+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:35.030 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:35.044+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:35.055 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:35.055 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:35.065 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-20T01:48:35.131 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:35.146+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:35.209 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:35.209 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:35.210 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-04-20T01:48:35.210 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:35.221 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-20T01:48:35.265 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:35.279+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:35.333 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:35.348+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:35.364 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:35.364 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:35.364 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-04-20T01:48:35.364 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-04-20T01:48:35.364 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:35.375 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-20T01:48:35.377 INFO:teuthology.orchestra.run.smithi123.stdout: marking collection for removal 2024-04-20T01:48:35.377 INFO:teuthology.orchestra.run.smithi123.stdout:setting '_remove' omap key 2024-04-20T01:48:35.377 INFO:teuthology.orchestra.run.smithi123.stdout:finish_remove_pgs 3.5s3_head removing 3.5s3 2024-04-20T01:48:35.378 INFO:teuthology.orchestra.run.smithi123.stdout:Remove successful 2024-04-20T01:48:35.459 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:35.474+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:35.535 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:35.549+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:35.563 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:35.564 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-04-20T01:48:35.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-04-20T01:48:35.564 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-04-20T01:48:35.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.564 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.7s3 36/30 ECSubWrite(tid=58, reqid=client.4643.0:211, at_version=36'8, trim_to=0'0, roll_forward_to=33'5))", 2024-04-20T01:48:35.564 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.782029+0000", 2024-04-20T01:48:35.564 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 57.795017444000003, 2024-04-20T01:48:35.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0082533060000000002, 2024-04-20T01:48:35.564 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:35.564 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:35.564 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:35.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:35.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782029+0000", 2024-04-20T01:48:35.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:35.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782029+0000", 2024-04-20T01:48:35.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:35.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782028+0000", 2024-04-20T01:48:35.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:35.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:35.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782164+0000", 2024-04-20T01:48:35.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013607300000000001 2024-04-20T01:48:35.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:35.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782181+0000", 2024-04-20T01:48:35.566 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6651999999999999e-05 2024-04-20T01:48:35.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.566 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.566 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:35.566 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782196+0000", 2024-04-20T01:48:35.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5370999999999999e-05 2024-04-20T01:48:35.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.567 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.783434+0000", 2024-04-20T01:48:35.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001237581 2024-04-20T01:48:35.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.567 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:35.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.783444+0000", 2024-04-20T01:48:35.567 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0686999999999999e-05 2024-04-20T01:48:35.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.567 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.567 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:35.567 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.790251+0000", 2024-04-20T01:48:35.568 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068071729999999997 2024-04-20T01:48:35.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.568 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:35.568 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.790283+0000", 2024-04-20T01:48:35.568 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1250000000000001e-05 2024-04-20T01:48:35.568 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.568 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.568 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.568 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.568 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs1 36/30 ECSubWrite(tid=59, reqid=client.4643.0:214, at_version=36'14, trim_to=0'0, roll_forward_to=33'12))", 2024-04-20T01:48:35.568 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.783999+0000", 2024-04-20T01:48:35.568 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 57.793048368000001, 2024-04-20T01:48:35.568 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011678028, 2024-04-20T01:48:35.569 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:35.569 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:35.569 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:35.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.569 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:35.569 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.783999+0000", 2024-04-20T01:48:35.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.569 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:35.569 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.783999+0000", 2024-04-20T01:48:35.569 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.569 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.569 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:35.569 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.783997+0000", 2024-04-20T01:48:35.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:35.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:35.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.784024+0000", 2024-04-20T01:48:35.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6801999999999999e-05 2024-04-20T01:48:35.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.570 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:35.570 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.784037+0000", 2024-04-20T01:48:35.570 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2361000000000001e-05 2024-04-20T01:48:35.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:35.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.784046+0000", 2024-04-20T01:48:35.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5009999999999993e-06 2024-04-20T01:48:35.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.571 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.785306+0000", 2024-04-20T01:48:35.571 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001259942 2024-04-20T01:48:35.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.571 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.571 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:35.572 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.785316+0000", 2024-04-20T01:48:35.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0309000000000001e-05 2024-04-20T01:48:35.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.572 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:35.572 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.795648+0000", 2024-04-20T01:48:35.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010331127000000001 2024-04-20T01:48:35.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.572 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.572 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:35.572 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.795677+0000", 2024-04-20T01:48:35.572 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9016000000000001e-05 2024-04-20T01:48:35.572 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.572 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.572 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.573 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.573 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs1 37/30 ECSubWrite(tid=64, reqid=client.4643.0:225, at_version=37'16, trim_to=0'0, roll_forward_to=36'14))", 2024-04-20T01:48:35.573 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.788088+0000", 2024-04-20T01:48:35.573 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 56.788959347999999, 2024-04-20T01:48:35.573 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010015382, 2024-04-20T01:48:35.573 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:35.573 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:35.573 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:35.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.574 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:35.574 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788088+0000", 2024-04-20T01:48:35.574 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.574 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:35.574 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788088+0000", 2024-04-20T01:48:35.574 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.574 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.574 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:35.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788087+0000", 2024-04-20T01:48:35.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:35.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:35.575 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788230+0000", 2024-04-20T01:48:35.575 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014296 2024-04-20T01:48:35.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.575 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.575 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:35.576 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788255+0000", 2024-04-20T01:48:35.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4790000000000002e-05 2024-04-20T01:48:35.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.576 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:35.576 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788276+0000", 2024-04-20T01:48:35.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0806999999999999e-05 2024-04-20T01:48:35.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.576 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.576 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791925+0000", 2024-04-20T01:48:35.576 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003649949 2024-04-20T01:48:35.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.576 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:35.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791937+0000", 2024-04-20T01:48:35.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1544e-05 2024-04-20T01:48:35.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:35.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.798083+0000", 2024-04-20T01:48:35.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0061460079999999997 2024-04-20T01:48:35.577 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.577 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:35.577 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.798103+0000", 2024-04-20T01:48:35.577 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9907000000000001e-05 2024-04-20T01:48:35.577 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.578 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.578 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.578 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.8s1 47/30 ECSubWrite(tid=40, reqid=osd.5.0:40, at_version=47'29, trim_to=0'0, roll_forward_to=46'25))", 2024-04-20T01:48:35.578 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.719174+0000", 2024-04-20T01:48:35.578 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 46.857872808000003, 2024-04-20T01:48:35.578 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0103924, 2024-04-20T01:48:35.578 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:35.578 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:35.578 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:35.578 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.578 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:35.578 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719174+0000", 2024-04-20T01:48:35.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.579 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:35.579 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719174+0000", 2024-04-20T01:48:35.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.579 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:35.579 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719173+0000", 2024-04-20T01:48:35.579 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:35.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.579 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.579 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:35.579 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719197+0000", 2024-04-20T01:48:35.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4196e-05 2024-04-20T01:48:35.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:35.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719214+0000", 2024-04-20T01:48:35.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6276000000000001e-05 2024-04-20T01:48:35.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:35.580 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719223+0000", 2024-04-20T01:48:35.580 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6919999999999997e-06 2024-04-20T01:48:35.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.580 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.580 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720898+0000", 2024-04-20T01:48:35.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0016744259999999999 2024-04-20T01:48:35.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:35.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720905+0000", 2024-04-20T01:48:35.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6660000000000008e-06 2024-04-20T01:48:35.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.581 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:35.581 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.729531+0000", 2024-04-20T01:48:35.581 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086254020000000008 2024-04-20T01:48:35.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.581 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.582 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:35.582 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.729566+0000", 2024-04-20T01:48:35.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5621000000000003e-05 2024-04-20T01:48:35.582 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.582 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.582 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.582 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.582 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s4 47/30 ECSubWrite(tid=80, reqid=osd.8.0:80, at_version=47'65, trim_to=0'0, roll_forward_to=44'61))", 2024-04-20T01:48:35.582 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.720292+0000", 2024-04-20T01:48:35.582 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 46.856754383000002, 2024-04-20T01:48:35.582 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0092900529999999995, 2024-04-20T01:48:35.582 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:35.583 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:35.583 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:35.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:35.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720292+0000", 2024-04-20T01:48:35.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.583 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:35.583 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720292+0000", 2024-04-20T01:48:35.583 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.583 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:35.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720292+0000", 2024-04-20T01:48:35.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:35.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:35.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720310+0000", 2024-04-20T01:48:35.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7717e-05 2024-04-20T01:48:35.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.584 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.584 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:35.584 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720318+0000", 2024-04-20T01:48:35.584 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3129999999999999e-06 2024-04-20T01:48:35.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:35.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720325+0000", 2024-04-20T01:48:35.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.832e-06 2024-04-20T01:48:35.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720837+0000", 2024-04-20T01:48:35.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00051157600000000002 2024-04-20T01:48:35.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.585 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:35.585 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.720846+0000", 2024-04-20T01:48:35.585 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3260000000000002e-06 2024-04-20T01:48:35.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:35.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.729552+0000", 2024-04-20T01:48:35.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0087057520000000006 2024-04-20T01:48:35.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.586 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.586 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:35.586 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.729583+0000", 2024-04-20T01:48:35.586 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0923999999999997e-05 2024-04-20T01:48:35.586 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.586 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.586 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.587 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s4 48/30 ECSubWrite(tid=85, reqid=client.4643.0:399, at_version=48'71, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:35.587 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.711859+0000", 2024-04-20T01:48:35.587 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 45.865187519000003, 2024-04-20T01:48:35.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0082489499999999997, 2024-04-20T01:48:35.587 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:35.587 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:35.587 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:35.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.587 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:35.587 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711859+0000", 2024-04-20T01:48:35.587 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:35.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711859+0000", 2024-04-20T01:48:35.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:35.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711858+0000", 2024-04-20T01:48:35.588 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:35.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.588 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.588 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:35.588 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711887+0000", 2024-04-20T01:48:35.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8487e-05 2024-04-20T01:48:35.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:35.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711908+0000", 2024-04-20T01:48:35.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1625e-05 2024-04-20T01:48:35.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:35.589 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711926+0000", 2024-04-20T01:48:35.589 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8113000000000001e-05 2024-04-20T01:48:35.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.589 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.589 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712063+0000", 2024-04-20T01:48:35.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000136907 2024-04-20T01:48:35.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:35.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712076+0000", 2024-04-20T01:48:35.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2145e-05 2024-04-20T01:48:35.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.590 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:35.590 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.720075+0000", 2024-04-20T01:48:35.590 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0079993730000000006 2024-04-20T01:48:35.590 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.590 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.591 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:35.591 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.720108+0000", 2024-04-20T01:48:35.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3386000000000002e-05 2024-04-20T01:48:35.591 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.591 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.591 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.591 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.591 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s4 48/30 ECSubWrite(tid=86, reqid=client.4643.0:400, at_version=48'72, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:35.591 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.713114+0000", 2024-04-20T01:48:35.591 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 45.863933256999999, 2024-04-20T01:48:35.591 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0088368690000000007, 2024-04-20T01:48:35.591 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:35.591 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:35.592 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:35.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:35.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713114+0000", 2024-04-20T01:48:35.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:35.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713114+0000", 2024-04-20T01:48:35.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.592 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.592 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:35.592 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713113+0000", 2024-04-20T01:48:35.592 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:35.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:35.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713136+0000", 2024-04-20T01:48:35.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2974999999999999e-05 2024-04-20T01:48:35.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:35.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713149+0000", 2024-04-20T01:48:35.593 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2945000000000001e-05 2024-04-20T01:48:35.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.593 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.593 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:35.593 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713159+0000", 2024-04-20T01:48:35.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9909999999999997e-06 2024-04-20T01:48:35.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713217+0000", 2024-04-20T01:48:35.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8085999999999999e-05 2024-04-20T01:48:35.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.594 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:35.594 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713228+0000", 2024-04-20T01:48:35.594 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1345999999999999e-05 2024-04-20T01:48:35.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.594 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:35.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.721915+0000", 2024-04-20T01:48:35.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086870850000000006 2024-04-20T01:48:35.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.595 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:35.595 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.721950+0000", 2024-04-20T01:48:35.595 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5352999999999998e-05 2024-04-20T01:48:35.595 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.595 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.595 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.595 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.595 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs1 52/30 ECSubWrite(tid=124, reqid=client.4643.0:434, at_version=52'55, trim_to=0'0, roll_forward_to=48'53))", 2024-04-20T01:48:35.596 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:53.736586+0000", 2024-04-20T01:48:35.596 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 41.840460379, 2024-04-20T01:48:35.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0094278839999999992, 2024-04-20T01:48:35.596 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:35.596 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:35.596 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:35.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:35.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736586+0000", 2024-04-20T01:48:35.596 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.596 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.596 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.596 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:35.596 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736586+0000", 2024-04-20T01:48:35.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:35.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736586+0000", 2024-04-20T01:48:35.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:35.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:35.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736681+0000", 2024-04-20T01:48:35.597 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4703000000000005e-05 2024-04-20T01:48:35.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.597 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:35.597 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736716+0000", 2024-04-20T01:48:35.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5434e-05 2024-04-20T01:48:35.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:35.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736737+0000", 2024-04-20T01:48:35.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0645999999999999e-05 2024-04-20T01:48:35.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.598 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736860+0000", 2024-04-20T01:48:35.598 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012254099999999999 2024-04-20T01:48:35.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.598 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:35.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.736872+0000", 2024-04-20T01:48:35.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2824e-05 2024-04-20T01:48:35.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.599 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:35.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.745990+0000", 2024-04-20T01:48:35.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0091180250000000001 2024-04-20T01:48:35.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.599 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.599 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:35.599 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:53.746014+0000", 2024-04-20T01:48:35.599 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3983000000000001e-05 2024-04-20T01:48:35.600 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.600 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.600 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.600 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.600 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:573 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [create,setxattr POm]dpWYd`[NXWUkjTTrrjndlWhqT (15156) in=29b,setxattr ]bTTcZk (15156) in=7b,setxattr okg_granXfaiUobbllnc]Sk^[Vrj`[aRthnaRdoV (278) in=40b,setxattr gT`mrtdoaf^^ekYVVj`ZirVVqeUahfeOZdgUdQsV (382) in=40b,setxattr s\\mP`nbegjZ`iam]machaefN\\hcflgW[VmU^ik`_ (16840) in=40b,setxattr mPmpkek_rY\\scmjOek]degahQ^_SNk_epPi]p^fm (14456) in=40b,setxattr agOmkqtrc^nSastfh_^gPhplQZgnQkXRsXYbkW\\\\ (1789) in=40b,setxattr `Y[Zq^Pd_hpqld\\XRg`sTalN_ORNZ_aPUn_Nl\\ZQ (1330) in=40b,setxattr cnbkPlXlpjTSQl\\mdTVtmTO^ZjP[ra]OpgkWbsbZ (139) in=40b,setxattr Vjd]bV^O[pjq`XZai`go_goUTcrph[sfZXdaeo_d (453) in=40b,setxattr lPk`g]XVSTSfXOdjZeUQRqXTXgf]dUqZpmQ^hNh^ (841) in=40b,setxattr \\a\\rN^\\m`c^OlQbo^WerVQqiWZldlTX]n_o`^fcY (19224) in=40b,setxattr ehmtodtgcpX\\UmnfaWbtiaUQhSWb_lXn_qeXprOR (139) in=40b,setxattr VqR_VPdeQ]UeaTRh\\UfdplrbOaOSOsaSa_`pgZWX (10033) in=40b,setxattr U (15156) in=1b,setxattr `bXaUYb]oeUoqmrfqXgPlttVbU_laZ[rnYRZfRdd (418) in=40b,setxattr gPijt_qqRThf_ZmN`fYRO`UYUW^p_io^Sh\\_\\afh (1086) in=40b,setxattr WoqshhWZOap_beQ^YcYjaPktbdU[aac[`b\\d\\haa (734) in=40b,setxattr oOgQZWqrpnb_PhmYPbqgfRkbr_VjboWXsVpeUcPq (769) in=40b,setxattr t[S\\OOdRkdshlT\\lbkes]tbinQacc[aa^OYTep_d (1821) in=40b] snapc 1f=[1f,1b,1a,b] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-20T01:48:35.600 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:04.764695+0000", 2024-04-20T01:48:35.600 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 30.812351577000001, 2024-04-20T01:48:35.600 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0091856230000000004, 2024-04-20T01:48:35.600 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:35.600 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:35.600 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:35.600 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:35.600 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:35.601 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 573 2024-04-20T01:48:35.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.601 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:35.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:35.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.764695+0000", 2024-04-20T01:48:35.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:35.601 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.764695+0000", 2024-04-20T01:48:35.601 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.601 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.601 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:35.602 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.764694+0000", 2024-04-20T01:48:35.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:35.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.602 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:35.602 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.765516+0000", 2024-04-20T01:48:35.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00082247099999999997 2024-04-20T01:48:35.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.602 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:35.602 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.765528+0000", 2024-04-20T01:48:35.602 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2191999999999999e-05 2024-04-20T01:48:35.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.602 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:35.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.765535+0000", 2024-04-20T01:48:35.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4119999999999998e-06 2024-04-20T01:48:35.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.765582+0000", 2024-04-20T01:48:35.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7707999999999998e-05 2024-04-20T01:48:35.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:35.603 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.765695+0000", 2024-04-20T01:48:35.603 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000112675 2024-04-20T01:48:35.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.603 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.603 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:35.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.766572+0000", 2024-04-20T01:48:35.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00087748600000000002 2024-04-20T01:48:35.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:35.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.773424+0000", 2024-04-20T01:48:35.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068519499999999999 2024-04-20T01:48:35.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:35.604 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.773548+0000", 2024-04-20T01:48:35.604 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012316900000000001 2024-04-20T01:48:35.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.604 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.604 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:35.605 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:04.773881+0000", 2024-04-20T01:48:35.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033335299999999999 2024-04-20T01:48:35.605 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.605 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.605 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.605 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:619 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [rollback 1a] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:35.605 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.811454+0000", 2024-04-20T01:48:35.605 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 25.765592595000001, 2024-04-20T01:48:35.605 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0082693799999999998, 2024-04-20T01:48:35.606 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:35.606 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:35.606 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:35.606 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:35.606 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:35.606 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 619 2024-04-20T01:48:35.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.606 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:35.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.606 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:35.606 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.811454+0000", 2024-04-20T01:48:35.606 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.606 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:35.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.811454+0000", 2024-04-20T01:48:35.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:35.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.811454+0000", 2024-04-20T01:48:35.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:35.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.607 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:35.607 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.811457+0000", 2024-04-20T01:48:35.607 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9900000000000002e-06 2024-04-20T01:48:35.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.607 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:35.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.811464+0000", 2024-04-20T01:48:35.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8789999999999997e-06 2024-04-20T01:48:35.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:35.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.811470+0000", 2024-04-20T01:48:35.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.959e-06 2024-04-20T01:48:35.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.608 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.608 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.812533+0000", 2024-04-20T01:48:35.608 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0010629719999999999 2024-04-20T01:48:35.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.608 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:35.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.812647+0000", 2024-04-20T01:48:35.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000113911 2024-04-20T01:48:35.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:35.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.813311+0000", 2024-04-20T01:48:35.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00066420200000000002 2024-04-20T01:48:35.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.609 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.609 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:35.609 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.814350+0000", 2024-04-20T01:48:35.609 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0010393749999999999 2024-04-20T01:48:35.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:35.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.819649+0000", 2024-04-20T01:48:35.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052988430000000001 2024-04-20T01:48:35.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.610 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.610 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:35.610 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.819724+0000", 2024-04-20T01:48:35.610 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4702000000000004e-05 2024-04-20T01:48:35.610 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.610 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.610 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.611 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:641 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [append 0~507904] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:35.611 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.871779+0000", 2024-04-20T01:48:35.611 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 25.705267537000001, 2024-04-20T01:48:35.611 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0091809730000000003, 2024-04-20T01:48:35.611 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:35.611 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:35.611 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:35.611 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:35.611 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:35.611 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 641 2024-04-20T01:48:35.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.611 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:35.611 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.611 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:35.612 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.871779+0000", 2024-04-20T01:48:35.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.612 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:35.612 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.871779+0000", 2024-04-20T01:48:35.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.612 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:35.612 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.871778+0000", 2024-04-20T01:48:35.612 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:35.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.612 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.612 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:35.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.872406+0000", 2024-04-20T01:48:35.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00062772399999999997 2024-04-20T01:48:35.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:35.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.872415+0000", 2024-04-20T01:48:35.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1460000000000002e-06 2024-04-20T01:48:35.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:35.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.872422+0000", 2024-04-20T01:48:35.613 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2430000000000002e-06 2024-04-20T01:48:35.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.613 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.613 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.872471+0000", 2024-04-20T01:48:35.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9867999999999998e-05 2024-04-20T01:48:35.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:35.614 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.872544+0000", 2024-04-20T01:48:35.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2841000000000003e-05 2024-04-20T01:48:35.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:35.614 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.873203+0000", 2024-04-20T01:48:35.614 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00065825699999999998 2024-04-20T01:48:35.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.614 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:35.614 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.875894+0000", 2024-04-20T01:48:35.615 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0026911959999999999 2024-04-20T01:48:35.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.615 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:35.615 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.880878+0000", 2024-04-20T01:48:35.615 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0049847559999999999 2024-04-20T01:48:35.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.615 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.615 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:35.615 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.880960+0000", 2024-04-20T01:48:35.615 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.1835000000000005e-05 2024-04-20T01:48:35.615 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.615 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.615 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.616 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:642 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [append 0~507904] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:35.616 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.874202+0000", 2024-04-20T01:48:35.616 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 25.702844891000002, 2024-04-20T01:48:35.616 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0093451449999999991, 2024-04-20T01:48:35.616 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:35.616 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:35.616 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:35.616 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:35.616 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:35.616 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 642 2024-04-20T01:48:35.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.616 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:35.616 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.616 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:35.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874202+0000", 2024-04-20T01:48:35.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:35.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874202+0000", 2024-04-20T01:48:35.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.617 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.617 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:35.617 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874201+0000", 2024-04-20T01:48:35.617 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:35.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:35.618 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874787+0000", 2024-04-20T01:48:35.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00058594599999999995 2024-04-20T01:48:35.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:35.618 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874796+0000", 2024-04-20T01:48:35.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2730000000000008e-06 2024-04-20T01:48:35.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.618 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.618 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:35.618 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874802+0000", 2024-04-20T01:48:35.618 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3619999999999997e-06 2024-04-20T01:48:35.618 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874835+0000", 2024-04-20T01:48:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3577000000000001e-05 2024-04-20T01:48:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.874902+0000", 2024-04-20T01:48:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6989999999999994e-05 2024-04-20T01:48:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.875456+0000", 2024-04-20T01:48:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00055350099999999997 2024-04-20T01:48:35.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.620 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:35.620 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.876715+0000", 2024-04-20T01:48:35.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0012589260000000001 2024-04-20T01:48:35.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.620 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:35.620 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.883407+0000", 2024-04-20T01:48:35.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0066922270000000002 2024-04-20T01:48:35.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.620 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.620 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:35.620 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.883547+0000", 2024-04-20T01:48:35.620 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00014018 2024-04-20T01:48:35.620 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.621 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.621 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.621 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.621 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:643 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [append 0~458752] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:35.621 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.876828+0000", 2024-04-20T01:48:35.621 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 25.700218682999999, 2024-04-20T01:48:35.621 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096858259999999998, 2024-04-20T01:48:35.621 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:35.621 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:35.621 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:35.621 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:35.621 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:35.621 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 643 2024-04-20T01:48:35.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.622 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:35.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:35.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.876828+0000", 2024-04-20T01:48:35.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:35.622 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.876828+0000", 2024-04-20T01:48:35.622 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.622 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.622 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.876827+0000", 2024-04-20T01:48:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877322+0000", 2024-04-20T01:48:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049457899999999998 2024-04-20T01:48:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877333+0000", 2024-04-20T01:48:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1316e-05 2024-04-20T01:48:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:35.623 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877341+0000", 2024-04-20T01:48:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9910000000000002e-06 2024-04-20T01:48:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877374+0000", 2024-04-20T01:48:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3173000000000002e-05 2024-04-20T01:48:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877450+0000", 2024-04-20T01:48:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5864000000000004e-05 2024-04-20T01:48:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.878174+0000", 2024-04-20T01:48:35.624 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00072389000000000004 2024-04-20T01:48:35.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:35.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.880693+0000", 2024-04-20T01:48:35.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0025188430000000002 2024-04-20T01:48:35.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:35.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.886426+0000", 2024-04-20T01:48:35.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0057329540000000002 2024-04-20T01:48:35.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.625 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.625 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:35.625 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.886514+0000", 2024-04-20T01:48:35.625 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8096000000000005e-05 2024-04-20T01:48:35.625 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.626 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.626 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.626 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.626 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:644 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [append 0~81920] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:35.626 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.877354+0000", 2024-04-20T01:48:35.626 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 25.699692698, 2024-04-20T01:48:35.626 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0094386420000000006, 2024-04-20T01:48:35.626 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:35.626 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:35.626 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:35.626 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:35.626 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:35.626 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 644 2024-04-20T01:48:35.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.627 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:35.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:35.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877354+0000", 2024-04-20T01:48:35.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:35.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877354+0000", 2024-04-20T01:48:35.627 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.627 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.627 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:35.627 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877353+0000", 2024-04-20T01:48:35.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:35.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:35.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877455+0000", 2024-04-20T01:48:35.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000101229 2024-04-20T01:48:35.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:35.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877467+0000", 2024-04-20T01:48:35.628 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2061e-05 2024-04-20T01:48:35.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.628 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.628 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:35.628 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877474+0000", 2024-04-20T01:48:35.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5730000000000002e-06 2024-04-20T01:48:35.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.878740+0000", 2024-04-20T01:48:35.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001266145 2024-04-20T01:48:35.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:35.629 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.878801+0000", 2024-04-20T01:48:35.629 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0432e-05 2024-04-20T01:48:35.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.629 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.629 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:35.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.879293+0000", 2024-04-20T01:48:35.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00049227199999999998 2024-04-20T01:48:35.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:35.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.881226+0000", 2024-04-20T01:48:35.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00193319 2024-04-20T01:48:35.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.630 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.630 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:35.630 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.886717+0000", 2024-04-20T01:48:35.630 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0054906019999999998 2024-04-20T01:48:35.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.631 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:35.631 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.886793+0000", 2024-04-20T01:48:35.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5957999999999993e-05 2024-04-20T01:48:35.631 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.631 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.631 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.631 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.631 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:645 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [setxattr _header (56) in=7b] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:35.631 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.877483+0000", 2024-04-20T01:48:35.631 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 25.699563792999999, 2024-04-20T01:48:35.631 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011773402000000001, 2024-04-20T01:48:35.631 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:35.631 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:35.632 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:35.632 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:35.632 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:35.632 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 645 2024-04-20T01:48:35.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.632 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:35.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:35.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877483+0000", 2024-04-20T01:48:35.632 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.632 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.632 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:35.632 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877483+0000", 2024-04-20T01:48:35.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.633 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:35.633 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877482+0000", 2024-04-20T01:48:35.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:35.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.633 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:35.633 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877496+0000", 2024-04-20T01:48:35.633 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3591000000000001e-05 2024-04-20T01:48:35.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.633 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.633 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:35.633 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877504+0000", 2024-04-20T01:48:35.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9880000000000004e-06 2024-04-20T01:48:35.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:35.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877510+0000", 2024-04-20T01:48:35.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1970000000000001e-06 2024-04-20T01:48:35.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.634 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.879785+0000", 2024-04-20T01:48:35.634 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022748339999999999 2024-04-20T01:48:35.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.634 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:35.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.879861+0000", 2024-04-20T01:48:35.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5965999999999997e-05 2024-04-20T01:48:35.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:35.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.880287+0000", 2024-04-20T01:48:35.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00042561299999999998 2024-04-20T01:48:35.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:35.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.881229+0000", 2024-04-20T01:48:35.635 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00094196299999999998 2024-04-20T01:48:35.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.635 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.635 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:35.635 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.889146+0000", 2024-04-20T01:48:35.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0079171320000000003 2024-04-20T01:48:35.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.636 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:35.636 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.889256+0000", 2024-04-20T01:48:35.636 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011071899999999999 2024-04-20T01:48:35.636 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.636 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.636 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.636 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.636 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:646 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:35.636 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.877516+0000", 2024-04-20T01:48:35.636 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 25.699531021999999, 2024-04-20T01:48:35.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.017708380999999999, 2024-04-20T01:48:35.637 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:35.637 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:35.637 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:35.637 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:35.637 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:35.637 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 646 2024-04-20T01:48:35.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.637 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:35.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.637 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:35.637 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877516+0000", 2024-04-20T01:48:35.637 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.637 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:35.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877516+0000", 2024-04-20T01:48:35.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:35.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877515+0000", 2024-04-20T01:48:35.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:35.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.638 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.638 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:35.638 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877518+0000", 2024-04-20T01:48:35.638 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8940000000000002e-06 2024-04-20T01:48:35.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877530+0000", 2024-04-20T01:48:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2103000000000001e-05 2024-04-20T01:48:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877536+0000", 2024-04-20T01:48:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8390000000000003e-06 2024-04-20T01:48:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.880487+0000", 2024-04-20T01:48:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0029510949999999999 2024-04-20T01:48:35.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:35.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.880557+0000", 2024-04-20T01:48:35.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9443999999999994e-05 2024-04-20T01:48:35.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.889586+0000", 2024-04-20T01:48:35.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0090290790000000006 2024-04-20T01:48:35.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.640 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:35.640 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.889691+0000", 2024-04-20T01:48:35.640 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000105386 2024-04-20T01:48:35.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.641 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:35.641 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.895224+0000", 2024-04-20T01:48:35.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.005533044 2024-04-20T01:48:35.641 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.641 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.641 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.641 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.641 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:679 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [copy-from ver 32] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:35.641 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.962971+0000", 2024-04-20T01:48:35.641 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 25.614076089000001, 2024-04-20T01:48:35.641 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011067029000000001, 2024-04-20T01:48:35.642 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:35.642 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:35.642 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:35.642 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:35.642 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:35.642 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 679 2024-04-20T01:48:35.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.642 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:35.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:35.642 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.962971+0000", 2024-04-20T01:48:35.642 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.642 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:35.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.962971+0000", 2024-04-20T01:48:35.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:35.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.962969+0000", 2024-04-20T01:48:35.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:35.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:35.643 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.962976+0000", 2024-04-20T01:48:35.643 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9179999999999997e-06 2024-04-20T01:48:35.643 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.643 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.643 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:35.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.962987+0000", 2024-04-20T01:48:35.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0533000000000001e-05 2024-04-20T01:48:35.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:35.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.962993+0000", 2024-04-20T01:48:35.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2090000000000002e-06 2024-04-20T01:48:35.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.644 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963033+0000", 2024-04-20T01:48:35.644 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0207000000000003e-05 2024-04-20T01:48:35.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.644 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.644 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:35.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963106+0000", 2024-04-20T01:48:35.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3095000000000004e-05 2024-04-20T01:48:35.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:35.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.971796+0000", 2024-04-20T01:48:35.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086893639999999998 2024-04-20T01:48:35.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.645 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:35.645 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.972929+0000", 2024-04-20T01:48:35.645 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011335430000000001 2024-04-20T01:48:35.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.645 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:35.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.973994+0000", 2024-04-20T01:48:35.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001064923 2024-04-20T01:48:35.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.646 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:35.646 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.974038+0000", 2024-04-20T01:48:35.646 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3748000000000002e-05 2024-04-20T01:48:35.646 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.646 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.646 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.646 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.646 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:680 3.6s0 3:6f8ebb9c:::smithi16579500-8:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:35.646 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.963000+0000", 2024-04-20T01:48:35.647 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 25.614046806000001, 2024-04-20T01:48:35.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0095293779999999998, 2024-04-20T01:48:35.647 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:35.647 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:35.647 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:35.647 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:35.647 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:35.647 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 680 2024-04-20T01:48:35.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.647 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:35.647 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.647 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:35.647 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963000+0000", 2024-04-20T01:48:35.647 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:35.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963000+0000", 2024-04-20T01:48:35.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:35.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963000+0000", 2024-04-20T01:48:35.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:35.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.648 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.648 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:35.648 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963002+0000", 2024-04-20T01:48:35.648 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.125e-06 2024-04-20T01:48:35.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:35.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963007+0000", 2024-04-20T01:48:35.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.0529999999999997e-06 2024-04-20T01:48:35.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:35.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963010+0000", 2024-04-20T01:48:35.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4680000000000001e-06 2024-04-20T01:48:35.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.649 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.649 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963239+0000", 2024-04-20T01:48:35.649 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00022873699999999999 2024-04-20T01:48:35.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:35.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.963278+0000", 2024-04-20T01:48:35.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8733000000000003e-05 2024-04-20T01:48:35.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.650 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.972372+0000", 2024-04-20T01:48:35.650 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0090944210000000001 2024-04-20T01:48:35.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.650 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:35.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.972460+0000", 2024-04-20T01:48:35.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7349999999999995e-05 2024-04-20T01:48:35.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.651 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:35.651 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.972529+0000", 2024-04-20T01:48:35.651 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9809999999999997e-05 2024-04-20T01:48:35.651 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.651 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.651 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.651 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.651 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s4 80/57 ECSubWrite(tid=206, reqid=osd.8.0:206, at_version=80'149, trim_to=0'0, roll_forward_to=79'145))", 2024-04-20T01:48:35.652 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:21.738326+0000", 2024-04-20T01:48:35.652 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.838720984, 2024-04-20T01:48:35.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0085236089999999997, 2024-04-20T01:48:35.652 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:35.652 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:35.652 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:35.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:35.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.738326+0000", 2024-04-20T01:48:35.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.652 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.652 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:35.652 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.738326+0000", 2024-04-20T01:48:35.652 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:35.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.738325+0000", 2024-04-20T01:48:35.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:35.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:35.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.738349+0000", 2024-04-20T01:48:35.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4179999999999999e-05 2024-04-20T01:48:35.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.653 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.653 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:35.653 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.738366+0000", 2024-04-20T01:48:35.653 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6781000000000001e-05 2024-04-20T01:48:35.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:35.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.738377+0000", 2024-04-20T01:48:35.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0511e-05 2024-04-20T01:48:35.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.738474+0000", 2024-04-20T01:48:35.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7399999999999996e-05 2024-04-20T01:48:35.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.654 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.654 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:35.654 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.738486+0000", 2024-04-20T01:48:35.654 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1463e-05 2024-04-20T01:48:35.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:35.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.746818+0000", 2024-04-20T01:48:35.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0083327639999999994 2024-04-20T01:48:35.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.655 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:35.655 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.746849+0000", 2024-04-20T01:48:35.655 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.1124999999999998e-05 2024-04-20T01:48:35.655 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.655 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.655 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.655 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.655 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.7s3 80/30 ECSubWrite(tid=308, reqid=osd.4.0:308, at_version=80'30, trim_to=0'0, roll_forward_to=79'28))", 2024-04-20T01:48:35.656 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:21.741270+0000", 2024-04-20T01:48:35.656 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 13.835777151, 2024-04-20T01:48:35.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086490270000000001, 2024-04-20T01:48:35.656 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:35.656 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:35.656 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:35.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:35.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.741270+0000", 2024-04-20T01:48:35.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.656 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.656 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:35.656 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.741270+0000", 2024-04-20T01:48:35.656 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:35.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:35.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.741269+0000", 2024-04-20T01:48:35.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:35.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:35.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.741305+0000", 2024-04-20T01:48:35.657 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6591000000000003e-05 2024-04-20T01:48:35.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.657 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:35.657 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.741323+0000", 2024-04-20T01:48:35.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7734999999999999e-05 2024-04-20T01:48:35.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:35.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.741334+0000", 2024-04-20T01:48:35.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0542e-05 2024-04-20T01:48:35.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:35.658 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.741388+0000", 2024-04-20T01:48:35.658 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4307e-05 2024-04-20T01:48:35.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.658 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.658 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:35.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.741400+0000", 2024-04-20T01:48:35.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2004999999999999e-05 2024-04-20T01:48:35.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:35.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.749892+0000", 2024-04-20T01:48:35.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0084922959999999999 2024-04-20T01:48:35.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:35.659 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:35.659 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:35.659 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.749919+0000", 2024-04-20T01:48:35.659 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6641000000000001e-05 2024-04-20T01:48:35.659 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.659 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.659 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.659 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:35.659 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:35.660 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:35.661 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-20T01:48:35.662 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:35.675+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:35.762 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:35.777+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:35.786 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is unset 2024-04-20T01:48:35.854 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:35.855 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:35.855 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-04-20T01:48:35.855 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:35.862 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:35.877+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:35.866 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-20T01:48:35.892 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-14 --journal-path /var/lib/ceph/osd/ceph-14/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.5s3.14 2024-04-20T01:48:35.963 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:35.977+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:36.012 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:36.012 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:36.013 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-04-20T01:48:36.013 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-04-20T01:48:36.013 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:36.024 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-20T01:48:36.069 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:36.083+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:36.165 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:36.180+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:36.170 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:36.170 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-04-20T01:48:36.170 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-04-20T01:48:36.170 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-04-20T01:48:36.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.171 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:211 3.7s0 3:eb044288:::smithi16579500-44:head [append 0~376832] snapc 3=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:48:36.171 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.778254+0000", 2024-04-20T01:48:36.171 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 58.405491582000003, 2024-04-20T01:48:36.171 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01403427, 2024-04-20T01:48:36.171 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.171 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.171 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.171 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.171 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.171 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 211 2024-04-20T01:48:36.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.171 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.171 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778254+0000", 2024-04-20T01:48:36.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778254+0000", 2024-04-20T01:48:36.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.172 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.172 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.172 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.778252+0000", 2024-04-20T01:48:36.172 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:36.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780895+0000", 2024-04-20T01:48:36.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002642658 2024-04-20T01:48:36.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780910+0000", 2024-04-20T01:48:36.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4949999999999999e-05 2024-04-20T01:48:36.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.173 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.173 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780919+0000", 2024-04-20T01:48:36.173 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7770000000000008e-06 2024-04-20T01:48:36.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.173 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780949+0000", 2024-04-20T01:48:36.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0639999999999998e-05 2024-04-20T01:48:36.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781077+0000", 2024-04-20T01:48:36.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000127565 2024-04-20T01:48:36.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.174 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.174 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781928+0000", 2024-04-20T01:48:36.174 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00085169599999999999 2024-04-20T01:48:36.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.787994+0000", 2024-04-20T01:48:36.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00606545 2024-04-20T01:48:36.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.792213+0000", 2024-04-20T01:48:36.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0042187199999999996 2024-04-20T01:48:36.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.175 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.175 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.175 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.792289+0000", 2024-04-20T01:48:36.175 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5980000000000001e-05 2024-04-20T01:48:36.175 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.175 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.175 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.176 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.176 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:213 3.7s0 3:eb044288:::smithi16579500-44:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:48:36.176 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.780960+0000", 2024-04-20T01:48:36.176 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 58.402785500999997, 2024-04-20T01:48:36.176 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015313724000000001, 2024-04-20T01:48:36.176 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.176 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:36.176 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.176 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.176 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.176 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 213 2024-04-20T01:48:36.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.176 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780960+0000", 2024-04-20T01:48:36.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780960+0000", 2024-04-20T01:48:36.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.177 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.177 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.177 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780960+0000", 2024-04-20T01:48:36.177 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:36.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780964+0000", 2024-04-20T01:48:36.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6610000000000002e-06 2024-04-20T01:48:36.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780971+0000", 2024-04-20T01:48:36.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.9270000000000005e-06 2024-04-20T01:48:36.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.178 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.178 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.178 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.780978+0000", 2024-04-20T01:48:36.178 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1670000000000004e-06 2024-04-20T01:48:36.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.786222+0000", 2024-04-20T01:48:36.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0052446979999999999 2024-04-20T01:48:36.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:36.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.786324+0000", 2024-04-20T01:48:36.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000101801 2024-04-20T01:48:36.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.179 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.179 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.179 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.793221+0000", 2024-04-20T01:48:36.179 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068970510000000004 2024-04-20T01:48:36.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.180 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.793323+0000", 2024-04-20T01:48:36.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102289 2024-04-20T01:48:36.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.180 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.180 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.796274+0000", 2024-04-20T01:48:36.180 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0029506409999999999 2024-04-20T01:48:36.180 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.180 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.180 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.180 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.181 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:214 3.bs0 3:dc0bbe39:::smithi16579500-13:head [stat,delete] snapc 3=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:48:36.181 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.781222+0000", 2024-04-20T01:48:36.181 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 58.402524053999997, 2024-04-20T01:48:36.181 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015584361, 2024-04-20T01:48:36.181 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.181 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.181 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.181 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.181 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.181 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 214 2024-04-20T01:48:36.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.181 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.181 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.181 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.182 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781222+0000", 2024-04-20T01:48:36.182 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.182 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.182 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781222+0000", 2024-04-20T01:48:36.182 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.182 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.182 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781221+0000", 2024-04-20T01:48:36.182 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:36.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.182 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781228+0000", 2024-04-20T01:48:36.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6699999999999997e-06 2024-04-20T01:48:36.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781239+0000", 2024-04-20T01:48:36.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1574e-05 2024-04-20T01:48:36.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.183 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781247+0000", 2024-04-20T01:48:36.183 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.593e-06 2024-04-20T01:48:36.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.183 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.183 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781288+0000", 2024-04-20T01:48:36.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.1253e-05 2024-04-20T01:48:36.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.781383+0000", 2024-04-20T01:48:36.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4788000000000001e-05 2024-04-20T01:48:36.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.184 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.782250+0000", 2024-04-20T01:48:36.184 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00086664399999999996 2024-04-20T01:48:36.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.184 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.786611+0000", 2024-04-20T01:48:36.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004361714 2024-04-20T01:48:36.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.796739+0000", 2024-04-20T01:48:36.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010127519 2024-04-20T01:48:36.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.185 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.185 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.185 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.796806+0000", 2024-04-20T01:48:36.185 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7414000000000002e-05 2024-04-20T01:48:36.185 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.185 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.185 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.186 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.186 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:225 3.bs0 3:df90e529:::smithi16579500-45:head [append 0~622592] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:36.186 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.785410+0000", 2024-04-20T01:48:36.186 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 57.398335543000002, 2024-04-20T01:48:36.186 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013791425, 2024-04-20T01:48:36.186 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.186 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.186 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.186 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.186 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.186 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 225 2024-04-20T01:48:36.186 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.187 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.187 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.187 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.785410+0000", 2024-04-20T01:48:36.187 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.187 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.187 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.785410+0000", 2024-04-20T01:48:36.187 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.187 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.187 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.187 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.785407+0000", 2024-04-20T01:48:36.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999967 2024-04-20T01:48:36.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.786402+0000", 2024-04-20T01:48:36.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00099532000000000006 2024-04-20T01:48:36.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.786427+0000", 2024-04-20T01:48:36.188 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5304e-05 2024-04-20T01:48:36.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.188 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.188 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.188 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.786440+0000", 2024-04-20T01:48:36.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2238999999999999e-05 2024-04-20T01:48:36.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.189 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.189 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.786515+0000", 2024-04-20T01:48:36.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4923000000000001e-05 2024-04-20T01:48:36.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.189 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.189 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.786652+0000", 2024-04-20T01:48:36.189 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000137522 2024-04-20T01:48:36.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.189 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788170+0000", 2024-04-20T01:48:36.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001518003 2024-04-20T01:48:36.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796858+0000", 2024-04-20T01:48:36.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086877320000000001 2024-04-20T01:48:36.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.190 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.190 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.799118+0000", 2024-04-20T01:48:36.190 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022599600000000001 2024-04-20T01:48:36.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.190 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.191 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.191 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.799202+0000", 2024-04-20T01:48:36.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3954000000000003e-05 2024-04-20T01:48:36.191 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.191 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.191 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.191 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.191 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:227 3.bs0 3:df90e529:::smithi16579500-45:head [setxattr _header (56) in=7b] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:36.191 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787843+0000", 2024-04-20T01:48:36.191 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 57.395903265999998, 2024-04-20T01:48:36.191 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014686189000000001, 2024-04-20T01:48:36.191 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.191 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.192 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.192 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.192 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.192 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 227 2024-04-20T01:48:36.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.192 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.192 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.192 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787843+0000", 2024-04-20T01:48:36.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.192 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.192 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.192 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787843+0000", 2024-04-20T01:48:36.192 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.193 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.193 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787842+0000", 2024-04-20T01:48:36.193 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:36.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.193 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.193 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787847+0000", 2024-04-20T01:48:36.193 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8130000000000001e-06 2024-04-20T01:48:36.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.193 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.193 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.193 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787853+0000", 2024-04-20T01:48:36.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5119999999999999e-06 2024-04-20T01:48:36.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787859+0000", 2024-04-20T01:48:36.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8119999999999996e-06 2024-04-20T01:48:36.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.194 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.790718+0000", 2024-04-20T01:48:36.194 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002859267 2024-04-20T01:48:36.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.194 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.194 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.790827+0000", 2024-04-20T01:48:36.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000109471 2024-04-20T01:48:36.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791236+0000", 2024-04-20T01:48:36.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000408637 2024-04-20T01:48:36.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.195 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.796922+0000", 2024-04-20T01:48:36.195 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0056861139999999999 2024-04-20T01:48:36.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.195 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.195 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.802464+0000", 2024-04-20T01:48:36.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0055420449999999998 2024-04-20T01:48:36.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.196 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.196 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.802529+0000", 2024-04-20T01:48:36.196 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4554000000000006e-05 2024-04-20T01:48:36.196 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.196 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.196 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.196 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.196 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:228 3.bs0 3:df90e529:::smithi16579500-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:36.196 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787871+0000", 2024-04-20T01:48:36.196 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 57.395874550999999, 2024-04-20T01:48:36.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.016315307000000001, 2024-04-20T01:48:36.197 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.197 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:36.197 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.197 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.197 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.197 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 228 2024-04-20T01:48:36.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.197 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.197 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.197 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787871+0000", 2024-04-20T01:48:36.197 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.197 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.198 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.198 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787871+0000", 2024-04-20T01:48:36.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.198 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.198 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787871+0000", 2024-04-20T01:48:36.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:36.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.198 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.198 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.198 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787873+0000", 2024-04-20T01:48:36.198 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.08e-06 2024-04-20T01:48:36.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.199 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787878+0000", 2024-04-20T01:48:36.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7299999999999996e-06 2024-04-20T01:48:36.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.199 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787881+0000", 2024-04-20T01:48:36.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5209999999999998e-06 2024-04-20T01:48:36.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.199 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.199 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.199 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791461+0000", 2024-04-20T01:48:36.199 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.003579905 2024-04-20T01:48:36.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:36.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791554+0000", 2024-04-20T01:48:36.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2485000000000006e-05 2024-04-20T01:48:36.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.802822+0000", 2024-04-20T01:48:36.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011268251 2024-04-20T01:48:36.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.200 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.200 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.200 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.802922+0000", 2024-04-20T01:48:36.200 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010029700000000001 2024-04-20T01:48:36.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.201 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.201 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.804187+0000", 2024-04-20T01:48:36.201 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001264362 2024-04-20T01:48:36.201 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.201 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.201 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.201 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.201 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:385 3.1s0 3:8bb12819:::smithi16579500-21:head [copy-from ver 2] snapc d=[c,b] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-20T01:48:36.201 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.710205+0000", 2024-04-20T01:48:36.201 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.473540499999999, 2024-04-20T01:48:36.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.016348672000000002, 2024-04-20T01:48:36.202 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.202 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.202 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.202 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.202 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.202 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 385 2024-04-20T01:48:36.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.202 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.202 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.202 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.202 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710205+0000", 2024-04-20T01:48:36.202 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.203 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.203 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710205+0000", 2024-04-20T01:48:36.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.203 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.203 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710204+0000", 2024-04-20T01:48:36.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:36.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.203 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.203 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.203 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710212+0000", 2024-04-20T01:48:36.203 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.8250000000000005e-06 2024-04-20T01:48:36.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.204 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.204 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710222+0000", 2024-04-20T01:48:36.204 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0071000000000001e-05 2024-04-20T01:48:36.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.204 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.204 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710232+0000", 2024-04-20T01:48:36.204 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0583e-05 2024-04-20T01:48:36.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.204 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.204 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.204 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710273+0000", 2024-04-20T01:48:36.204 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0407999999999997e-05 2024-04-20T01:48:36.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.205 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.205 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.710360+0000", 2024-04-20T01:48:36.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6778000000000002e-05 2024-04-20T01:48:36.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.205 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.205 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.719733+0000", 2024-04-20T01:48:36.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0093733800000000006 2024-04-20T01:48:36.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.205 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.205 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.205 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.723689+0000", 2024-04-20T01:48:36.205 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0039562620000000003 2024-04-20T01:48:36.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.726462+0000", 2024-04-20T01:48:36.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0027724989999999999 2024-04-20T01:48:36.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.206 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.206 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.206 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.726554+0000", 2024-04-20T01:48:36.206 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.2262000000000005e-05 2024-04-20T01:48:36.206 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.206 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.206 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.207 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s2 48/30 ECSubWrite(tid=86, reqid=client.4643.0:400, at_version=48'72, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:36.207 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.713256+0000", 2024-04-20T01:48:36.207 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 46.470489751999999, 2024-04-20T01:48:36.207 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.017021191000000001, 2024-04-20T01:48:36.207 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.207 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:36.207 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.207 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.207 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713256+0000", 2024-04-20T01:48:36.207 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.207 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.208 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.208 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713256+0000", 2024-04-20T01:48:36.208 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.208 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.208 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713255+0000", 2024-04-20T01:48:36.208 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:36.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.208 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.208 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713277+0000", 2024-04-20T01:48:36.208 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1348e-05 2024-04-20T01:48:36.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.208 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713289+0000", 2024-04-20T01:48:36.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2692e-05 2024-04-20T01:48:36.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713299+0000", 2024-04-20T01:48:36.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0071999999999999e-05 2024-04-20T01:48:36.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.209 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.209 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.209 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714467+0000", 2024-04-20T01:48:36.209 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001167581 2024-04-20T01:48:36.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714478+0000", 2024-04-20T01:48:36.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1029999999999999e-05 2024-04-20T01:48:36.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730237+0000", 2024-04-20T01:48:36.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015758635 2024-04-20T01:48:36.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.210 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.210 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.210 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730277+0000", 2024-04-20T01:48:36.210 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0788999999999999e-05 2024-04-20T01:48:36.211 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.211 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.211 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.211 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.4s2 48/30 ECSubWrite(tid=87, reqid=client.4643.0:401, at_version=48'73, trim_to=0'0, roll_forward_to=47'65))", 2024-04-20T01:48:36.211 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.714498+0000", 2024-04-20T01:48:36.211 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 46.469248159000003, 2024-04-20T01:48:36.211 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015921002, 2024-04-20T01:48:36.211 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.211 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:36.211 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.211 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.211 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.212 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714498+0000", 2024-04-20T01:48:36.212 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.212 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.212 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714498+0000", 2024-04-20T01:48:36.212 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.212 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.212 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714497+0000", 2024-04-20T01:48:36.212 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:36.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.212 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.213 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.213 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714519+0000", 2024-04-20T01:48:36.213 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2574000000000001e-05 2024-04-20T01:48:36.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.213 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.213 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714531+0000", 2024-04-20T01:48:36.213 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1361e-05 2024-04-20T01:48:36.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.213 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.213 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714540+0000", 2024-04-20T01:48:36.213 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4769999999999992e-06 2024-04-20T01:48:36.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.213 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.214 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.214 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.716648+0000", 2024-04-20T01:48:36.214 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002108183 2024-04-20T01:48:36.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.214 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.214 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.716661+0000", 2024-04-20T01:48:36.214 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2490000000000001e-05 2024-04-20T01:48:36.214 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.214 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.214 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.214 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730395+0000", 2024-04-20T01:48:36.214 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013734333 2024-04-20T01:48:36.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.215 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.215 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730419+0000", 2024-04-20T01:48:36.215 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3489000000000002e-05 2024-04-20T01:48:36.215 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.215 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.215 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.215 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.3s2 48/30 ECSubWrite(tid=254, reqid=osd.13.0:251, at_version=0'0, trim_to=0'0, roll_forward_to=48'22))", 2024-04-20T01:48:36.215 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.719206+0000", 2024-04-20T01:48:36.215 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 46.46453966, 2024-04-20T01:48:36.215 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014281428000000001, 2024-04-20T01:48:36.215 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.216 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:36.216 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.216 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.216 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719206+0000", 2024-04-20T01:48:36.216 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.216 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.216 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719206+0000", 2024-04-20T01:48:36.216 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.216 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.216 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.216 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719206+0000", 2024-04-20T01:48:36.217 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:36.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.217 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.217 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719215+0000", 2024-04-20T01:48:36.217 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6229999999999992e-06 2024-04-20T01:48:36.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.217 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.217 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719237+0000", 2024-04-20T01:48:36.217 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1877000000000001e-05 2024-04-20T01:48:36.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.217 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.217 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.218 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.719249+0000", 2024-04-20T01:48:36.218 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1736000000000001e-05 2024-04-20T01:48:36.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.218 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.218 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.726201+0000", 2024-04-20T01:48:36.218 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0069514930000000004 2024-04-20T01:48:36.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.218 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.218 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.726206+0000", 2024-04-20T01:48:36.218 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7830000000000004e-06 2024-04-20T01:48:36.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.218 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.219 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.219 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.733466+0000", 2024-04-20T01:48:36.219 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072593550000000003 2024-04-20T01:48:36.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.219 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.219 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.733488+0000", 2024-04-20T01:48:36.219 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2007000000000001e-05 2024-04-20T01:48:36.219 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.219 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.219 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.219 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.219 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:493 3.1s0 3:8bb12819:::smithi16579500-21:head [copy-from ver 27] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:36.219 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.773406+0000", 2024-04-20T01:48:36.220 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 36.410340247999997, 2024-04-20T01:48:36.220 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014247365999999999, 2024-04-20T01:48:36.220 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.220 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.220 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.220 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.220 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.220 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 493 2024-04-20T01:48:36.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.220 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.220 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.220 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.220 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773406+0000", 2024-04-20T01:48:36.221 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.221 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.221 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773406+0000", 2024-04-20T01:48:36.221 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.221 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.221 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773404+0000", 2024-04-20T01:48:36.221 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:36.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.221 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.221 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.221 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773412+0000", 2024-04-20T01:48:36.222 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7309999999999993e-06 2024-04-20T01:48:36.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.222 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.222 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773427+0000", 2024-04-20T01:48:36.222 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5167999999999999e-05 2024-04-20T01:48:36.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.222 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.222 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773435+0000", 2024-04-20T01:48:36.222 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.4710000000000007e-06 2024-04-20T01:48:36.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.222 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.222 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.223 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773496+0000", 2024-04-20T01:48:36.223 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.0325000000000003e-05 2024-04-20T01:48:36.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.223 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op must wait for map", 2024-04-20T01:48:36.223 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.773506+0000", 2024-04-20T01:48:36.223 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9739999999999997e-06 2024-04-20T01:48:36.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.223 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.223 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.775199+0000", 2024-04-20T01:48:36.223 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001693214 2024-04-20T01:48:36.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.223 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.224 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.224 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.775299+0000", 2024-04-20T01:48:36.224 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000100079 2024-04-20T01:48:36.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.224 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.224 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.782700+0000", 2024-04-20T01:48:36.224 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0074012640000000003 2024-04-20T01:48:36.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.224 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.224 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.787277+0000", 2024-04-20T01:48:36.224 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045765349999999996 2024-04-20T01:48:36.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.224 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.225 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.225 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.787341+0000", 2024-04-20T01:48:36.225 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3998000000000006e-05 2024-04-20T01:48:36.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.225 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.225 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.787653+0000", 2024-04-20T01:48:36.225 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00031239000000000001 2024-04-20T01:48:36.225 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.225 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.225 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.225 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.225 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.225 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:584 3.bs0 3:d86ca78e:::smithi16579500-26:1f [read 0~4816896,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-04-20T01:48:36.225 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:05.758316+0000", 2024-04-20T01:48:36.226 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 30.425429678, 2024-04-20T01:48:36.226 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015613514, 2024-04-20T01:48:36.226 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.226 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:36.226 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.226 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.226 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.226 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 584 2024-04-20T01:48:36.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.226 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.226 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.226 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.226 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.758316+0000", 2024-04-20T01:48:36.227 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.227 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.227 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.758316+0000", 2024-04-20T01:48:36.227 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.227 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.227 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.758316+0000", 2024-04-20T01:48:36.227 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:36.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.227 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.227 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.228 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.758321+0000", 2024-04-20T01:48:36.228 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9099999999999996e-06 2024-04-20T01:48:36.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.228 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.228 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.228 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.758328+0000", 2024-04-20T01:48:36.228 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4619999999999999e-06 2024-04-20T01:48:36.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.228 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.228 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.228 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.758334+0000", 2024-04-20T01:48:36.228 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7960000000000001e-06 2024-04-20T01:48:36.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.228 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.228 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.229 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.759436+0000", 2024-04-20T01:48:36.229 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0011024730000000001 2024-04-20T01:48:36.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.229 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.229 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.759539+0000", 2024-04-20T01:48:36.229 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102896 2024-04-20T01:48:36.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.229 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.229 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.229 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:05.773930+0000", 2024-04-20T01:48:36.229 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014390464 2024-04-20T01:48:36.229 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.229 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.229 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.230 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.230 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:650 3.bs0 3:da81107c:::smithi16579500-34:head [copy-from ver 38] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:36.230 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.877476+0000", 2024-04-20T01:48:36.230 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 26.306270390999998, 2024-04-20T01:48:36.230 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014109823000000001, 2024-04-20T01:48:36.230 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.230 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.230 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.230 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.230 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.230 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 650 2024-04-20T01:48:36.230 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.231 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.231 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.231 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877476+0000", 2024-04-20T01:48:36.231 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.231 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.231 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877476+0000", 2024-04-20T01:48:36.231 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.231 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.231 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877470+0000", 2024-04-20T01:48:36.232 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999943 2024-04-20T01:48:36.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.232 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.232 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877485+0000", 2024-04-20T01:48:36.232 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4973e-05 2024-04-20T01:48:36.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.232 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.232 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877509+0000", 2024-04-20T01:48:36.232 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4056000000000001e-05 2024-04-20T01:48:36.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.232 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.232 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.233 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877521+0000", 2024-04-20T01:48:36.233 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2462e-05 2024-04-20T01:48:36.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.233 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.233 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877587+0000", 2024-04-20T01:48:36.233 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6008999999999998e-05 2024-04-20T01:48:36.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.233 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.233 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.877709+0000", 2024-04-20T01:48:36.233 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000121537 2024-04-20T01:48:36.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.233 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.234 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.234 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.887306+0000", 2024-04-20T01:48:36.234 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0095964940000000006 2024-04-20T01:48:36.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.234 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.234 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.889750+0000", 2024-04-20T01:48:36.234 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0024441659999999998 2024-04-20T01:48:36.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.234 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.234 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.234 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.891529+0000", 2024-04-20T01:48:36.234 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001778916 2024-04-20T01:48:36.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.235 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.235 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.891585+0000", 2024-04-20T01:48:36.235 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6725999999999999e-05 2024-04-20T01:48:36.235 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.235 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.235 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.235 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.235 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:715 3.bs0 3:dc0bbe39:::smithi16579500-13:head [copy-from ver 32] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:36.235 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:10.113717+0000", 2024-04-20T01:48:36.235 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 26.070028693000001, 2024-04-20T01:48:36.235 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.020694802000000002, 2024-04-20T01:48:36.235 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.235 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.236 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.236 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.236 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.236 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 715 2024-04-20T01:48:36.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.236 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.236 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.236 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.236 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113717+0000", 2024-04-20T01:48:36.236 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.236 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.236 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.236 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113717+0000", 2024-04-20T01:48:36.236 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.237 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.237 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113715+0000", 2024-04-20T01:48:36.237 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:36.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.237 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.237 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113731+0000", 2024-04-20T01:48:36.237 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6231000000000001e-05 2024-04-20T01:48:36.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.237 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.237 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.237 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113751+0000", 2024-04-20T01:48:36.237 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9666000000000001e-05 2024-04-20T01:48:36.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.238 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.238 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113762+0000", 2024-04-20T01:48:36.238 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0596e-05 2024-04-20T01:48:36.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.238 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.238 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113837+0000", 2024-04-20T01:48:36.238 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.5783000000000003e-05 2024-04-20T01:48:36.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.238 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.238 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.238 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113990+0000", 2024-04-20T01:48:36.239 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000152118 2024-04-20T01:48:36.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.239 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.239 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.127616+0000", 2024-04-20T01:48:36.239 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013625906 2024-04-20T01:48:36.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.239 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.239 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.129856+0000", 2024-04-20T01:48:36.239 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022408490000000001 2024-04-20T01:48:36.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.239 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.239 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.239 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.134361+0000", 2024-04-20T01:48:36.240 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045043349999999999 2024-04-20T01:48:36.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.240 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.240 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.134412+0000", 2024-04-20T01:48:36.240 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1310999999999997e-05 2024-04-20T01:48:36.240 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.240 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.240 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.240 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.240 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:716 3.bs0 3:dc0bbe39:::smithi16579500-13:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:36.240 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:10.113774+0000", 2024-04-20T01:48:36.240 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 26.069972248999999, 2024-04-20T01:48:36.241 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015615891, 2024-04-20T01:48:36.241 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.241 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:36.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.241 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.241 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 716 2024-04-20T01:48:36.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.241 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.241 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.241 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113774+0000", 2024-04-20T01:48:36.241 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.241 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.242 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113774+0000", 2024-04-20T01:48:36.242 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.242 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113773+0000", 2024-04-20T01:48:36.242 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:36.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.242 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.242 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.242 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113776+0000", 2024-04-20T01:48:36.242 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.61e-06 2024-04-20T01:48:36.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.243 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.243 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113781+0000", 2024-04-20T01:48:36.243 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.6369999999999998e-06 2024-04-20T01:48:36.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.243 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.243 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113784+0000", 2024-04-20T01:48:36.243 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4489999999999999e-06 2024-04-20T01:48:36.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.243 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.243 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.243 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.114218+0000", 2024-04-20T01:48:36.244 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00043373499999999998 2024-04-20T01:48:36.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.244 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:36.244 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.114253+0000", 2024-04-20T01:48:36.244 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5410000000000001e-05 2024-04-20T01:48:36.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.244 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.244 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.129136+0000", 2024-04-20T01:48:36.244 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014882532 2024-04-20T01:48:36.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.244 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.244 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.244 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.129263+0000", 2024-04-20T01:48:36.245 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000126919 2024-04-20T01:48:36.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.245 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.245 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.129390+0000", 2024-04-20T01:48:36.245 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000126957 2024-04-20T01:48:36.245 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.245 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.245 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.245 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.245 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:720 3.7s0 3:e950587e:::smithi16579500-49:head [copy-from ver 100] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:36.245 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:10.113830+0000", 2024-04-20T01:48:36.245 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 26.069916193000001, 2024-04-20T01:48:36.245 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.016212712000000001, 2024-04-20T01:48:36.246 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.246 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.246 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.246 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.246 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.246 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 720 2024-04-20T01:48:36.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.246 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.246 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.246 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113830+0000", 2024-04-20T01:48:36.246 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.246 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.246 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113830+0000", 2024-04-20T01:48:36.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113829+0000", 2024-04-20T01:48:36.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:36.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.247 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113833+0000", 2024-04-20T01:48:36.247 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2239999999999997e-06 2024-04-20T01:48:36.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.247 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.247 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.248 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113843+0000", 2024-04-20T01:48:36.248 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6220000000000004e-06 2024-04-20T01:48:36.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.248 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.248 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113847+0000", 2024-04-20T01:48:36.248 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4150000000000001e-06 2024-04-20T01:48:36.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.248 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.248 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113890+0000", 2024-04-20T01:48:36.248 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2287999999999999e-05 2024-04-20T01:48:36.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.248 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.248 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.249 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.113987+0000", 2024-04-20T01:48:36.249 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7149000000000001e-05 2024-04-20T01:48:36.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.249 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.249 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.124314+0000", 2024-04-20T01:48:36.249 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010327325 2024-04-20T01:48:36.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.249 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.249 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.128903+0000", 2024-04-20T01:48:36.249 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0045891949999999999 2024-04-20T01:48:36.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.249 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.250 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.129948+0000", 2024-04-20T01:48:36.250 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0010448409999999999 2024-04-20T01:48:36.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.250 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.250 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.250 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.130042+0000", 2024-04-20T01:48:36.250 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.4158000000000002e-05 2024-04-20T01:48:36.250 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.250 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.250 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.250 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.250 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:783 3.bs0 3:d8d507c1:::smithi16579500-31:head [create,setxattr ]pZOqXa[dZ]SgadRhTYkaVmms]loYXV]OhjYgpQb (558) in=40b,setxattr tY^OWiahkbeO^^QXeQkskkmiqmg]^idV]]Wa (945) in=36b,setxattr ddsnSab`pWQQbVa^PY_ZP\\YZWq`iW]YPicUPUh (945) in=38b,setxattr f]\\heUc_VXgVXWYScTg_cOpgRekOSNq (945) in=31b,setxattr acUl[g`Obg (945) in=10b,setxattr OfSUabmdN^X_rZU`aai`O]e^cSeoSjpUR]obY_Y^ (17824) in=40b,setxattr ^Zotgjmkt]eXjWi\\Tsd]sWTiUkT\\r_diXa^ecsVT (557) in=40b,setxattr qUenNgU\\cb`[WW_fr]XX]kX[cPqhYt\\YtUSfs]T[ (1227) in=40b,setxattr SNqcNPWmeZRq_\\cnZRprZ_QZ]Pnnga_fpsVl (945) in=36b,setxattr _lqa`PaWRk\\gjqtki]amP[Uj]Pp\\T]TkSrdd]gai (771) in=40b,setxattr _S]YVmre[VNi]bcbZ[Rrgql (945) in=23b,setxattr pWktdcOsRSfRQ[_p^[csnUlhN`tgS]imf\\[`oleX (1015) in=40b,setxattr kcaO\\qSco_ObZP^qWjYlY[a]bpYRO`t\\cnQZNqsS (877) in=40b,setxattr ]lcnY\\pohsUfNjfgp`jb]^oXfRta__l`jjOY`iOk (174) in=40b,setxattr T`NOae[on`a_soaoWQ_irbZpdd_^NT]aVpXTr^fa (1438) in=40b,setxattr `aheahaRqeqg_YgcYglVNh\\fkl`gd]opppUSOR[e (11996) in=40b,setxattr n[QWooeNtePf_\\]^TpS^_YmQgQh]dmQSWqUVNrl^ (243) in=40b,setxattr o_hhpPi[PnXZtoj\\Q] (945) in=18b,setxattr PiUqpcbjVsW_[lgpZXliUgOpPRas\\\\a[TZQm[onO (487) in=40b,setxattr gjp\\eY`ptqelkrVpfW[r^ilVhqmpc[^Un\\^Tgfhq (1610) in=40b] snapc 2a=[2a,28] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-04-20T01:48:36.251 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:16.724535+0000", 2024-04-20T01:48:36.251 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 19.459210654, 2024-04-20T01:48:36.251 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.026026878999999999, 2024-04-20T01:48:36.251 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.251 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.251 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.251 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.251 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.251 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 783 2024-04-20T01:48:36.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.251 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.251 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.251 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.251 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.724535+0000", 2024-04-20T01:48:36.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.724535+0000", 2024-04-20T01:48:36.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.724533+0000", 2024-04-20T01:48:36.252 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:36.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.252 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.252 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.252 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.725134+0000", 2024-04-20T01:48:36.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00060092299999999995 2024-04-20T01:48:36.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.253 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.725152+0000", 2024-04-20T01:48:36.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8297000000000001e-05 2024-04-20T01:48:36.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.253 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.725164+0000", 2024-04-20T01:48:36.253 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.204e-05 2024-04-20T01:48:36.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.253 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.729187+0000", 2024-04-20T01:48:36.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0040223999999999998 2024-04-20T01:48:36.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.729293+0000", 2024-04-20T01:48:36.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000106703 2024-04-20T01:48:36.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.254 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.254 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.730663+0000", 2024-04-20T01:48:36.254 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0013691300000000001 2024-04-20T01:48:36.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.254 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.734534+0000", 2024-04-20T01:48:36.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0038714470000000001 2024-04-20T01:48:36.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.750462+0000", 2024-04-20T01:48:36.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015927802000000001 2024-04-20T01:48:36.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.255 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.255 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.255 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.750562+0000", 2024-04-20T01:48:36.255 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010027900000000001 2024-04-20T01:48:36.255 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.255 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.256 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.256 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.256 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:1004 3.7s0 3:e950587e:::smithi16579500-49:head [append 0~278528] snapc 38=[36,35,31] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:36.256 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:31.807822+0000", 2024-04-20T01:48:36.256 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.3759242780000003, 2024-04-20T01:48:36.256 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.935333478, 2024-04-20T01:48:36.256 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.256 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.256 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.256 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.256 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.256 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1004 2024-04-20T01:48:36.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.256 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.807822+0000", 2024-04-20T01:48:36.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.807822+0000", 2024-04-20T01:48:36.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.257 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.257 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.257 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.807819+0000", 2024-04-20T01:48:36.257 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:36.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808222+0000", 2024-04-20T01:48:36.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00040291099999999999 2024-04-20T01:48:36.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808237+0000", 2024-04-20T01:48:36.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5002e-05 2024-04-20T01:48:36.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.258 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.258 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.258 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808245+0000", 2024-04-20T01:48:36.258 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4170000000000003e-06 2024-04-20T01:48:36.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.259 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808317+0000", 2024-04-20T01:48:36.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.2002000000000006e-05 2024-04-20T01:48:36.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.259 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-04-20T01:48:36.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808320+0000", 2024-04-20T01:48:36.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3689999999999998e-06 2024-04-20T01:48:36.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.259 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.259 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.259 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.733738+0000", 2024-04-20T01:48:36.259 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.925417991 2024-04-20T01:48:36.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.260 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.260 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.733934+0000", 2024-04-20T01:48:36.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000196286 2024-04-20T01:48:36.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.260 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.260 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.734668+0000", 2024-04-20T01:48:36.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00073316799999999997 2024-04-20T01:48:36.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.260 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.260 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.260 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.741643+0000", 2024-04-20T01:48:36.260 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0069755149999999998 2024-04-20T01:48:36.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.261 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.743035+0000", 2024-04-20T01:48:36.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0013920270000000001 2024-04-20T01:48:36.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.261 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.261 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.261 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.743155+0000", 2024-04-20T01:48:36.261 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000119949 2024-04-20T01:48:36.261 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.261 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.261 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.262 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:1005 3.7s0 3:e950587e:::smithi16579500-49:head [setxattr _header (57) in=7b] snapc 38=[36,35,31] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:36.262 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:31.808253+0000", 2024-04-20T01:48:36.262 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.3754933630000004, 2024-04-20T01:48:36.262 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.93510864199999999, 2024-04-20T01:48:36.262 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.262 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.262 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.262 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.262 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.262 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1005 2024-04-20T01:48:36.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.262 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.262 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808253+0000", 2024-04-20T01:48:36.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.263 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808253+0000", 2024-04-20T01:48:36.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.263 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.263 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808252+0000", 2024-04-20T01:48:36.263 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:36.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.263 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.264 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.264 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808256+0000", 2024-04-20T01:48:36.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.8010000000000001e-06 2024-04-20T01:48:36.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.264 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.264 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808260+0000", 2024-04-20T01:48:36.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2869999999999998e-06 2024-04-20T01:48:36.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.264 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.264 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808264+0000", 2024-04-20T01:48:36.264 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.301e-06 2024-04-20T01:48:36.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.264 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808350+0000", 2024-04-20T01:48:36.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.5507000000000007e-05 2024-04-20T01:48:36.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-04-20T01:48:36.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808352+0000", 2024-04-20T01:48:36.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2730000000000001e-06 2024-04-20T01:48:36.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.265 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.265 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.736131+0000", 2024-04-20T01:48:36.265 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.92777927100000002 2024-04-20T01:48:36.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.265 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.736239+0000", 2024-04-20T01:48:36.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010754 2024-04-20T01:48:36.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.736798+0000", 2024-04-20T01:48:36.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00055902700000000003 2024-04-20T01:48:36.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.266 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.266 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.266 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.741646+0000", 2024-04-20T01:48:36.266 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048478569999999997 2024-04-20T01:48:36.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.267 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.743239+0000", 2024-04-20T01:48:36.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0015928439999999999 2024-04-20T01:48:36.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.267 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.267 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.743361+0000", 2024-04-20T01:48:36.267 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012247000000000001 2024-04-20T01:48:36.267 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.267 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.267 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.267 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.267 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:1006 3.7s0 3:e950587e:::smithi16579500-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:36.268 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:31.808310+0000", 2024-04-20T01:48:36.268 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.3754355770000002, 2024-04-20T01:48:36.268 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.93690937200000002, 2024-04-20T01:48:36.268 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.268 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:36.268 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.268 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.268 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.268 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 1006 2024-04-20T01:48:36.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.268 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.268 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.268 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.268 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808310+0000", 2024-04-20T01:48:36.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.269 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.269 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.269 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808310+0000", 2024-04-20T01:48:36.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.269 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.269 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.269 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808309+0000", 2024-04-20T01:48:36.269 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:36.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.269 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.269 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.269 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808315+0000", 2024-04-20T01:48:36.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8810000000000001e-06 2024-04-20T01:48:36.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808329+0000", 2024-04-20T01:48:36.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4e-05 2024-04-20T01:48:36.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.270 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808345+0000", 2024-04-20T01:48:36.270 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5311e-05 2024-04-20T01:48:36.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.270 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.270 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808384+0000", 2024-04-20T01:48:36.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9402e-05 2024-04-20T01:48:36.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-04-20T01:48:36.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:31.808386+0000", 2024-04-20T01:48:36.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8789999999999999e-06 2024-04-20T01:48:36.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.271 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.737178+0000", 2024-04-20T01:48:36.271 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.92879243300000003 2024-04-20T01:48:36.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.271 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.271 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:36.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.737293+0000", 2024-04-20T01:48:36.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000114374 2024-04-20T01:48:36.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.272 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.743790+0000", 2024-04-20T01:48:36.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0064973690000000002 2024-04-20T01:48:36.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.272 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.272 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.743924+0000", 2024-04-20T01:48:36.272 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000133934 2024-04-20T01:48:36.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.272 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.272 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.273 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.745220+0000", 2024-04-20T01:48:36.273 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0012957089999999999 2024-04-20T01:48:36.273 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.273 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.273 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.273 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.273 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.273 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:36.275 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-20T01:48:36.275 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:36.289+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:36.305 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:48:36.306 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:48:36.306 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:48:36.306 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:48:36.306 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:48:36.306 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:48:36.306 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:48:36.306 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:48:36.306 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:48:36.306 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:48:36.306 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:48:36.306 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:48:36.307 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:48:36.307 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:48:36.307 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:48:36.307 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.14 2024-04-20T01:48:36.307 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:48:36.309 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-20T01:48:36.310 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-20T01:48:36.375 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:36.389+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:36.455 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:36.455 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:36.455 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-04-20T01:48:36.455 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:36.467 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-20T01:48:36.476 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:36.491+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:36.578 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:36.592+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:36.652 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:36.652 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:36.652 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-04-20T01:48:36.652 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-04-20T01:48:36.652 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:36.664 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-20T01:48:36.705 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:36.719+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:36.779 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:36.793+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:36.853 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:36.853 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-04-20T01:48:36.853 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-04-20T01:48:36.853 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-04-20T01:48:36.853 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.853 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:215 3.4s0 3:28069c2b:::smithi16579500-39:head [copy-from ver 4] snapc 3=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:48:36.853 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.797216+0000", 2024-04-20T01:48:36.854 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.068815585000003, 2024-04-20T01:48:36.854 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.020029964000000001, 2024-04-20T01:48:36.854 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.854 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.854 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.854 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.854 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.854 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 215 2024-04-20T01:48:36.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.854 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.854 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.854 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.854 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797216+0000", 2024-04-20T01:48:36.855 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.855 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.855 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.855 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797216+0000", 2024-04-20T01:48:36.855 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.855 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.855 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.855 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797214+0000", 2024-04-20T01:48:36.855 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:36.855 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.855 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.856 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.856 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797223+0000", 2024-04-20T01:48:36.856 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9709999999999993e-06 2024-04-20T01:48:36.856 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.856 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.856 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.856 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797240+0000", 2024-04-20T01:48:36.856 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6872e-05 2024-04-20T01:48:36.856 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.856 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.856 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.856 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797249+0000", 2024-04-20T01:48:36.856 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1500000000000005e-06 2024-04-20T01:48:36.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.857 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.857 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.857 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797298+0000", 2024-04-20T01:48:36.857 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9715000000000001e-05 2024-04-20T01:48:36.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.857 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.857 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.857 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797408+0000", 2024-04-20T01:48:36.857 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000109883 2024-04-20T01:48:36.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.857 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.857 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.858 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.808715+0000", 2024-04-20T01:48:36.858 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011306145 2024-04-20T01:48:36.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.858 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.858 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.858 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.812376+0000", 2024-04-20T01:48:36.858 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0036618029999999999 2024-04-20T01:48:36.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.858 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.858 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.858 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.817198+0000", 2024-04-20T01:48:36.858 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0048218749999999998 2024-04-20T01:48:36.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.858 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.859 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.859 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.817246+0000", 2024-04-20T01:48:36.859 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7787000000000002e-05 2024-04-20T01:48:36.859 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.859 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.859 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.859 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.859 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:216 3.4s0 3:28069c2b:::smithi16579500-39:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-20T01:48:36.859 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:37.797261+0000", 2024-04-20T01:48:36.859 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 59.068770856999997, 2024-04-20T01:48:36.859 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012431373000000001, 2024-04-20T01:48:36.859 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.859 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:36.860 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.860 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.860 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.860 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 216 2024-04-20T01:48:36.860 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.860 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.860 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.860 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.860 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797261+0000", 2024-04-20T01:48:36.860 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.860 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.860 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.860 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.860 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797261+0000", 2024-04-20T01:48:36.860 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.861 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.861 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797260+0000", 2024-04-20T01:48:36.861 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:36.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.861 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.861 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797266+0000", 2024-04-20T01:48:36.861 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2140000000000001e-06 2024-04-20T01:48:36.861 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.861 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.861 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.861 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797275+0000", 2024-04-20T01:48:36.861 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.9309999999999997e-06 2024-04-20T01:48:36.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.862 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.862 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797282+0000", 2024-04-20T01:48:36.862 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.5169999999999996e-06 2024-04-20T01:48:36.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.862 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.862 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797635+0000", 2024-04-20T01:48:36.862 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00035326500000000001 2024-04-20T01:48:36.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.862 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.862 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:36.862 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.797689+0000", 2024-04-20T01:48:36.863 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3687999999999998e-05 2024-04-20T01:48:36.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.863 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.863 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.863 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.809488+0000", 2024-04-20T01:48:36.863 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011799417 2024-04-20T01:48:36.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.863 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.863 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.863 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.809591+0000", 2024-04-20T01:48:36.863 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000102729 2024-04-20T01:48:36.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.863 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.863 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.863 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:37.809692+0000", 2024-04-20T01:48:36.864 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000101202 2024-04-20T01:48:36.864 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.864 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.864 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.864 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.864 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:240 3.4s0 3:28069c2b:::smithi16579500-39:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:36.864 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787662+0000", 2024-04-20T01:48:36.864 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 58.078369444000003, 2024-04-20T01:48:36.864 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013455973, 2024-04-20T01:48:36.864 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.864 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.864 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.864 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.865 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.865 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 240 2024-04-20T01:48:36.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.865 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.865 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.865 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.865 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787662+0000", 2024-04-20T01:48:36.865 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.865 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.865 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.865 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787662+0000", 2024-04-20T01:48:36.865 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.866 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.866 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787659+0000", 2024-04-20T01:48:36.866 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999971 2024-04-20T01:48:36.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.866 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.866 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787676+0000", 2024-04-20T01:48:36.866 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6371999999999998e-05 2024-04-20T01:48:36.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.866 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.866 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.867 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787703+0000", 2024-04-20T01:48:36.867 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6902000000000001e-05 2024-04-20T01:48:36.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.867 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.867 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787718+0000", 2024-04-20T01:48:36.867 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5231999999999999e-05 2024-04-20T01:48:36.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.867 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.867 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.867 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788023+0000", 2024-04-20T01:48:36.867 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000304825 2024-04-20T01:48:36.868 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.868 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.868 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "op must wait for map", 2024-04-20T01:48:36.868 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788031+0000", 2024-04-20T01:48:36.868 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7560000000000005e-06 2024-04-20T01:48:36.868 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.868 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.868 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.868 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791096+0000", 2024-04-20T01:48:36.868 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0030642849999999999 2024-04-20T01:48:36.868 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.868 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.868 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.868 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.791195+0000", 2024-04-20T01:48:36.869 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9277000000000005e-05 2024-04-20T01:48:36.869 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.869 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.869 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.869 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.792028+0000", 2024-04-20T01:48:36.869 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00083305200000000003 2024-04-20T01:48:36.869 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.869 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.869 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.869 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.800720+0000", 2024-04-20T01:48:36.869 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086918859999999994 2024-04-20T01:48:36.869 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.869 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.870 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.870 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.801035+0000", 2024-04-20T01:48:36.870 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00031470199999999998 2024-04-20T01:48:36.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.870 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.870 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.870 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.801118+0000", 2024-04-20T01:48:36.870 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3541999999999994e-05 2024-04-20T01:48:36.870 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.870 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.870 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.870 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.870 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:241 3.4s0 3:28069c2b:::smithi16579500-39:head [rollback 2] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:36.871 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787738+0000", 2024-04-20T01:48:36.871 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 58.078293502000001, 2024-04-20T01:48:36.871 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01358739, 2024-04-20T01:48:36.871 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.871 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.871 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.871 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.871 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.871 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 241 2024-04-20T01:48:36.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.871 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.871 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.871 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.871 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787738+0000", 2024-04-20T01:48:36.871 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.872 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.872 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.872 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787738+0000", 2024-04-20T01:48:36.872 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.872 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.872 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.872 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787737+0000", 2024-04-20T01:48:36.872 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:36.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.872 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.872 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.872 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787742+0000", 2024-04-20T01:48:36.873 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.4050000000000002e-06 2024-04-20T01:48:36.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.873 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.873 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.873 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787751+0000", 2024-04-20T01:48:36.873 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3710000000000006e-06 2024-04-20T01:48:36.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.873 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.873 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.873 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787757+0000", 2024-04-20T01:48:36.873 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2600000000000002e-06 2024-04-20T01:48:36.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.873 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.874 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.874 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788094+0000", 2024-04-20T01:48:36.874 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033620200000000002 2024-04-20T01:48:36.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.874 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:36.874 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788099+0000", 2024-04-20T01:48:36.874 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2469999999999999e-06 2024-04-20T01:48:36.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.874 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.874 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.874 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.792538+0000", 2024-04-20T01:48:36.874 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0044393949999999996 2024-04-20T01:48:36.874 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.875 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.875 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.875 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.792627+0000", 2024-04-20T01:48:36.875 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.8646999999999993e-05 2024-04-20T01:48:36.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.875 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.875 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.875 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.793450+0000", 2024-04-20T01:48:36.875 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00082310900000000004 2024-04-20T01:48:36.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.875 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.875 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.875 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.800753+0000", 2024-04-20T01:48:36.875 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073032540000000003 2024-04-20T01:48:36.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.876 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.876 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.876 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.801274+0000", 2024-04-20T01:48:36.876 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00052095799999999997 2024-04-20T01:48:36.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.876 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.876 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.876 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.801325+0000", 2024-04-20T01:48:36.876 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1251000000000001e-05 2024-04-20T01:48:36.876 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.876 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.876 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.876 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.877 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:242 3.4s0 3:28069c2b:::smithi16579500-39:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:36.877 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:38.787763+0000", 2024-04-20T01:48:36.877 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 58.078268774000001, 2024-04-20T01:48:36.877 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013716078, 2024-04-20T01:48:36.877 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.877 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.877 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.877 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.877 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.877 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 242 2024-04-20T01:48:36.877 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.877 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.877 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.877 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.878 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787763+0000", 2024-04-20T01:48:36.878 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.878 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.878 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787763+0000", 2024-04-20T01:48:36.878 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.878 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.878 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787762+0000", 2024-04-20T01:48:36.878 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:36.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.878 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.879 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787765+0000", 2024-04-20T01:48:36.879 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.084e-06 2024-04-20T01:48:36.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.879 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787771+0000", 2024-04-20T01:48:36.879 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.9020000000000004e-06 2024-04-20T01:48:36.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.879 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.879 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.787781+0000", 2024-04-20T01:48:36.879 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.1630000000000002e-06 2024-04-20T01:48:36.879 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.879 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.880 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.880 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788118+0000", 2024-04-20T01:48:36.880 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00033771499999999999 2024-04-20T01:48:36.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.880 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.880 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:36.880 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.788123+0000", 2024-04-20T01:48:36.880 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9490000000000004e-06 2024-04-20T01:48:36.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.880 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.880 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.880 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.793864+0000", 2024-04-20T01:48:36.880 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0057404309999999998 2024-04-20T01:48:36.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.880 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.881 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.881 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.793978+0000", 2024-04-20T01:48:36.881 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000114274 2024-04-20T01:48:36.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.881 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.881 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.881 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.794417+0000", 2024-04-20T01:48:36.881 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00043935000000000003 2024-04-20T01:48:36.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.881 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.881 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.881 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.800777+0000", 2024-04-20T01:48:36.881 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0063594230000000003 2024-04-20T01:48:36.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.881 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.882 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.882 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.801420+0000", 2024-04-20T01:48:36.882 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00064355900000000004 2024-04-20T01:48:36.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.882 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.882 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:38.801479+0000", 2024-04-20T01:48:36.882 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.8693000000000002e-05 2024-04-20T01:48:36.882 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.882 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.882 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.882 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.882 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.883 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:292 3.4s0 3:27ac9303:::smithi16579500-19:head [read 0~1261568,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:36.883 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:43.807455+0000", 2024-04-20T01:48:36.883 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 53.058576148999997, 2024-04-20T01:48:36.883 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.01155932, 2024-04-20T01:48:36.883 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.883 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:36.883 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.883 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.883 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.883 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 292 2024-04-20T01:48:36.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.883 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.883 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.883 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.883 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807455+0000", 2024-04-20T01:48:36.884 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.884 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.884 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807455+0000", 2024-04-20T01:48:36.884 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.884 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.884 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807455+0000", 2024-04-20T01:48:36.884 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:36.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.884 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.884 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.884 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807460+0000", 2024-04-20T01:48:36.885 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7879999999999997e-06 2024-04-20T01:48:36.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.885 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.885 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807465+0000", 2024-04-20T01:48:36.885 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3260000000000003e-06 2024-04-20T01:48:36.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.885 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.885 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807469+0000", 2024-04-20T01:48:36.885 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.703e-06 2024-04-20T01:48:36.885 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.885 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.885 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.885 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807605+0000", 2024-04-20T01:48:36.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00013600600000000001 2024-04-20T01:48:36.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.886 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:36.886 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807612+0000", 2024-04-20T01:48:36.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3969999999999997e-06 2024-04-20T01:48:36.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.886 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.886 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.812370+0000", 2024-04-20T01:48:36.886 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0047582049999999997 2024-04-20T01:48:36.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.886 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.886 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.886 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.812466+0000", 2024-04-20T01:48:36.887 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5373999999999999e-05 2024-04-20T01:48:36.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.887 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.887 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.819015+0000", 2024-04-20T01:48:36.887 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.006549048 2024-04-20T01:48:36.887 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.887 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.887 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.887 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.887 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:293 3.4s0 3:27ac9303:::smithi16579500-19:head [read 0~1261568,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:36.887 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:43.807567+0000", 2024-04-20T01:48:36.887 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 53.058464311000002, 2024-04-20T01:48:36.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012280444999999999, 2024-04-20T01:48:36.888 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.888 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:36.888 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.888 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.888 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.888 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 293 2024-04-20T01:48:36.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.888 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.888 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.888 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.888 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807567+0000", 2024-04-20T01:48:36.888 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807567+0000", 2024-04-20T01:48:36.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807567+0000", 2024-04-20T01:48:36.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:36.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.889 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.889 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.889 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807573+0000", 2024-04-20T01:48:36.889 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6050000000000003e-06 2024-04-20T01:48:36.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807583+0000", 2024-04-20T01:48:36.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0274e-05 2024-04-20T01:48:36.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807591+0000", 2024-04-20T01:48:36.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.892e-06 2024-04-20T01:48:36.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.890 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.890 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.890 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807645+0000", 2024-04-20T01:48:36.890 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3202000000000003e-05 2024-04-20T01:48:36.890 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.891 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:36.891 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.807651+0000", 2024-04-20T01:48:36.891 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6699999999999997e-06 2024-04-20T01:48:36.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.891 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.891 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.812731+0000", 2024-04-20T01:48:36.891 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0050797480000000002 2024-04-20T01:48:36.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.891 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.891 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.891 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.812817+0000", 2024-04-20T01:48:36.891 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6415999999999994e-05 2024-04-20T01:48:36.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.892 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.892 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:43.819848+0000", 2024-04-20T01:48:36.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0070303809999999996 2024-04-20T01:48:36.892 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.892 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.892 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.892 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.892 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:399 3.4s0 3:2d59a0da:::smithi16579500-10:head [append 0~0] snapc d=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-04-20T01:48:36.892 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.711018+0000", 2024-04-20T01:48:36.892 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.155013509, 2024-04-20T01:48:36.892 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.014803088000000001, 2024-04-20T01:48:36.892 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.893 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.893 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.893 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.893 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.893 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 399 2024-04-20T01:48:36.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.893 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.893 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.893 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711018+0000", 2024-04-20T01:48:36.893 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.893 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.893 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.893 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711018+0000", 2024-04-20T01:48:36.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.894 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.894 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711017+0000", 2024-04-20T01:48:36.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:36.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.894 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.894 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711023+0000", 2024-04-20T01:48:36.894 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7760000000000003e-06 2024-04-20T01:48:36.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.894 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.895 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.895 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711036+0000", 2024-04-20T01:48:36.895 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3754000000000001e-05 2024-04-20T01:48:36.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.895 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.895 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.895 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711044+0000", 2024-04-20T01:48:36.895 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.435e-06 2024-04-20T01:48:36.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.895 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.895 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.895 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711095+0000", 2024-04-20T01:48:36.895 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.1038999999999997e-05 2024-04-20T01:48:36.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711221+0000", 2024-04-20T01:48:36.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000126515 2024-04-20T01:48:36.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711890+0000", 2024-04-20T01:48:36.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00066828000000000005 2024-04-20T01:48:36.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.896 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.896 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.896 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.714226+0000", 2024-04-20T01:48:36.896 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023359040000000002 2024-04-20T01:48:36.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.897 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.897 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.725772+0000", 2024-04-20T01:48:36.897 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011546529 2024-04-20T01:48:36.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.897 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.897 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.897 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.725821+0000", 2024-04-20T01:48:36.897 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9129999999999999e-05 2024-04-20T01:48:36.897 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.897 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.897 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.898 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:400 3.4s0 3:2d59a0da:::smithi16579500-10:head [rollback b] snapc d=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-04-20T01:48:36.898 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.711049+0000", 2024-04-20T01:48:36.898 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.154982885999999, 2024-04-20T01:48:36.898 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.019555619, 2024-04-20T01:48:36.898 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.898 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.898 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.898 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.898 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.898 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 400 2024-04-20T01:48:36.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.898 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.898 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.898 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.899 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711049+0000", 2024-04-20T01:48:36.899 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.899 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.899 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.899 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711049+0000", 2024-04-20T01:48:36.899 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.899 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.899 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.899 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711048+0000", 2024-04-20T01:48:36.899 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:36.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.899 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.900 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711050+0000", 2024-04-20T01:48:36.900 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.905e-06 2024-04-20T01:48:36.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.900 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711054+0000", 2024-04-20T01:48:36.900 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0280000000000002e-06 2024-04-20T01:48:36.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.900 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.900 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711058+0000", 2024-04-20T01:48:36.900 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5159999999999999e-06 2024-04-20T01:48:36.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.901 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.901 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712434+0000", 2024-04-20T01:48:36.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001375937 2024-04-20T01:48:36.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.901 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.901 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.712543+0000", 2024-04-20T01:48:36.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010908000000000001 2024-04-20T01:48:36.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.901 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.901 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.901 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713115+0000", 2024-04-20T01:48:36.901 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00057183500000000003 2024-04-20T01:48:36.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.902 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.902 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.715444+0000", 2024-04-20T01:48:36.902 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0023292930000000001 2024-04-20T01:48:36.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.902 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.902 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730509+0000", 2024-04-20T01:48:36.902 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.015064546 2024-04-20T01:48:36.902 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.902 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.902 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.902 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730604+0000", 2024-04-20T01:48:36.902 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5814000000000001e-05 2024-04-20T01:48:36.902 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.903 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.903 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.903 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.903 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:401 3.4s0 3:2d59a0da:::smithi16579500-10:head [append 0~0] snapc d=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-04-20T01:48:36.903 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:49.711061+0000", 2024-04-20T01:48:36.903 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 47.154970532999997, 2024-04-20T01:48:36.903 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.019877855999999999, 2024-04-20T01:48:36.903 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.903 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.903 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.903 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.903 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.903 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 401 2024-04-20T01:48:36.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.904 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.904 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.904 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.904 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711061+0000", 2024-04-20T01:48:36.904 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.904 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.904 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.904 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711061+0000", 2024-04-20T01:48:36.904 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.904 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.904 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.904 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711061+0000", 2024-04-20T01:48:36.905 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:36.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.905 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711064+0000", 2024-04-20T01:48:36.905 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8480000000000001e-06 2024-04-20T01:48:36.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.905 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711067+0000", 2024-04-20T01:48:36.905 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5559999999999999e-06 2024-04-20T01:48:36.905 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.905 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.905 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.905 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.711070+0000", 2024-04-20T01:48:36.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.044e-06 2024-04-20T01:48:36.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.906 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.906 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.906 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713427+0000", 2024-04-20T01:48:36.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.002356585 2024-04-20T01:48:36.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.906 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.906 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.906 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713534+0000", 2024-04-20T01:48:36.906 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00010668200000000001 2024-04-20T01:48:36.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.906 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.906 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.713981+0000", 2024-04-20T01:48:36.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00044727999999999999 2024-04-20T01:48:36.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.718977+0000", 2024-04-20T01:48:36.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0049960330000000004 2024-04-20T01:48:36.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.907 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730869+0000", 2024-04-20T01:48:36.907 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011892606 2024-04-20T01:48:36.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.907 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.907 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.908 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:49.730939+0000", 2024-04-20T01:48:36.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.9470999999999999e-05 2024-04-20T01:48:36.908 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.908 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.908 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.908 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.908 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:479 3.4s0 3:257d96c1:::smithi16579500-15:19 [read 0~245760,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:36.908 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.727997+0000", 2024-04-20T01:48:36.908 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.138034181999998, 2024-04-20T01:48:36.908 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.066708136000000001, 2024-04-20T01:48:36.908 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.908 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:36.908 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.909 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.909 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.909 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 479 2024-04-20T01:48:36.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.909 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.909 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.909 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.727997+0000", 2024-04-20T01:48:36.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.909 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.909 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.909 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.727997+0000", 2024-04-20T01:48:36.909 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.727995+0000", 2024-04-20T01:48:36.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:36.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728006+0000", 2024-04-20T01:48:36.910 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0479000000000001e-05 2024-04-20T01:48:36.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.910 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.910 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.910 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728024+0000", 2024-04-20T01:48:36.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7819999999999999e-05 2024-04-20T01:48:36.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.911 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.911 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728037+0000", 2024-04-20T01:48:36.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3691e-05 2024-04-20T01:48:36.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.911 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.911 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.730187+0000", 2024-04-20T01:48:36.911 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021495189999999999 2024-04-20T01:48:36.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.911 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.911 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-04-20T01:48:36.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.730193+0000", 2024-04-20T01:48:36.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4060000000000002e-06 2024-04-20T01:48:36.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.784252+0000", 2024-04-20T01:48:36.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.054058782999999999 2024-04-20T01:48:36.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-04-20T01:48:36.912 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.784915+0000", 2024-04-20T01:48:36.912 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00066283800000000001 2024-04-20T01:48:36.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.912 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.912 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.791659+0000", 2024-04-20T01:48:36.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0067437089999999996 2024-04-20T01:48:36.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.791831+0000", 2024-04-20T01:48:36.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000172473 2024-04-20T01:48:36.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.913 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.913 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.913 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.794706+0000", 2024-04-20T01:48:36.913 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0028743559999999998 2024-04-20T01:48:36.913 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.913 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.913 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.914 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.914 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:480 3.4s0 3:257d96c1:::smithi16579500-15:19 [read 0~245760,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:36.914 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.728048+0000", 2024-04-20T01:48:36.914 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.137983855999998, 2024-04-20T01:48:36.914 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.068695271000000002, 2024-04-20T01:48:36.914 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.914 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:36.914 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.914 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.914 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.914 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 480 2024-04-20T01:48:36.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.914 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728048+0000", 2024-04-20T01:48:36.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728048+0000", 2024-04-20T01:48:36.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.915 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.915 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.915 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728047+0000", 2024-04-20T01:48:36.915 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:36.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728053+0000", 2024-04-20T01:48:36.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6529999999999999e-06 2024-04-20T01:48:36.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728061+0000", 2024-04-20T01:48:36.916 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.867e-06 2024-04-20T01:48:36.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.916 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.916 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.916 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728068+0000", 2024-04-20T01:48:36.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.596e-06 2024-04-20T01:48:36.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.730233+0000", 2024-04-20T01:48:36.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021647680000000001 2024-04-20T01:48:36.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-04-20T01:48:36.917 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.730235+0000", 2024-04-20T01:48:36.917 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.694e-06 2024-04-20T01:48:36.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.917 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.917 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.785790+0000", 2024-04-20T01:48:36.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.055555805 2024-04-20T01:48:36.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-04-20T01:48:36.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.785867+0000", 2024-04-20T01:48:36.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.6564999999999996e-05 2024-04-20T01:48:36.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.918 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.918 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.794785+0000", 2024-04-20T01:48:36.918 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0089179169999999992 2024-04-20T01:48:36.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.794908+0000", 2024-04-20T01:48:36.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012334100000000001 2024-04-20T01:48:36.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.919 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.919 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.796743+0000", 2024-04-20T01:48:36.919 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0018348559999999999 2024-04-20T01:48:36.919 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.919 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.919 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.919 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.919 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:481 3.4s0 3:257d96c1:::smithi16579500-15:19 [read 0~245760,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:36.919 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.728074+0000", 2024-04-20T01:48:36.920 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.137957317000001, 2024-04-20T01:48:36.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.070786587999999998, 2024-04-20T01:48:36.920 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.920 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:36.920 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.920 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.920 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.920 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 481 2024-04-20T01:48:36.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.920 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.920 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.920 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728074+0000", 2024-04-20T01:48:36.920 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.920 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728074+0000", 2024-04-20T01:48:36.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728074+0000", 2024-04-20T01:48:36.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:36.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.921 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.921 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.921 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728077+0000", 2024-04-20T01:48:36.921 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4630000000000001e-06 2024-04-20T01:48:36.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728082+0000", 2024-04-20T01:48:36.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9280000000000001e-06 2024-04-20T01:48:36.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.728085+0000", 2024-04-20T01:48:36.922 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4809999999999998e-06 2024-04-20T01:48:36.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.922 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.922 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.922 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.730277+0000", 2024-04-20T01:48:36.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0021910639999999999 2024-04-20T01:48:36.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-04-20T01:48:36.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.730277+0000", 2024-04-20T01:48:36.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.99e-07 2024-04-20T01:48:36.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.786625+0000", 2024-04-20T01:48:36.923 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.056347795999999999 2024-04-20T01:48:36.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.923 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.923 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-04-20T01:48:36.923 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.786743+0000", 2024-04-20T01:48:36.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00011777600000000001 2024-04-20T01:48:36.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.796834+0000", 2024-04-20T01:48:36.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010091086000000001 2024-04-20T01:48:36.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.924 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.796945+0000", 2024-04-20T01:48:36.924 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000110646 2024-04-20T01:48:36.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.924 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.924 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.925 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.798861+0000", 2024-04-20T01:48:36.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0019162999999999999 2024-04-20T01:48:36.925 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.925 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.925 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.925 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.925 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:482 3.4s0 3:27ac9303:::smithi16579500-19:head [append 0~802816] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:36.925 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.753140+0000", 2024-04-20T01:48:36.925 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.112891175999998, 2024-04-20T01:48:36.925 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.064361493000000006, 2024-04-20T01:48:36.925 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.925 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.925 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.925 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.925 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.926 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 482 2024-04-20T01:48:36.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.926 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.753140+0000", 2024-04-20T01:48:36.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.926 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.926 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.753140+0000", 2024-04-20T01:48:36.926 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.926 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.753137+0000", 2024-04-20T01:48:36.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999967 2024-04-20T01:48:36.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.754063+0000", 2024-04-20T01:48:36.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00092535499999999999 2024-04-20T01:48:36.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.927 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.927 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.927 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.754087+0000", 2024-04-20T01:48:36.927 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4181000000000001e-05 2024-04-20T01:48:36.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.754099+0000", 2024-04-20T01:48:36.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2594e-05 2024-04-20T01:48:36.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.754174+0000", 2024-04-20T01:48:36.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4932999999999996e-05 2024-04-20T01:48:36.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.928 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.928 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-04-20T01:48:36.928 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.754180+0000", 2024-04-20T01:48:36.928 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.2820000000000004e-06 2024-04-20T01:48:36.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.929 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.929 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.786868+0000", 2024-04-20T01:48:36.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.032688506999999999 2024-04-20T01:48:36.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.929 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-04-20T01:48:36.929 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.787722+0000", 2024-04-20T01:48:36.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00085391199999999997 2024-04-20T01:48:36.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.929 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.929 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.811320+0000", 2024-04-20T01:48:36.929 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.023598219 2024-04-20T01:48:36.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.811471+0000", 2024-04-20T01:48:36.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015034900000000001 2024-04-20T01:48:36.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.812850+0000", 2024-04-20T01:48:36.930 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0013796290000000001 2024-04-20T01:48:36.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.930 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.930 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.930 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.816794+0000", 2024-04-20T01:48:36.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0039441110000000001 2024-04-20T01:48:36.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.817128+0000", 2024-04-20T01:48:36.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000333543 2024-04-20T01:48:36.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.931 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.931 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.931 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.817502+0000", 2024-04-20T01:48:36.931 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00037399800000000001 2024-04-20T01:48:36.931 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.931 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.931 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.932 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.932 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:483 3.4s0 3:27ac9303:::smithi16579500-19:head [append 0~638976] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:36.932 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.756616+0000", 2024-04-20T01:48:36.932 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.109415231, 2024-04-20T01:48:36.932 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.064169961999999997, 2024-04-20T01:48:36.932 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.932 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.932 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.932 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.932 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.932 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 483 2024-04-20T01:48:36.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.932 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.933 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.933 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.756616+0000", 2024-04-20T01:48:36.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.933 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.933 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.756616+0000", 2024-04-20T01:48:36.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.933 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.933 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.933 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.756615+0000", 2024-04-20T01:48:36.933 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:36.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.934 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757340+0000", 2024-04-20T01:48:36.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00072406300000000005 2024-04-20T01:48:36.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.934 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757351+0000", 2024-04-20T01:48:36.934 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1399e-05 2024-04-20T01:48:36.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.934 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.934 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.934 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757357+0000", 2024-04-20T01:48:36.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2770000000000002e-06 2024-04-20T01:48:36.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757432+0000", 2024-04-20T01:48:36.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.4639000000000001e-05 2024-04-20T01:48:36.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-04-20T01:48:36.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757434+0000", 2024-04-20T01:48:36.935 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.3350000000000001e-06 2024-04-20T01:48:36.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.935 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.935 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.935 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.787877+0000", 2024-04-20T01:48:36.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.030443274999999999 2024-04-20T01:48:36.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-04-20T01:48:36.936 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.788002+0000", 2024-04-20T01:48:36.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012432900000000001 2024-04-20T01:48:36.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.936 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.814493+0000", 2024-04-20T01:48:36.936 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.026491297 2024-04-20T01:48:36.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.936 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.936 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.937 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.814665+0000", 2024-04-20T01:48:36.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017212799999999999 2024-04-20T01:48:36.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.937 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.937 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.815607+0000", 2024-04-20T01:48:36.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000942212 2024-04-20T01:48:36.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.937 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.937 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.817608+0000", 2024-04-20T01:48:36.937 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0020010269999999998 2024-04-20T01:48:36.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.937 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.937 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.937 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.820465+0000", 2024-04-20T01:48:36.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0028566260000000001 2024-04-20T01:48:36.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.938 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.938 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.820786+0000", 2024-04-20T01:48:36.938 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00032123299999999998 2024-04-20T01:48:36.938 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.938 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.938 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.938 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.938 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:484 3.4s0 3:27ac9303:::smithi16579500-19:head [append 0~131072] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:36.938 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.757369+0000", 2024-04-20T01:48:36.938 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.108662774999999, 2024-04-20T01:48:36.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.066086373000000004, 2024-04-20T01:48:36.939 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.939 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.939 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.939 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.939 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.939 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 484 2024-04-20T01:48:36.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.939 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.939 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.939 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757369+0000", 2024-04-20T01:48:36.939 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.939 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757369+0000", 2024-04-20T01:48:36.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757368+0000", 2024-04-20T01:48:36.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:36.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.940 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.940 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757524+0000", 2024-04-20T01:48:36.940 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000155892 2024-04-20T01:48:36.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.940 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.941 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.941 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757533+0000", 2024-04-20T01:48:36.941 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7739999999999993e-06 2024-04-20T01:48:36.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.941 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.941 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757538+0000", 2024-04-20T01:48:36.941 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.9099999999999996e-06 2024-04-20T01:48:36.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.941 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.941 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.941 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757586+0000", 2024-04-20T01:48:36.941 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.7988999999999997e-05 2024-04-20T01:48:36.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.942 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-04-20T01:48:36.942 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757588+0000", 2024-04-20T01:48:36.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.922e-06 2024-04-20T01:48:36.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.942 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.942 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.788061+0000", 2024-04-20T01:48:36.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.030473666999999999 2024-04-20T01:48:36.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.942 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.942 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-04-20T01:48:36.942 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.788190+0000", 2024-04-20T01:48:36.942 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000128415 2024-04-20T01:48:36.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.817711+0000", 2024-04-20T01:48:36.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.029521546999999999 2024-04-20T01:48:36.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.817811+0000", 2024-04-20T01:48:36.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.9242999999999998e-05 2024-04-20T01:48:36.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.943 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.943 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.943 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.818379+0000", 2024-04-20T01:48:36.943 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000568039 2024-04-20T01:48:36.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.823163+0000", 2024-04-20T01:48:36.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004784122 2024-04-20T01:48:36.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.823230+0000", 2024-04-20T01:48:36.944 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7317000000000006e-05 2024-04-20T01:48:36.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.944 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.944 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.944 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.823455+0000", 2024-04-20T01:48:36.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000225076 2024-04-20T01:48:36.945 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.945 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.945 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.945 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.945 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:485 3.4s0 3:27ac9303:::smithi16579500-19:head [setxattr _header (57) in=7b] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:36.945 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.757544+0000", 2024-04-20T01:48:36.945 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.108487668000002, 2024-04-20T01:48:36.945 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.068448555999999994, 2024-04-20T01:48:36.945 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.945 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:36.945 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.945 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.945 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.945 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 485 2024-04-20T01:48:36.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.946 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757544+0000", 2024-04-20T01:48:36.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.946 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757544+0000", 2024-04-20T01:48:36.946 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.946 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.946 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757544+0000", 2024-04-20T01:48:36.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:36.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757547+0000", 2024-04-20T01:48:36.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.089e-06 2024-04-20T01:48:36.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.947 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.947 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757556+0000", 2024-04-20T01:48:36.947 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7119999999999995e-06 2024-04-20T01:48:36.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.947 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757560+0000", 2024-04-20T01:48:36.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3830000000000001e-06 2024-04-20T01:48:36.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757618+0000", 2024-04-20T01:48:36.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.838e-05 2024-04-20T01:48:36.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.948 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-04-20T01:48:36.948 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757619+0000", 2024-04-20T01:48:36.948 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.6899999999999996e-07 2024-04-20T01:48:36.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.948 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.788519+0000", 2024-04-20T01:48:36.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.030900542999999999 2024-04-20T01:48:36.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-04-20T01:48:36.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.788645+0000", 2024-04-20T01:48:36.949 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00012514399999999999 2024-04-20T01:48:36.949 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.949 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.949 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.819052+0000", 2024-04-20T01:48:36.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.030407828000000001 2024-04-20T01:48:36.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.950 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.819150+0000", 2024-04-20T01:48:36.950 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.7280000000000004e-05 2024-04-20T01:48:36.950 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.950 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.950 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.819689+0000", 2024-04-20T01:48:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00053913300000000002 2024-04-20T01:48:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.824654+0000", 2024-04-20T01:48:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0049648260000000003 2024-04-20T01:48:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "commit_sent", 2024-04-20T01:48:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.825598+0000", 2024-04-20T01:48:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00094450800000000004 2024-04-20T01:48:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.951 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.952 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.952 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.825992+0000", 2024-04-20T01:48:36.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00039428400000000001 2024-04-20T01:48:36.952 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.952 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.952 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.952 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.952 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "osd_op(client.4643.0:486 3.4s0 3:27ac9303:::smithi16579500-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:36.952 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.757577+0000", 2024-04-20T01:48:36.952 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.108454983000001, 2024-04-20T01:48:36.952 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.073599585999999995, 2024-04-20T01:48:36.952 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.952 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "started", 2024-04-20T01:48:36.952 INFO:teuthology.orchestra.run.smithi044.stdout: "client_info": { 2024-04-20T01:48:36.953 INFO:teuthology.orchestra.run.smithi044.stdout: "client": "client.4643", 2024-04-20T01:48:36.953 INFO:teuthology.orchestra.run.smithi044.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:36.953 INFO:teuthology.orchestra.run.smithi044.stdout: "tid": 486 2024-04-20T01:48:36.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.953 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757577+0000", 2024-04-20T01:48:36.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.953 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.953 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.953 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757577+0000", 2024-04-20T01:48:36.953 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757576+0000", 2024-04-20T01:48:36.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:36.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757578+0000", 2024-04-20T01:48:36.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.075e-06 2024-04-20T01:48:36.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.954 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.954 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.954 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757583+0000", 2024-04-20T01:48:36.954 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.51e-06 2024-04-20T01:48:36.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757586+0000", 2024-04-20T01:48:36.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.4580000000000002e-06 2024-04-20T01:48:36.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757640+0000", 2024-04-20T01:48:36.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3338999999999999e-05 2024-04-20T01:48:36.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.955 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.955 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for peered", 2024-04-20T01:48:36.955 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.757640+0000", 2024-04-20T01:48:36.955 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.61e-07 2024-04-20T01:48:36.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.788692+0000", 2024-04-20T01:48:36.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.031051856999999999 2024-04-20T01:48:36.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for missing object", 2024-04-20T01:48:36.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.788773+0000", 2024-04-20T01:48:36.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.0569999999999996e-05 2024-04-20T01:48:36.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.956 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.956 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.956 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.819959+0000", 2024-04-20T01:48:36.956 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.031186445 2024-04-20T01:48:36.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:36.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.820117+0000", 2024-04-20T01:48:36.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015805999999999999 2024-04-20T01:48:36.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.826421+0000", 2024-04-20T01:48:36.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0063032319999999998 2024-04-20T01:48:36.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.957 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "started", 2024-04-20T01:48:36.957 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.826576+0000", 2024-04-20T01:48:36.957 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00015581500000000001 2024-04-20T01:48:36.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.958 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.958 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.831176+0000", 2024-04-20T01:48:36.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.004599812 2024-04-20T01:48:36.958 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.958 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.958 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.958 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.958 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.958 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDPGRecoveryDeleteReply(3.4s0 e58,57 3:22cffa5d:::smithi16579500-2:7,41'41)", 2024-04-20T01:48:36.958 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.810383+0000", 2024-04-20T01:48:36.958 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.055648646000002, 2024-04-20T01:48:36.958 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011240485, 2024-04-20T01:48:36.959 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.959 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:36.959 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810383+0000", 2024-04-20T01:48:36.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.959 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810383+0000", 2024-04-20T01:48:36.959 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.959 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.959 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810382+0000", 2024-04-20T01:48:36.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:36.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810389+0000", 2024-04-20T01:48:36.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8950000000000001e-06 2024-04-20T01:48:36.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.960 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810403+0000", 2024-04-20T01:48:36.960 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4149e-05 2024-04-20T01:48:36.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.960 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.960 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810409+0000", 2024-04-20T01:48:36.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.1310000000000003e-06 2024-04-20T01:48:36.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.821321+0000", 2024-04-20T01:48:36.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010911666 2024-04-20T01:48:36.961 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.961 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.961 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.961 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.821623+0000", 2024-04-20T01:48:36.961 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00030231000000000002 2024-04-20T01:48:36.961 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.961 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.961 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.962 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDPGRecoveryDeleteReply(3.4s0 e58,57 3:22cffa5d:::smithi16579500-2:7,41'41)", 2024-04-20T01:48:36.962 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:59.810504+0000", 2024-04-20T01:48:36.962 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 37.055527929999997, 2024-04-20T01:48:36.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.012539292, 2024-04-20T01:48:36.962 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:36.962 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:36.962 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:36.962 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.962 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:36.962 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810504+0000", 2024-04-20T01:48:36.962 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:36.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810504+0000", 2024-04-20T01:48:36.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:36.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:36.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810503+0000", 2024-04-20T01:48:36.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:36.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.963 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:36.963 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810508+0000", 2024-04-20T01:48:36.963 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3460000000000004e-06 2024-04-20T01:48:36.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.963 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:36.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810515+0000", 2024-04-20T01:48:36.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.3320000000000003e-06 2024-04-20T01:48:36.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:36.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.810521+0000", 2024-04-20T01:48:36.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.083e-06 2024-04-20T01:48:36.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.964 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:36.964 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.822436+0000", 2024-04-20T01:48:36.964 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.011915179 2024-04-20T01:48:36.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:36.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:36.965 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:36.965 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:59.823043+0000", 2024-04-20T01:48:36.965 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00060685600000000002 2024-04-20T01:48:36.965 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.965 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.965 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.965 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:36.965 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:36.965 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:36.967 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-20T01:48:36.967 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:36.981+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:37.044 INFO:teuthology.orchestra.run.smithi123.stdout:get_pg_num_history pg_num_history pg_num_history(e88 pg_nums {1={22=1},2={25=8},3={30=16}} deleted_pools ) 2024-04-20T01:48:37.044 INFO:teuthology.orchestra.run.smithi123.stdout:Importing pgid 3.5s3 2024-04-20T01:48:37.045 INFO:teuthology.orchestra.run.smithi123.stdout:Write 3#3:a2e9ac6b:::smithi16579500-4:head# 2024-04-20T01:48:37.045 INFO:teuthology.orchestra.run.smithi123.stdout:snapset 2d=[]:{} 2024-04-20T01:48:37.046 INFO:teuthology.orchestra.run.smithi123.stdout:Write 3#3:ae9f92b8:::smithi16579500-22:28# 2024-04-20T01:48:37.047 INFO:teuthology.orchestra.run.smithi123.stdout:Write 3#3:ae9f92b8:::smithi16579500-22:head# 2024-04-20T01:48:37.047 INFO:teuthology.orchestra.run.smithi123.stdout:snapset 28=[]:{28=[28]} 2024-04-20T01:48:37.048 INFO:teuthology.orchestra.run.smithi123.stdout:write_pg epoch 87 info 3.5s3( v 77'33 (0'0,77'33] local-lis/les=30/31 n=3 ec=30/30 lis/c=30/30 les/c/f=31/31/0 sis=30) 2024-04-20T01:48:37.049 INFO:teuthology.orchestra.run.smithi123.stdout:Import successful 2024-04-20T01:48:37.067 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:37.082+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:37.162 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:37.162 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:37.162 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-04-20T01:48:37.162 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:37.168 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:37.182+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:37.173 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-20T01:48:37.269 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:37.283+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:37.317 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:37.317 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:37.317 INFO:teuthology.orchestra.run.smithi044.stdout: "complaint_time": 30, 2024-04-20T01:48:37.317 INFO:teuthology.orchestra.run.smithi044.stdout: "num_blocked_ops": 0 2024-04-20T01:48:37.317 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:37.329 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-20T01:48:37.370 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:37.385+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:37.472 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:37.486+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:37.474 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:37.475 INFO:teuthology.orchestra.run.smithi044.stdout: "size": 20, 2024-04-20T01:48:37.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 600, 2024-04-20T01:48:37.475 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [ 2024-04-20T01:48:37.475 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.475 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.5s4 33/30 ECSubWrite(tid=2, reqid=client.4643.0:11, at_version=0'0, trim_to=0'0, roll_forward_to=32'2))", 2024-04-20T01:48:37.475 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.765877+0000", 2024-04-20T01:48:37.475 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 62.722467876000003, 2024-04-20T01:48:37.475 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0075032049999999998, 2024-04-20T01:48:37.475 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:37.475 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.476 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:37.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.476 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:37.476 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765877+0000", 2024-04-20T01:48:37.476 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.476 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:37.476 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765877+0000", 2024-04-20T01:48:37.476 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.476 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.476 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:37.477 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765877+0000", 2024-04-20T01:48:37.477 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:37.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.477 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.477 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:37.477 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765904+0000", 2024-04-20T01:48:37.477 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7461e-05 2024-04-20T01:48:37.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.477 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.477 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:37.477 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765927+0000", 2024-04-20T01:48:37.477 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.2697999999999999e-05 2024-04-20T01:48:37.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.477 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.477 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.478 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.765939+0000", 2024-04-20T01:48:37.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2142e-05 2024-04-20T01:48:37.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.478 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.478 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:37.478 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.768205+0000", 2024-04-20T01:48:37.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0022662239999999998 2024-04-20T01:48:37.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.478 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.478 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.478 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.768221+0000", 2024-04-20T01:48:37.478 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6161e-05 2024-04-20T01:48:37.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.478 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.479 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.479 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.773346+0000", 2024-04-20T01:48:37.479 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0051250469999999998 2024-04-20T01:48:37.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.479 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:37.479 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.773380+0000", 2024-04-20T01:48:37.479 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3760999999999997e-05 2024-04-20T01:48:37.479 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.479 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.479 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.479 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.479 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/30 ECSubWrite(tid=4, reqid=client.4643.0:58, at_version=33'3, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:48:37.479 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.780211+0000", 2024-04-20T01:48:37.480 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 62.708134287, 2024-04-20T01:48:37.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0098815680000000003, 2024-04-20T01:48:37.480 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:37.480 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.480 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:37.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:37.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780211+0000", 2024-04-20T01:48:37.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.480 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:37.480 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780211+0000", 2024-04-20T01:48:37.480 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.481 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:37.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780209+0000", 2024-04-20T01:48:37.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:37.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.481 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:37.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780307+0000", 2024-04-20T01:48:37.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.8575999999999999e-05 2024-04-20T01:48:37.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.481 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.481 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:37.481 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780328+0000", 2024-04-20T01:48:37.481 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0537999999999999e-05 2024-04-20T01:48:37.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780340+0000", 2024-04-20T01:48:37.482 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1997999999999999e-05 2024-04-20T01:48:37.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:37.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780423+0000", 2024-04-20T01:48:37.482 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2812999999999995e-05 2024-04-20T01:48:37.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.482 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.482 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.482 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.780434+0000", 2024-04-20T01:48:37.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1028e-05 2024-04-20T01:48:37.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.483 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790048+0000", 2024-04-20T01:48:37.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0096147339999999998 2024-04-20T01:48:37.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.483 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:37.483 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790092+0000", 2024-04-20T01:48:37.483 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.3649999999999997e-05 2024-04-20T01:48:37.483 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.483 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.483 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.483 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.484 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/30 ECSubWrite(tid=5, reqid=client.4643.0:59, at_version=33'4, trim_to=0'0, roll_forward_to=33'2))", 2024-04-20T01:48:37.484 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.781201+0000", 2024-04-20T01:48:37.484 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 62.707144049, 2024-04-20T01:48:37.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0089164740000000006, 2024-04-20T01:48:37.484 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:37.484 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.484 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:37.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.484 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:37.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781201+0000", 2024-04-20T01:48:37.484 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.484 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:37.484 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781201+0000", 2024-04-20T01:48:37.485 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.485 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:37.485 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781200+0000", 2024-04-20T01:48:37.485 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:37.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.485 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:37.485 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781219+0000", 2024-04-20T01:48:37.485 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8748000000000001e-05 2024-04-20T01:48:37.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.485 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.485 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:37.485 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781236+0000", 2024-04-20T01:48:37.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7251000000000001e-05 2024-04-20T01:48:37.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.486 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.486 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.781246+0000", 2024-04-20T01:48:37.486 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0026e-05 2024-04-20T01:48:37.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.486 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.486 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:37.486 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.782742+0000", 2024-04-20T01:48:37.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.001496079 2024-04-20T01:48:37.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.782753+0000", 2024-04-20T01:48:37.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0767999999999999e-05 2024-04-20T01:48:37.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790103+0000", 2024-04-20T01:48:37.487 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0073499259999999997 2024-04-20T01:48:37.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.487 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.487 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:37.487 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790117+0000", 2024-04-20T01:48:37.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4729000000000001e-05 2024-04-20T01:48:37.488 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.488 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.488 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.488 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.5s4 33/30 ECSubWrite(tid=5, reqid=client.4643.0:65, at_version=33'4, trim_to=0'0, roll_forward_to=32'2))", 2024-04-20T01:48:37.488 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.783666+0000", 2024-04-20T01:48:37.488 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 62.704679028000001, 2024-04-20T01:48:37.488 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0063804899999999999, 2024-04-20T01:48:37.488 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:37.488 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.488 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:37.488 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:37.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783666+0000", 2024-04-20T01:48:37.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:37.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783666+0000", 2024-04-20T01:48:37.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.489 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:37.489 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783665+0000", 2024-04-20T01:48:37.489 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:37.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.489 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:37.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783685+0000", 2024-04-20T01:48:37.490 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9735e-05 2024-04-20T01:48:37.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.490 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:37.490 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783701+0000", 2024-04-20T01:48:37.490 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6175e-05 2024-04-20T01:48:37.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.490 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.491 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.491 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783710+0000", 2024-04-20T01:48:37.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.3649999999999993e-06 2024-04-20T01:48:37.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.491 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.491 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:37.491 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783800+0000", 2024-04-20T01:48:37.491 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0061000000000004e-05 2024-04-20T01:48:37.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.492 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.492 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.783809+0000", 2024-04-20T01:48:37.492 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.0750000000000004e-06 2024-04-20T01:48:37.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.492 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.492 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790003+0000", 2024-04-20T01:48:37.492 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0061941590000000003 2024-04-20T01:48:37.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.492 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.492 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:37.492 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.790046+0000", 2024-04-20T01:48:37.492 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2831999999999999e-05 2024-04-20T01:48:37.493 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.493 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.493 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.493 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs5 33/30 ECSubWrite(tid=0, reqid=client.4643.0:106, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:48:37.493 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.825844+0000", 2024-04-20T01:48:37.493 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 62.662501065999997, 2024-04-20T01:48:37.493 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0062702069999999999, 2024-04-20T01:48:37.493 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:37.493 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.493 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:37.493 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.493 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:37.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.825844+0000", 2024-04-20T01:48:37.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:37.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.825844+0000", 2024-04-20T01:48:37.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.494 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:37.494 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.825843+0000", 2024-04-20T01:48:37.494 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:37.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826258+0000", 2024-04-20T01:48:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00041487099999999998 2024-04-20T01:48:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826269+0000", 2024-04-20T01:48:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0264e-05 2024-04-20T01:48:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826275+0000", 2024-04-20T01:48:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.3010000000000003e-06 2024-04-20T01:48:37.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:37.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826329+0000", 2024-04-20T01:48:37.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.4036999999999998e-05 2024-04-20T01:48:37.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.496 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.496 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.826337+0000", 2024-04-20T01:48:37.496 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.7470000000000005e-06 2024-04-20T01:48:37.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.496 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.497 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.832089+0000", 2024-04-20T01:48:37.497 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0057520339999999996 2024-04-20T01:48:37.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.497 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:37.497 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.832114+0000", 2024-04-20T01:48:37.497 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5267999999999999e-05 2024-04-20T01:48:37.497 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.497 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.497 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.497 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.498 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.es6 33/30 ECSubWrite(tid=3, reqid=client.4643.0:104, at_version=33'4, trim_to=0'0, roll_forward_to=33'1))", 2024-04-20T01:48:37.498 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:34.966859+0000", 2024-04-20T01:48:37.498 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 62.521485368999997, 2024-04-20T01:48:37.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0063969730000000002, 2024-04-20T01:48:37.498 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:37.498 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.498 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:37.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.498 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:37.498 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966859+0000", 2024-04-20T01:48:37.498 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.498 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.498 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:37.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966859+0000", 2024-04-20T01:48:37.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.499 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:37.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966859+0000", 2024-04-20T01:48:37.499 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:37.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.499 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.499 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:37.499 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966876+0000", 2024-04-20T01:48:37.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7116999999999999e-05 2024-04-20T01:48:37.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.500 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.500 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:37.500 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966887+0000", 2024-04-20T01:48:37.500 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1508999999999999e-05 2024-04-20T01:48:37.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.501 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.966894+0000", 2024-04-20T01:48:37.501 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.4219999999999997e-06 2024-04-20T01:48:37.501 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.501 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:37.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.967342+0000", 2024-04-20T01:48:37.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00044805799999999998 2024-04-20T01:48:37.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.502 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.967352+0000", 2024-04-20T01:48:37.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0497e-05 2024-04-20T01:48:37.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.502 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.502 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.973223+0000", 2024-04-20T01:48:37.502 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0058706749999999997 2024-04-20T01:48:37.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.502 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.502 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:37.503 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:34.973256+0000", 2024-04-20T01:48:37.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3544999999999998e-05 2024-04-20T01:48:37.503 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.503 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.503 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.503 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 47/30 ECSubWrite(tid=114, reqid=client.4643.0:392, at_version=47'49, trim_to=0'0, roll_forward_to=47'47))", 2024-04-20T01:48:37.503 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:47:48.746767+0000", 2024-04-20T01:48:37.503 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 48.741577864999996, 2024-04-20T01:48:37.503 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0061268629999999998, 2024-04-20T01:48:37.503 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:37.503 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.503 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:37.503 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.504 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:37.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746767+0000", 2024-04-20T01:48:37.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.504 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:37.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746767+0000", 2024-04-20T01:48:37.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.504 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.504 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:37.504 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746766+0000", 2024-04-20T01:48:37.504 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:37.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:37.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746823+0000", 2024-04-20T01:48:37.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.6743999999999997e-05 2024-04-20T01:48:37.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:37.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746840+0000", 2024-04-20T01:48:37.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7300999999999999e-05 2024-04-20T01:48:37.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.505 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.505 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.505 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746850+0000", 2024-04-20T01:48:37.505 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0356999999999999e-05 2024-04-20T01:48:37.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:37.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746919+0000", 2024-04-20T01:48:37.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8180000000000001e-05 2024-04-20T01:48:37.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.746927+0000", 2024-04-20T01:48:37.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.3659999999999992e-06 2024-04-20T01:48:37.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.506 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.506 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.506 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.752855+0000", 2024-04-20T01:48:37.506 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0059277779999999999 2024-04-20T01:48:37.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.507 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:37.507 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:47:48.752894+0000", 2024-04-20T01:48:37.507 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.9057999999999998e-05 2024-04-20T01:48:37.507 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.507 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.507 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.507 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 60/30 ECSubWrite(tid=144, reqid=client.4643.0:506, at_version=60'65, trim_to=0'0, roll_forward_to=60'62))", 2024-04-20T01:48:37.508 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:01.773724+0000", 2024-04-20T01:48:37.508 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 35.714620644, 2024-04-20T01:48:37.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0062789609999999996, 2024-04-20T01:48:37.508 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:37.508 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.508 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:37.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.508 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:37.508 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773724+0000", 2024-04-20T01:48:37.508 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.508 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.508 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:37.509 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773724+0000", 2024-04-20T01:48:37.509 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:37.509 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773723+0000", 2024-04-20T01:48:37.509 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:37.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:37.509 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773743+0000", 2024-04-20T01:48:37.509 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.0032000000000001e-05 2024-04-20T01:48:37.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.509 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.509 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:37.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773755+0000", 2024-04-20T01:48:37.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1156e-05 2024-04-20T01:48:37.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773763+0000", 2024-04-20T01:48:37.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.2870000000000004e-06 2024-04-20T01:48:37.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:37.510 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773830+0000", 2024-04-20T01:48:37.510 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7269999999999998e-05 2024-04-20T01:48:37.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.510 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.510 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.773840+0000", 2024-04-20T01:48:37.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0284e-05 2024-04-20T01:48:37.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.511 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.779970+0000", 2024-04-20T01:48:37.511 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.006129364 2024-04-20T01:48:37.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.511 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.511 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:01.780003+0000", 2024-04-20T01:48:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.3402000000000003e-05 2024-04-20T01:48:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 65/30 ECSubWrite(tid=174, reqid=osd.4.0:174, at_version=65'88, trim_to=0'0, roll_forward_to=63'84))", 2024-04-20T01:48:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:06.763588+0000", 2024-04-20T01:48:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 30.724756724999999, 2024-04-20T01:48:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0081115820000000009, 2024-04-20T01:48:37.512 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.763588+0000", 2024-04-20T01:48:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.763588+0000", 2024-04-20T01:48:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:37.513 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.763587+0000", 2024-04-20T01:48:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.763612+0000", 2024-04-20T01:48:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.5106999999999999e-05 2024-04-20T01:48:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.763624+0000", 2024-04-20T01:48:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1487e-05 2024-04-20T01:48:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.514 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.763631+0000", 2024-04-20T01:48:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0249999999999997e-06 2024-04-20T01:48:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.764079+0000", 2024-04-20T01:48:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00044799400000000001 2024-04-20T01:48:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.764088+0000", 2024-04-20T01:48:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5240000000000006e-06 2024-04-20T01:48:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.515 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.516 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.771664+0000", 2024-04-20T01:48:37.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0075759160000000002 2024-04-20T01:48:37.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.516 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:37.516 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:06.771700+0000", 2024-04-20T01:48:37.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.5493000000000001e-05 2024-04-20T01:48:37.516 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.516 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.516 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.516 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.516 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 68/30 ECSubWrite(tid=209, reqid=client.4643.0:670, at_version=68'110, trim_to=0'0, roll_forward_to=68'107))", 2024-04-20T01:48:37.516 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:09.945938+0000", 2024-04-20T01:48:37.516 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.542407309000001, 2024-04-20T01:48:37.516 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.006638194, 2024-04-20T01:48:37.517 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:37.517 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.517 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:37.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:37.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.945938+0000", 2024-04-20T01:48:37.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.517 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.517 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:37.517 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.945938+0000", 2024-04-20T01:48:37.517 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.518 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:37.518 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.945937+0000", 2024-04-20T01:48:37.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:37.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.518 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.518 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:37.518 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.946003+0000", 2024-04-20T01:48:37.518 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.6026999999999997e-05 2024-04-20T01:48:37.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.519 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.519 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:37.519 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.946019+0000", 2024-04-20T01:48:37.519 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6458000000000001e-05 2024-04-20T01:48:37.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.519 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.519 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.519 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.946029+0000", 2024-04-20T01:48:37.519 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.55e-06 2024-04-20T01:48:37.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.519 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.519 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:37.519 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.946077+0000", 2024-04-20T01:48:37.519 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8205999999999999e-05 2024-04-20T01:48:37.519 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.520 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.520 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.520 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.946089+0000", 2024-04-20T01:48:37.520 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2e-05 2024-04-20T01:48:37.520 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.520 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.520 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.520 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.952557+0000", 2024-04-20T01:48:37.520 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0064676919999999997 2024-04-20T01:48:37.520 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.520 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.520 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:37.520 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:09.952576+0000", 2024-04-20T01:48:37.520 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8838000000000002e-05 2024-04-20T01:48:37.520 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.521 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.521 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.521 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 68/30 ECSubWrite(tid=220, reqid=client.4643.0:703, at_version=68'117, trim_to=0'0, roll_forward_to=68'116))", 2024-04-20T01:48:37.521 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:10.059553+0000", 2024-04-20T01:48:37.521 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.428791602, 2024-04-20T01:48:37.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0062243860000000002, 2024-04-20T01:48:37.521 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:37.521 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.521 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:37.521 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.521 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:37.521 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059553+0000", 2024-04-20T01:48:37.521 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:37.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059553+0000", 2024-04-20T01:48:37.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:37.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059552+0000", 2024-04-20T01:48:37.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:37.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.522 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:37.522 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059733+0000", 2024-04-20T01:48:37.522 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00018084899999999999 2024-04-20T01:48:37.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:37.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059773+0000", 2024-04-20T01:48:37.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.0985999999999997e-05 2024-04-20T01:48:37.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059801+0000", 2024-04-20T01:48:37.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.7727999999999999e-05 2024-04-20T01:48:37.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.523 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.523 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:37.523 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059897+0000", 2024-04-20T01:48:37.523 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.5471999999999997e-05 2024-04-20T01:48:37.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.059911+0000", 2024-04-20T01:48:37.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4644000000000001e-05 2024-04-20T01:48:37.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.524 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.524 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.065740+0000", 2024-04-20T01:48:37.524 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0058290709999999999 2024-04-20T01:48:37.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.524 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.525 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:37.525 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.065778+0000", 2024-04-20T01:48:37.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7234999999999998e-05 2024-04-20T01:48:37.525 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.525 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.525 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.525 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.525 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 68/30 ECSubWrite(tid=221, reqid=client.4643.0:704, at_version=68'118, trim_to=0'0, roll_forward_to=68'116))", 2024-04-20T01:48:37.525 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:10.064387+0000", 2024-04-20T01:48:37.525 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 27.423957739999999, 2024-04-20T01:48:37.525 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068350370000000004, 2024-04-20T01:48:37.526 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:37.526 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.526 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:37.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:37.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064387+0000", 2024-04-20T01:48:37.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.526 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:37.526 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064387+0000", 2024-04-20T01:48:37.526 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.527 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:37.527 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064386+0000", 2024-04-20T01:48:37.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:37.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.527 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:37.527 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064602+0000", 2024-04-20T01:48:37.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00021609999999999999 2024-04-20T01:48:37.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.527 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:37.527 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064651+0000", 2024-04-20T01:48:37.527 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8976e-05 2024-04-20T01:48:37.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.527 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064664+0000", 2024-04-20T01:48:37.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2632999999999999e-05 2024-04-20T01:48:37.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.528 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:37.528 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064732+0000", 2024-04-20T01:48:37.528 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.8047999999999996e-05 2024-04-20T01:48:37.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.528 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.064748+0000", 2024-04-20T01:48:37.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6640999999999999e-05 2024-04-20T01:48:37.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.071185+0000", 2024-04-20T01:48:37.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0064368109999999997 2024-04-20T01:48:37.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.529 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.529 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:37.529 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:10.071222+0000", 2024-04-20T01:48:37.529 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6942e-05 2024-04-20T01:48:37.529 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.529 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.530 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.530 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs5 75/30 ECSubWrite(tid=117, reqid=client.4643.0:785, at_version=75'98, trim_to=0'0, roll_forward_to=75'93))", 2024-04-20T01:48:37.530 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:16.750149+0000", 2024-04-20T01:48:37.530 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 20.738196250000001, 2024-04-20T01:48:37.530 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0088344489999999994, 2024-04-20T01:48:37.530 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:37.530 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.530 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:37.530 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.530 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:37.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.750149+0000", 2024-04-20T01:48:37.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:37.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.750149+0000", 2024-04-20T01:48:37.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:37.531 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.750148+0000", 2024-04-20T01:48:37.531 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:37.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.531 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.531 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:37.532 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.750323+0000", 2024-04-20T01:48:37.532 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017526299999999999 2024-04-20T01:48:37.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.532 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:37.532 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.750340+0000", 2024-04-20T01:48:37.532 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7241999999999999e-05 2024-04-20T01:48:37.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.532 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.532 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.750351+0000", 2024-04-20T01:48:37.532 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0968000000000001e-05 2024-04-20T01:48:37.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.532 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.532 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:37.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.750419+0000", 2024-04-20T01:48:37.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.7500000000000001e-05 2024-04-20T01:48:37.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.750432+0000", 2024-04-20T01:48:37.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3232e-05 2024-04-20T01:48:37.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.533 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.758945+0000", 2024-04-20T01:48:37.533 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0085134830000000005 2024-04-20T01:48:37.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.533 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.533 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:37.534 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.758983+0000", 2024-04-20T01:48:37.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.7734000000000002e-05 2024-04-20T01:48:37.534 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.534 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.534 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.534 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.534 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs5 75/30 ECSubWrite(tid=118, reqid=client.4643.0:786, at_version=75'99, trim_to=0'0, roll_forward_to=75'95))", 2024-04-20T01:48:37.534 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:16.754332+0000", 2024-04-20T01:48:37.534 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 20.734012979999999, 2024-04-20T01:48:37.534 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0068657700000000002, 2024-04-20T01:48:37.534 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:37.534 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.535 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:37.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:37.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.754332+0000", 2024-04-20T01:48:37.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:37.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.754332+0000", 2024-04-20T01:48:37.535 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.535 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:37.535 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.754327+0000", 2024-04-20T01:48:37.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999952 2024-04-20T01:48:37.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:37.536 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.754656+0000", 2024-04-20T01:48:37.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00032840000000000001 2024-04-20T01:48:37.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:37.536 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.754677+0000", 2024-04-20T01:48:37.536 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.1089000000000001e-05 2024-04-20T01:48:37.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.536 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.536 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.537 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.754687+0000", 2024-04-20T01:48:37.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0838e-05 2024-04-20T01:48:37.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.537 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:37.537 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.754736+0000", 2024-04-20T01:48:37.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.8596e-05 2024-04-20T01:48:37.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.537 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.537 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.754743+0000", 2024-04-20T01:48:37.537 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 7.0160000000000003e-06 2024-04-20T01:48:37.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.537 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.537 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.537 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.761161+0000", 2024-04-20T01:48:37.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0064178580000000002 2024-04-20T01:48:37.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.538 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:37.538 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.761198+0000", 2024-04-20T01:48:37.538 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6733999999999998e-05 2024-04-20T01:48:37.538 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.538 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.538 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.538 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs5 75/30 ECSubWrite(tid=119, reqid=client.4643.0:787, at_version=75'100, trim_to=0'0, roll_forward_to=75'97))", 2024-04-20T01:48:37.538 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:16.755830+0000", 2024-04-20T01:48:37.539 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 20.732514963, 2024-04-20T01:48:37.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0087352760000000002, 2024-04-20T01:48:37.539 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:37.539 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.539 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:37.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.539 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:37.539 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.755830+0000", 2024-04-20T01:48:37.539 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.539 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:37.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.755830+0000", 2024-04-20T01:48:37.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:37.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.755829+0000", 2024-04-20T01:48:37.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:37.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:37.540 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.755930+0000", 2024-04-20T01:48:37.540 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.000100809 2024-04-20T01:48:37.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.540 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.540 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:37.541 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.755946+0000", 2024-04-20T01:48:37.541 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.6334e-05 2024-04-20T01:48:37.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.541 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.541 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.755964+0000", 2024-04-20T01:48:37.541 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.7442e-05 2024-04-20T01:48:37.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.542 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:37.542 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.756402+0000", 2024-04-20T01:48:37.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00043873000000000002 2024-04-20T01:48:37.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.542 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.542 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.756415+0000", 2024-04-20T01:48:37.542 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2687000000000001e-05 2024-04-20T01:48:37.542 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.542 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.542 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.542 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.764523+0000", 2024-04-20T01:48:37.543 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0081081409999999993 2024-04-20T01:48:37.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.543 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.543 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:37.543 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.764565+0000", 2024-04-20T01:48:37.543 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4.2067000000000002e-05 2024-04-20T01:48:37.543 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.543 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.544 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.544 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 75/30 ECSubWrite(tid=265, reqid=osd.4.0:261, at_version=0'0, trim_to=0'0, roll_forward_to=75'141))", 2024-04-20T01:48:37.544 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:16.757192+0000", 2024-04-20T01:48:37.544 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 20.731153176999999, 2024-04-20T01:48:37.544 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0074381890000000004, 2024-04-20T01:48:37.544 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:37.544 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.544 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:37.544 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.544 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:37.544 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.757192+0000", 2024-04-20T01:48:37.544 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.545 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:37.545 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.757192+0000", 2024-04-20T01:48:37.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.545 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:37.545 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.757191+0000", 2024-04-20T01:48:37.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:37.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.545 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.545 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:37.545 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.757220+0000", 2024-04-20T01:48:37.545 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9322000000000001e-05 2024-04-20T01:48:37.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.546 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:37.546 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.757240+0000", 2024-04-20T01:48:37.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9558000000000001e-05 2024-04-20T01:48:37.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.546 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.546 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.757251+0000", 2024-04-20T01:48:37.546 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1151e-05 2024-04-20T01:48:37.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.546 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.546 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:37.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.757339+0000", 2024-04-20T01:48:37.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 8.7825999999999996e-05 2024-04-20T01:48:37.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.757349+0000", 2024-04-20T01:48:37.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.0633e-05 2024-04-20T01:48:37.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.547 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.547 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.764574+0000", 2024-04-20T01:48:37.547 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0072251939999999999 2024-04-20T01:48:37.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.548 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:37.548 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:16.764630+0000", 2024-04-20T01:48:37.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.5454999999999997e-05 2024-04-20T01:48:37.548 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.548 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.548 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.548 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.cs5 80/30 ECSubWrite(tid=132, reqid=osd.14.0:132, at_version=80'111, trim_to=0'0, roll_forward_to=77'109))", 2024-04-20T01:48:37.548 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:21.730090+0000", 2024-04-20T01:48:37.548 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 15.758254357, 2024-04-20T01:48:37.548 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0095969000000000002, 2024-04-20T01:48:37.548 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:37.548 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.549 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:37.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:37.549 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.730090+0000", 2024-04-20T01:48:37.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.549 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.549 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:37.549 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.730090+0000", 2024-04-20T01:48:37.549 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.550 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.550 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:37.550 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.730090+0000", 2024-04-20T01:48:37.550 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:37.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.550 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.550 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:37.550 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.730114+0000", 2024-04-20T01:48:37.550 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.4318e-05 2024-04-20T01:48:37.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.550 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.550 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:37.550 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.730129+0000", 2024-04-20T01:48:37.551 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.4748e-05 2024-04-20T01:48:37.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.551 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.551 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.730141+0000", 2024-04-20T01:48:37.551 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1473e-05 2024-04-20T01:48:37.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.551 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:37.551 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.730312+0000", 2024-04-20T01:48:37.551 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.00017122200000000001 2024-04-20T01:48:37.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.551 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.551 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.551 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.730327+0000", 2024-04-20T01:48:37.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5393e-05 2024-04-20T01:48:37.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.739651+0000", 2024-04-20T01:48:37.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0093232639999999995 2024-04-20T01:48:37.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.552 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.552 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:37.552 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.739687+0000", 2024-04-20T01:48:37.552 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6862e-05 2024-04-20T01:48:37.552 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.552 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.552 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.553 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 80/30 ECSubWrite(tid=303, reqid=osd.4.0:303, at_version=80'162, trim_to=0'0, roll_forward_to=77'158))", 2024-04-20T01:48:37.553 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:21.730837+0000", 2024-04-20T01:48:37.553 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 15.757507542999999, 2024-04-20T01:48:37.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013235546000000001, 2024-04-20T01:48:37.553 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:37.553 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.553 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:37.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.553 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:37.553 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.730837+0000", 2024-04-20T01:48:37.553 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.553 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.553 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:37.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.730837+0000", 2024-04-20T01:48:37.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:37.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.730837+0000", 2024-04-20T01:48:37.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.999999 2024-04-20T01:48:37.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.554 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:37.554 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.730863+0000", 2024-04-20T01:48:37.554 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.6123e-05 2024-04-20T01:48:37.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.555 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:37.555 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.730875+0000", 2024-04-20T01:48:37.555 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2712e-05 2024-04-20T01:48:37.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.555 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.555 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.730904+0000", 2024-04-20T01:48:37.555 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.8249e-05 2024-04-20T01:48:37.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.555 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:37.555 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.730961+0000", 2024-04-20T01:48:37.555 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.7388e-05 2024-04-20T01:48:37.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.556 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.556 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.730974+0000", 2024-04-20T01:48:37.556 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3189e-05 2024-04-20T01:48:37.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.556 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.556 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.744044+0000", 2024-04-20T01:48:37.556 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.013069646000000001 2024-04-20T01:48:37.556 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.556 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.556 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:37.556 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.744073+0000", 2024-04-20T01:48:37.556 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 2.9014999999999999e-05 2024-04-20T01:48:37.556 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.556 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.557 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.557 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.es6 80/30 ECSubWrite(tid=112, reqid=osd.11.0:112, at_version=80'88, trim_to=0'0, roll_forward_to=77'86))", 2024-04-20T01:48:37.557 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:21.733171+0000", 2024-04-20T01:48:37.557 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 15.755174036, 2024-04-20T01:48:37.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010916385000000001, 2024-04-20T01:48:37.557 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:37.557 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.557 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:37.557 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.557 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:37.557 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.733171+0000", 2024-04-20T01:48:37.557 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:37.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.733171+0000", 2024-04-20T01:48:37.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:37.558 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.733170+0000", 2024-04-20T01:48:37.558 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:37.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.558 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.558 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:37.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.733201+0000", 2024-04-20T01:48:37.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.0633000000000003e-05 2024-04-20T01:48:37.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.559 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:37.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.733220+0000", 2024-04-20T01:48:37.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.8529000000000002e-05 2024-04-20T01:48:37.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.559 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.559 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.733231+0000", 2024-04-20T01:48:37.559 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1080999999999999e-05 2024-04-20T01:48:37.559 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.559 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.559 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:37.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.733327+0000", 2024-04-20T01:48:37.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 9.6430999999999999e-05 2024-04-20T01:48:37.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.560 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.560 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.560 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.733341+0000", 2024-04-20T01:48:37.560 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.3875e-05 2024-04-20T01:48:37.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.560 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.560 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.561 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.744024+0000", 2024-04-20T01:48:37.561 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.010683181999999999 2024-04-20T01:48:37.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.561 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.561 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:37.561 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:21.744087+0000", 2024-04-20T01:48:37.561 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 6.2985999999999996e-05 2024-04-20T01:48:37.561 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.561 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.561 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.562 INFO:teuthology.orchestra.run.smithi044.stdout: "description": "MOSDECSubOpWrite(3.bs3 91/30 ECSubWrite(tid=344, reqid=osd.4.0:343, at_version=0'0, trim_to=0'0, roll_forward_to=91'174))", 2024-04-20T01:48:37.562 INFO:teuthology.orchestra.run.smithi044.stdout: "initiated_at": "2024-04-20T01:48:32.726517+0000", 2024-04-20T01:48:37.562 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 4.7618275590000003, 2024-04-20T01:48:37.562 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0087826999999999992, 2024-04-20T01:48:37.562 INFO:teuthology.orchestra.run.smithi044.stdout: "type_data": { 2024-04-20T01:48:37.562 INFO:teuthology.orchestra.run.smithi044.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.562 INFO:teuthology.orchestra.run.smithi044.stdout: "events": [ 2024-04-20T01:48:37.562 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.562 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "initiated", 2024-04-20T01:48:37.562 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.726517+0000", 2024-04-20T01:48:37.562 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.563 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "throttled", 2024-04-20T01:48:37.563 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.726517+0000", 2024-04-20T01:48:37.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0 2024-04-20T01:48:37.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.563 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "header_read", 2024-04-20T01:48:37.563 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.726516+0000", 2024-04-20T01:48:37.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:37.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.563 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "all_read", 2024-04-20T01:48:37.563 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.726531+0000", 2024-04-20T01:48:37.563 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.5029e-05 2024-04-20T01:48:37.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "dispatched", 2024-04-20T01:48:37.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.726550+0000", 2024-04-20T01:48:37.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.9426999999999999e-05 2024-04-20T01:48:37.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.726562+0000", 2024-04-20T01:48:37.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.1455e-05 2024-04-20T01:48:37.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.564 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.564 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "reached_pg", 2024-04-20T01:48:37.564 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.726615+0000", 2024-04-20T01:48:37.564 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 5.3211000000000003e-05 2024-04-20T01:48:37.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.726627+0000", 2024-04-20T01:48:37.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 1.2397e-05 2024-04-20T01:48:37.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.735264+0000", 2024-04-20T01:48:37.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 0.0086363510000000004 2024-04-20T01:48:37.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-20T01:48:37.565 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-20T01:48:37.565 INFO:teuthology.orchestra.run.smithi044.stdout: "event": "done", 2024-04-20T01:48:37.565 INFO:teuthology.orchestra.run.smithi044.stdout: "time": "2024-04-20T01:48:32.735300+0000", 2024-04-20T01:48:37.565 INFO:teuthology.orchestra.run.smithi044.stdout: "duration": 3.6198999999999997e-05 2024-04-20T01:48:37.565 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.565 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.566 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.566 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-20T01:48:37.566 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-20T01:48:37.566 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:37.567 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /var/log/ceph/exp.3.5s3.14 2024-04-20T01:48:37.568 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-20T01:48:37.572 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:37.585+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:37.672 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:37.686+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:37.709 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:37.709 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:37.710 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 0 2024-04-20T01:48:37.710 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:37.720 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-20T01:48:37.772 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:37.787+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:37.800 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-20T01:48:37.849 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:37.849 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:37.849 INFO:teuthology.orchestra.run.smithi121.stdout: "complaint_time": 30, 2024-04-20T01:48:37.849 INFO:teuthology.orchestra.run.smithi121.stdout: "num_blocked_ops": 0 2024-04-20T01:48:37.850 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:37.859 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-20T01:48:37.874 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:37.888+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:37.975 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:37.990+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:37.991 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:37.991 INFO:teuthology.orchestra.run.smithi121.stdout: "size": 20, 2024-04-20T01:48:37.991 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 600, 2024-04-20T01:48:37.991 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-04-20T01:48:37.991 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.991 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 47/30 ECSubWrite(tid=246, reqid=client.4643.0:388, at_version=47'38, trim_to=0'0, roll_forward_to=47'35))", 2024-04-20T01:48:37.991 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.745219+0000", 2024-04-20T01:48:37.991 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 49.259030064000001, 2024-04-20T01:48:37.991 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0073977050000000001, 2024-04-20T01:48:37.991 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:37.992 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.992 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:37.992 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.992 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:37.992 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745219+0000", 2024-04-20T01:48:37.992 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:37.992 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:37.992 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.992 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:37.992 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745219+0000", 2024-04-20T01:48:37.992 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:37.992 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:37.992 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.992 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:37.993 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745218+0000", 2024-04-20T01:48:37.993 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:37.993 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:37.993 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.993 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:37.993 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745266+0000", 2024-04-20T01:48:37.993 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.7012999999999999e-05 2024-04-20T01:48:37.993 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:37.993 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.993 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:37.993 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745277+0000", 2024-04-20T01:48:37.993 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1348999999999999e-05 2024-04-20T01:48:37.994 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:37.994 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.994 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.994 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745284+0000", 2024-04-20T01:48:37.994 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0060000000000003e-06 2024-04-20T01:48:37.994 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:37.994 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.994 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:37.994 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745340+0000", 2024-04-20T01:48:37.994 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6023000000000002e-05 2024-04-20T01:48:37.994 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:37.994 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.994 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.994 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.745361+0000", 2024-04-20T01:48:37.995 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1356999999999999e-05 2024-04-20T01:48:37.995 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:37.995 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.995 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.995 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.752597+0000", 2024-04-20T01:48:37.995 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0072354910000000001 2024-04-20T01:48:37.995 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:37.995 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.995 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:37.995 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.752617+0000", 2024-04-20T01:48:37.995 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0341000000000001e-05 2024-04-20T01:48:37.995 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:37.995 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:37.995 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:37.995 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:37.996 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.996 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 48/30 ECSubWrite(tid=252, reqid=osd.13.0:252, at_version=48'42, trim_to=0'0, roll_forward_to=47'40))", 2024-04-20T01:48:37.996 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.710137+0000", 2024-04-20T01:48:37.996 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 48.294112216000002, 2024-04-20T01:48:37.996 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0082943410000000002, 2024-04-20T01:48:37.996 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:37.996 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:37.996 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:37.996 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.996 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:37.996 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710137+0000", 2024-04-20T01:48:37.996 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:37.996 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:37.997 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.997 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:37.997 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710137+0000", 2024-04-20T01:48:37.997 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:37.997 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:37.997 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.997 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:37.997 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710137+0000", 2024-04-20T01:48:37.997 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:37.997 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:37.997 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.997 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:37.997 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710162+0000", 2024-04-20T01:48:37.997 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4768000000000001e-05 2024-04-20T01:48:37.997 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:37.998 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.998 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:37.998 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710174+0000", 2024-04-20T01:48:37.998 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2150999999999999e-05 2024-04-20T01:48:37.998 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:37.998 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.998 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:37.998 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710183+0000", 2024-04-20T01:48:37.998 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8200000000000003e-06 2024-04-20T01:48:37.998 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:37.998 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.998 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:37.998 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710238+0000", 2024-04-20T01:48:37.998 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.4954999999999998e-05 2024-04-20T01:48:37.999 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:37.999 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.999 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:37.999 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.710248+0000", 2024-04-20T01:48:37.999 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0222e-05 2024-04-20T01:48:37.999 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:37.999 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.999 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:37.999 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718408+0000", 2024-04-20T01:48:37.999 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0081605570000000006 2024-04-20T01:48:37.999 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:37.999 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:37.999 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:37.999 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718432+0000", 2024-04-20T01:48:38.000 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3129000000000002e-05 2024-04-20T01:48:38.000 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.000 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.000 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.000 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.000 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.000 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 48/30 ECSubWrite(tid=253, reqid=osd.13.0:253, at_version=48'44, trim_to=0'0, roll_forward_to=47'40))", 2024-04-20T01:48:38.000 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.711047+0000", 2024-04-20T01:48:38.000 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 48.293202798000003, 2024-04-20T01:48:38.000 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0093898609999999993, 2024-04-20T01:48:38.000 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.000 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.000 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.000 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.001 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.001 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711047+0000", 2024-04-20T01:48:38.001 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.001 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.001 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.001 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.001 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711047+0000", 2024-04-20T01:48:38.001 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.001 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.001 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.001 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.001 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711046+0000", 2024-04-20T01:48:38.001 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:38.002 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.002 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.002 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.002 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711066+0000", 2024-04-20T01:48:38.002 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0421999999999999e-05 2024-04-20T01:48:38.002 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.002 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.002 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.002 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711082+0000", 2024-04-20T01:48:38.002 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5231999999999999e-05 2024-04-20T01:48:38.002 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.002 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.002 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.002 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.711091+0000", 2024-04-20T01:48:38.003 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.1679999999999993e-06 2024-04-20T01:48:38.003 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.003 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.003 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.003 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.712543+0000", 2024-04-20T01:48:38.003 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001452545 2024-04-20T01:48:38.003 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.003 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.003 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.003 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.712549+0000", 2024-04-20T01:48:38.003 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.7450000000000001e-06 2024-04-20T01:48:38.003 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.003 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.003 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.004 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720419+0000", 2024-04-20T01:48:38.004 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0078696630000000007 2024-04-20T01:48:38.004 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.004 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.004 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.004 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.720436+0000", 2024-04-20T01:48:38.004 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7869000000000001e-05 2024-04-20T01:48:38.004 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.005 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.005 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.005 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.005 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.005 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 48/30 ECSubWrite(tid=121, reqid=osd.4.0:121, at_version=48'53, trim_to=0'0, roll_forward_to=47'49))", 2024-04-20T01:48:38.005 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.718319+0000", 2024-04-20T01:48:38.005 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 48.285930843000003, 2024-04-20T01:48:38.005 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0075908039999999996, 2024-04-20T01:48:38.005 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.005 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.005 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.005 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.005 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.006 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718319+0000", 2024-04-20T01:48:38.006 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.006 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.006 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.006 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.006 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718319+0000", 2024-04-20T01:48:38.006 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.006 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.006 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.006 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.006 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718318+0000", 2024-04-20T01:48:38.006 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:38.007 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.007 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.007 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.007 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718351+0000", 2024-04-20T01:48:38.007 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.3179000000000002e-05 2024-04-20T01:48:38.007 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.007 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.007 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.007 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718362+0000", 2024-04-20T01:48:38.007 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0689999999999999e-05 2024-04-20T01:48:38.007 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.007 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.007 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.007 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718370+0000", 2024-04-20T01:48:38.007 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4610000000000008e-06 2024-04-20T01:48:38.008 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.008 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.008 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.008 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718703+0000", 2024-04-20T01:48:38.008 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00033314100000000001 2024-04-20T01:48:38.008 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.008 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.008 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.008 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.718713+0000", 2024-04-20T01:48:38.008 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.5459999999999997e-06 2024-04-20T01:48:38.008 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.008 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.008 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.008 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.725887+0000", 2024-04-20T01:48:38.008 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0071743909999999996 2024-04-20T01:48:38.008 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.009 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.009 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.009 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.725909+0000", 2024-04-20T01:48:38.009 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1965999999999999e-05 2024-04-20T01:48:38.009 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.009 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.009 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.009 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.009 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.009 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 63/30 ECSubWrite(tid=166, reqid=client.4643.0:567, at_version=63'83, trim_to=0'0, roll_forward_to=62'79))", 2024-04-20T01:48:38.009 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:04.764484+0000", 2024-04-20T01:48:38.009 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 33.239765402000003, 2024-04-20T01:48:38.009 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0083177819999999993, 2024-04-20T01:48:38.010 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.010 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.010 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.010 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.010 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.010 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764484+0000", 2024-04-20T01:48:38.010 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.010 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.010 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.010 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.010 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764484+0000", 2024-04-20T01:48:38.010 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.010 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.010 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.011 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.011 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764483+0000", 2024-04-20T01:48:38.011 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:38.011 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.011 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.011 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.011 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764821+0000", 2024-04-20T01:48:38.011 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00033812600000000001 2024-04-20T01:48:38.011 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.012 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.012 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.012 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764833+0000", 2024-04-20T01:48:38.012 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1722000000000001e-05 2024-04-20T01:48:38.012 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.012 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.012 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.012 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764843+0000", 2024-04-20T01:48:38.012 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.431e-06 2024-04-20T01:48:38.012 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.012 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.012 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.012 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764896+0000", 2024-04-20T01:48:38.012 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.2988000000000002e-05 2024-04-20T01:48:38.012 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.013 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.013 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.013 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.764905+0000", 2024-04-20T01:48:38.013 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.1079999999999999e-06 2024-04-20T01:48:38.013 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.013 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.013 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.013 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.772779+0000", 2024-04-20T01:48:38.013 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0078742910000000003 2024-04-20T01:48:38.013 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.013 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.013 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.013 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.772802+0000", 2024-04-20T01:48:38.013 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2869000000000001e-05 2024-04-20T01:48:38.014 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.014 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.014 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.014 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.014 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.014 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 75/57 ECSubWrite(tid=188, reqid=osd.8.0:188, at_version=75'132, trim_to=0'0, roll_forward_to=75'128))", 2024-04-20T01:48:38.014 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.730586+0000", 2024-04-20T01:48:38.014 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.273663055, 2024-04-20T01:48:38.014 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010377753999999999, 2024-04-20T01:48:38.014 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.014 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.014 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.014 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.014 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.015 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730586+0000", 2024-04-20T01:48:38.015 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.015 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.015 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.015 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.015 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730586+0000", 2024-04-20T01:48:38.015 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.015 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.015 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.015 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.015 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730586+0000", 2024-04-20T01:48:38.015 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:38.015 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.015 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.015 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.016 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730610+0000", 2024-04-20T01:48:38.016 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4423e-05 2024-04-20T01:48:38.016 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.016 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.016 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.016 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730626+0000", 2024-04-20T01:48:38.016 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5549999999999999e-05 2024-04-20T01:48:38.016 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.016 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.016 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.016 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730635+0000", 2024-04-20T01:48:38.016 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7589999999999992e-06 2024-04-20T01:48:38.016 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.017 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.017 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.017 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730677+0000", 2024-04-20T01:48:38.017 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.1668999999999997e-05 2024-04-20T01:48:38.017 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.017 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.017 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.017 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730685+0000", 2024-04-20T01:48:38.017 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.1009999999999992e-06 2024-04-20T01:48:38.017 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.017 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.017 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.017 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740942+0000", 2024-04-20T01:48:38.017 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010257252 2024-04-20T01:48:38.018 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.018 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.018 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.018 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740964+0000", 2024-04-20T01:48:38.018 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1838999999999999e-05 2024-04-20T01:48:38.018 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.018 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.018 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.018 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.018 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.018 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 75/30 ECSubWrite(tid=256, reqid=client.4643.0:783, at_version=75'137, trim_to=0'0, roll_forward_to=73'131))", 2024-04-20T01:48:38.018 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.730626+0000", 2024-04-20T01:48:38.018 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.273623784000002, 2024-04-20T01:48:38.018 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.014031138, 2024-04-20T01:48:38.019 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.019 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.019 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.019 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.019 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.019 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730626+0000", 2024-04-20T01:48:38.019 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.019 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.019 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.019 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.019 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730626+0000", 2024-04-20T01:48:38.019 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.019 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.019 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.020 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.020 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730625+0000", 2024-04-20T01:48:38.020 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:38.020 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.020 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.020 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.020 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730868+0000", 2024-04-20T01:48:38.020 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000242778 2024-04-20T01:48:38.020 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.020 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.020 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.020 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730887+0000", 2024-04-20T01:48:38.020 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8913999999999999e-05 2024-04-20T01:48:38.020 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.021 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.021 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.021 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730903+0000", 2024-04-20T01:48:38.021 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6841999999999999e-05 2024-04-20T01:48:38.021 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.021 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.021 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.021 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730953+0000", 2024-04-20T01:48:38.021 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.9836999999999997e-05 2024-04-20T01:48:38.021 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.021 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.021 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.021 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730964+0000", 2024-04-20T01:48:38.021 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0209e-05 2024-04-20T01:48:38.022 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.022 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.022 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.022 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744636+0000", 2024-04-20T01:48:38.022 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013672253000000001 2024-04-20T01:48:38.022 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.022 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.022 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.022 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744657+0000", 2024-04-20T01:48:38.022 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1016999999999999e-05 2024-04-20T01:48:38.022 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.022 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.022 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.023 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.023 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.023 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s3 75/30 ECSubWrite(tid=55, reqid=client.4643.0:781, at_version=75'41, trim_to=0'0, roll_forward_to=66'39))", 2024-04-20T01:48:38.023 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.734920+0000", 2024-04-20T01:48:38.023 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.269329797000001, 2024-04-20T01:48:38.023 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.017734534, 2024-04-20T01:48:38.023 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.023 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.023 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.023 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.023 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.023 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.734920+0000", 2024-04-20T01:48:38.023 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.024 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.024 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.024 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.024 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.734920+0000", 2024-04-20T01:48:38.024 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.024 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.024 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.024 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.024 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.734919+0000", 2024-04-20T01:48:38.025 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:48:38.025 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.025 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.025 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.025 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735179+0000", 2024-04-20T01:48:38.025 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00025959699999999997 2024-04-20T01:48:38.025 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.025 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.025 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.025 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735191+0000", 2024-04-20T01:48:38.025 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1606e-05 2024-04-20T01:48:38.025 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.025 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.025 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.025 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735198+0000", 2024-04-20T01:48:38.026 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.2840000000000001e-06 2024-04-20T01:48:38.026 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.026 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.026 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.026 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735251+0000", 2024-04-20T01:48:38.026 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3513000000000001e-05 2024-04-20T01:48:38.026 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.026 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.026 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.026 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735262+0000", 2024-04-20T01:48:38.026 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0556e-05 2024-04-20T01:48:38.027 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.027 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.027 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.027 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.752626+0000", 2024-04-20T01:48:38.027 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.017363983 2024-04-20T01:48:38.027 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.027 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.027 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.027 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.752654+0000", 2024-04-20T01:48:38.027 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8135000000000001e-05 2024-04-20T01:48:38.027 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.028 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.028 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.028 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.028 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.028 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 75/30 ECSubWrite(tid=356, reqid=osd.13.0:356, at_version=75'125, trim_to=0'0, roll_forward_to=75'123))", 2024-04-20T01:48:38.028 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.739126+0000", 2024-04-20T01:48:38.028 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.265123624000001, 2024-04-20T01:48:38.028 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0079893540000000006, 2024-04-20T01:48:38.028 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.028 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.028 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.028 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.029 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.029 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739126+0000", 2024-04-20T01:48:38.029 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.029 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.029 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.029 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.029 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739126+0000", 2024-04-20T01:48:38.029 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.029 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.029 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.030 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.030 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739125+0000", 2024-04-20T01:48:38.030 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:38.030 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.030 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.030 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.030 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739148+0000", 2024-04-20T01:48:38.030 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3448e-05 2024-04-20T01:48:38.030 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.030 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.031 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.031 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739162+0000", 2024-04-20T01:48:38.031 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3492000000000001e-05 2024-04-20T01:48:38.031 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.031 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.031 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.031 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739171+0000", 2024-04-20T01:48:38.031 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3260000000000002e-06 2024-04-20T01:48:38.031 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.031 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.032 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.032 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739908+0000", 2024-04-20T01:48:38.032 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000736482 2024-04-20T01:48:38.032 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.032 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.032 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.032 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739920+0000", 2024-04-20T01:48:38.032 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1963999999999999e-05 2024-04-20T01:48:38.032 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.033 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.033 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.033 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747088+0000", 2024-04-20T01:48:38.033 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0071685769999999998 2024-04-20T01:48:38.033 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.033 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.033 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.033 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747115+0000", 2024-04-20T01:48:38.033 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6826e-05 2024-04-20T01:48:38.033 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.034 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.034 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.034 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.034 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.034 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.2s5 75/30 ECSubWrite(tid=357, reqid=osd.13.0:357, at_version=75'127, trim_to=0'0, roll_forward_to=75'123))", 2024-04-20T01:48:38.034 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.739696+0000", 2024-04-20T01:48:38.034 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.264553436, 2024-04-20T01:48:38.034 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010085257, 2024-04-20T01:48:38.034 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.035 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.035 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.035 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.035 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.035 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739696+0000", 2024-04-20T01:48:38.035 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.035 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.035 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.035 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.035 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739696+0000", 2024-04-20T01:48:38.035 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.036 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.036 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.036 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.036 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739695+0000", 2024-04-20T01:48:38.036 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:38.036 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.036 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.036 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.036 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739724+0000", 2024-04-20T01:48:38.036 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8401000000000001e-05 2024-04-20T01:48:38.036 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.036 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.036 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.036 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739736+0000", 2024-04-20T01:48:38.037 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2340999999999999e-05 2024-04-20T01:48:38.037 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.037 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.037 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.037 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739745+0000", 2024-04-20T01:48:38.037 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.5109999999999992e-06 2024-04-20T01:48:38.037 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.037 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.037 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.037 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.741433+0000", 2024-04-20T01:48:38.038 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0016870329999999999 2024-04-20T01:48:38.038 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.038 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.038 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.038 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.741442+0000", 2024-04-20T01:48:38.038 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7200000000000001e-06 2024-04-20T01:48:38.038 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.038 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.038 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.038 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.749753+0000", 2024-04-20T01:48:38.038 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0083103659999999996 2024-04-20T01:48:38.038 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.038 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.038 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.039 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.749781+0000", 2024-04-20T01:48:38.039 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8665000000000001e-05 2024-04-20T01:48:38.039 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.040 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.040 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.040 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.040 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.040 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 75/30 ECSubWrite(tid=260, reqid=osd.4.0:260, at_version=75'139, trim_to=0'0, roll_forward_to=75'135))", 2024-04-20T01:48:38.040 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.742436+0000", 2024-04-20T01:48:38.041 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.261813415999999, 2024-04-20T01:48:38.041 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.007991382, 2024-04-20T01:48:38.041 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.041 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.041 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.041 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.041 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.041 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742436+0000", 2024-04-20T01:48:38.042 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.042 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.042 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.042 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.042 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742436+0000", 2024-04-20T01:48:38.042 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.042 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.042 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.042 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.042 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742435+0000", 2024-04-20T01:48:38.043 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:38.043 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.043 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.043 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.043 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742465+0000", 2024-04-20T01:48:38.043 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.955e-05 2024-04-20T01:48:38.043 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.043 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.043 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.043 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742484+0000", 2024-04-20T01:48:38.043 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9009999999999999e-05 2024-04-20T01:48:38.043 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.043 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.043 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.043 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742493+0000", 2024-04-20T01:48:38.044 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4550000000000001e-06 2024-04-20T01:48:38.044 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.044 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.044 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.044 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742599+0000", 2024-04-20T01:48:38.044 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00010645999999999999 2024-04-20T01:48:38.044 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.044 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.044 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.044 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742606+0000", 2024-04-20T01:48:38.044 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.6080000000000001e-06 2024-04-20T01:48:38.044 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.044 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.045 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.045 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.750398+0000", 2024-04-20T01:48:38.045 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.007792312 2024-04-20T01:48:38.045 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.045 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.045 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.045 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.750427+0000", 2024-04-20T01:48:38.045 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9019000000000001e-05 2024-04-20T01:48:38.045 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.045 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.045 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.045 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.045 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.045 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 75/30 ECSubWrite(tid=261, reqid=osd.4.0:261, at_version=75'141, trim_to=0'0, roll_forward_to=75'135))", 2024-04-20T01:48:38.045 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.743991+0000", 2024-04-20T01:48:38.046 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.260258360999998, 2024-04-20T01:48:38.046 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011465479000000001, 2024-04-20T01:48:38.046 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.046 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.046 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.046 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.046 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.046 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.743991+0000", 2024-04-20T01:48:38.046 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.046 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.046 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.047 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.047 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.743991+0000", 2024-04-20T01:48:38.047 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.047 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.047 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.047 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.047 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.743990+0000", 2024-04-20T01:48:38.047 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:38.047 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.047 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.048 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.048 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744013+0000", 2024-04-20T01:48:38.048 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2344000000000002e-05 2024-04-20T01:48:38.048 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.048 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.048 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.048 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744023+0000", 2024-04-20T01:48:38.048 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0637e-05 2024-04-20T01:48:38.048 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.048 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.048 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.048 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744032+0000", 2024-04-20T01:48:38.048 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.1729999999999995e-06 2024-04-20T01:48:38.048 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.049 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.049 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.049 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744690+0000", 2024-04-20T01:48:38.049 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00065875600000000003 2024-04-20T01:48:38.049 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.049 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.049 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.049 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744700+0000", 2024-04-20T01:48:38.049 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3710000000000006e-06 2024-04-20T01:48:38.049 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.049 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.049 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.049 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.755437+0000", 2024-04-20T01:48:38.050 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010737412 2024-04-20T01:48:38.050 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.050 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.050 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.050 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.755457+0000", 2024-04-20T01:48:38.050 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9465000000000001e-05 2024-04-20T01:48:38.050 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.050 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.050 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.050 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.050 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.050 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 80/57 ECSubWrite(tid=206, reqid=osd.8.0:206, at_version=80'149, trim_to=0'0, roll_forward_to=79'145))", 2024-04-20T01:48:38.050 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:21.738502+0000", 2024-04-20T01:48:38.051 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 16.265747015999999, 2024-04-20T01:48:38.051 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010687011999999999, 2024-04-20T01:48:38.051 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.051 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.051 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.051 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.051 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.051 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.738502+0000", 2024-04-20T01:48:38.051 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.051 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.051 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.051 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.051 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.738502+0000", 2024-04-20T01:48:38.052 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.052 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.052 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.052 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.052 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.738502+0000", 2024-04-20T01:48:38.052 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:38.052 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.052 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.052 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.052 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.738517+0000", 2024-04-20T01:48:38.052 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5114e-05 2024-04-20T01:48:38.052 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.052 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.052 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.053 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.738525+0000", 2024-04-20T01:48:38.053 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.0190000000000006e-06 2024-04-20T01:48:38.053 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.053 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.053 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.053 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.738531+0000", 2024-04-20T01:48:38.053 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.5860000000000004e-06 2024-04-20T01:48:38.053 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.053 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.053 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.053 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.738790+0000", 2024-04-20T01:48:38.053 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00025967200000000002 2024-04-20T01:48:38.053 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.054 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.054 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.054 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.738802+0000", 2024-04-20T01:48:38.054 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1358e-05 2024-04-20T01:48:38.054 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.054 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.054 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.054 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.749170+0000", 2024-04-20T01:48:38.054 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010368352000000001 2024-04-20T01:48:38.054 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.054 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.054 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.054 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.749189+0000", 2024-04-20T01:48:38.055 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9439000000000001e-05 2024-04-20T01:48:38.055 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.055 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.055 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.055 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.055 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.055 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s3 80/30 ECSubWrite(tid=69, reqid=osd.3.0:69, at_version=80'51, trim_to=0'0, roll_forward_to=77'49))", 2024-04-20T01:48:38.055 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:21.740841+0000", 2024-04-20T01:48:38.055 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 16.263408070000001, 2024-04-20T01:48:38.055 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0083355860000000007, 2024-04-20T01:48:38.055 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.055 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.055 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.056 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.056 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.056 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.740841+0000", 2024-04-20T01:48:38.056 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.056 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.056 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.056 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.056 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.740841+0000", 2024-04-20T01:48:38.056 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.056 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.056 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.056 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.056 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.740841+0000", 2024-04-20T01:48:38.057 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:48:38.057 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.057 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.057 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.057 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.740862+0000", 2024-04-20T01:48:38.057 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0588e-05 2024-04-20T01:48:38.057 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.057 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.057 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.057 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.740873+0000", 2024-04-20T01:48:38.057 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1077000000000001e-05 2024-04-20T01:48:38.057 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.057 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.057 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.058 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.740884+0000", 2024-04-20T01:48:38.058 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1586999999999999e-05 2024-04-20T01:48:38.058 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.058 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.058 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.058 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.740952+0000", 2024-04-20T01:48:38.058 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.7162000000000005e-05 2024-04-20T01:48:38.058 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.058 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.058 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.058 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.740962+0000", 2024-04-20T01:48:38.058 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0859999999999999e-05 2024-04-20T01:48:38.058 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.058 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.059 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.059 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.749152+0000", 2024-04-20T01:48:38.059 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0081900280000000002 2024-04-20T01:48:38.059 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.059 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.059 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.059 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.749177+0000", 2024-04-20T01:48:38.059 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4482e-05 2024-04-20T01:48:38.059 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.059 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.059 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.059 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.059 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.059 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.as2 80/30 ECSubWrite(tid=45, reqid=client.4643.0:908, at_version=80'24, trim_to=0'0, roll_forward_to=66'20))", 2024-04-20T01:48:38.060 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:21.744476+0000", 2024-04-20T01:48:38.060 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 16.259773504000002, 2024-04-20T01:48:38.060 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.007582681, 2024-04-20T01:48:38.060 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.060 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.060 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.060 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.060 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.060 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744476+0000", 2024-04-20T01:48:38.060 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.060 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.060 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.060 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.060 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744476+0000", 2024-04-20T01:48:38.061 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.061 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.061 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.061 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.061 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744475+0000", 2024-04-20T01:48:38.061 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:38.061 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.061 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.061 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.061 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744492+0000", 2024-04-20T01:48:38.061 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6316000000000001e-05 2024-04-20T01:48:38.061 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.061 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.061 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.062 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744504+0000", 2024-04-20T01:48:38.062 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2262999999999999e-05 2024-04-20T01:48:38.062 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.062 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.062 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.062 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744514+0000", 2024-04-20T01:48:38.062 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0285999999999999e-05 2024-04-20T01:48:38.062 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.062 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.062 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.062 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744554+0000", 2024-04-20T01:48:38.062 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.9520000000000001e-05 2024-04-20T01:48:38.062 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.062 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.063 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.063 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744561+0000", 2024-04-20T01:48:38.063 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.3599999999999998e-06 2024-04-20T01:48:38.063 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.063 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.063 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.063 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.752037+0000", 2024-04-20T01:48:38.063 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0074760570000000004 2024-04-20T01:48:38.063 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.063 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.063 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.063 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.752059+0000", 2024-04-20T01:48:38.063 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1484e-05 2024-04-20T01:48:38.063 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.063 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.064 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.064 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 80/57 ECSubWrite(tid=207, reqid=osd.8.0:206, at_version=0'0, trim_to=0'0, roll_forward_to=80'149))", 2024-04-20T01:48:38.064 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:21.753666+0000", 2024-04-20T01:48:38.064 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 16.250582943000001, 2024-04-20T01:48:38.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011599262000000001, 2024-04-20T01:48:38.064 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.064 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.064 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.753666+0000", 2024-04-20T01:48:38.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.753666+0000", 2024-04-20T01:48:38.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.753666+0000", 2024-04-20T01:48:38.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:38.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.753676+0000", 2024-04-20T01:48:38.066 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0369e-05 2024-04-20T01:48:38.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.066 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.066 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.066 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.753688+0000", 2024-04-20T01:48:38.066 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1841999999999999e-05 2024-04-20T01:48:38.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.066 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.066 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.066 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.753698+0000", 2024-04-20T01:48:38.066 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.5740000000000002e-06 2024-04-20T01:48:38.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.066 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.066 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.066 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.753737+0000", 2024-04-20T01:48:38.067 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.8970000000000001e-05 2024-04-20T01:48:38.067 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.067 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.067 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.067 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.753744+0000", 2024-04-20T01:48:38.067 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.9500000000000004e-06 2024-04-20T01:48:38.067 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.067 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.067 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.067 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.765234+0000", 2024-04-20T01:48:38.067 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011490847 2024-04-20T01:48:38.067 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.067 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.067 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.068 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.765266+0000", 2024-04-20T01:48:38.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.1307999999999999e-05 2024-04-20T01:48:38.068 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.068 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.068 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.068 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.068 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.as2 80/30 ECSubWrite(tid=46, reqid=client.4643.0:908, at_version=0'0, trim_to=0'0, roll_forward_to=80'24))", 2024-04-20T01:48:38.068 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:21.754849+0000", 2024-04-20T01:48:38.068 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 16.249400799, 2024-04-20T01:48:38.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010683109, 2024-04-20T01:48:38.068 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.068 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.069 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.069 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.754849+0000", 2024-04-20T01:48:38.069 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.069 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.754849+0000", 2024-04-20T01:48:38.069 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.069 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.754848+0000", 2024-04-20T01:48:38.070 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:38.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.754867+0000", 2024-04-20T01:48:38.070 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8533e-05 2024-04-20T01:48:38.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.754879+0000", 2024-04-20T01:48:38.070 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2176e-05 2024-04-20T01:48:38.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.071 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.754893+0000", 2024-04-20T01:48:38.071 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4543e-05 2024-04-20T01:48:38.071 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.071 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.071 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.071 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.754932+0000", 2024-04-20T01:48:38.071 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.8804000000000003e-05 2024-04-20T01:48:38.071 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.071 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.071 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.071 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.754938+0000", 2024-04-20T01:48:38.071 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.7969999999999997e-06 2024-04-20T01:48:38.071 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.072 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.072 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.072 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.765512+0000", 2024-04-20T01:48:38.072 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010573747999999999 2024-04-20T01:48:38.072 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.072 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.072 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.072 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.765532+0000", 2024-04-20T01:48:38.072 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0176000000000001e-05 2024-04-20T01:48:38.072 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.072 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.072 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.072 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.072 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.072 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.4s3 80/57 ECSubWrite(tid=208, reqid=osd.8.0:208, at_version=80'151, trim_to=0'0, roll_forward_to=80'149))", 2024-04-20T01:48:38.073 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:21.754932+0000", 2024-04-20T01:48:38.073 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 16.249317435999998, 2024-04-20T01:48:38.073 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010601256, 2024-04-20T01:48:38.073 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.073 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.073 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.073 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.073 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.754932+0000", 2024-04-20T01:48:38.073 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.073 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.073 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.073 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.754932+0000", 2024-04-20T01:48:38.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.074 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.754932+0000", 2024-04-20T01:48:38.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:38.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.074 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.754950+0000", 2024-04-20T01:48:38.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8410000000000002e-05 2024-04-20T01:48:38.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.754961+0000", 2024-04-20T01:48:38.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1028e-05 2024-04-20T01:48:38.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.075 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.754969+0000", 2024-04-20T01:48:38.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.7430000000000002e-06 2024-04-20T01:48:38.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.075 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.755002+0000", 2024-04-20T01:48:38.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.3630999999999997e-05 2024-04-20T01:48:38.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.076 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.076 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.755008+0000", 2024-04-20T01:48:38.076 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.959e-06 2024-04-20T01:48:38.076 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.076 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.076 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.076 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.765517+0000", 2024-04-20T01:48:38.076 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010508257999999999 2024-04-20T01:48:38.076 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.076 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.076 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.076 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.765533+0000", 2024-04-20T01:48:38.076 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6617999999999999e-05 2024-04-20T01:48:38.076 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.076 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.077 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.077 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs2 91/30 ECSubWrite(tid=344, reqid=osd.4.0:343, at_version=0'0, trim_to=0'0, roll_forward_to=91'174))", 2024-04-20T01:48:38.077 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:32.726642+0000", 2024-04-20T01:48:38.077 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.2776074050000004, 2024-04-20T01:48:38.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0080847980000000007, 2024-04-20T01:48:38.077 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.077 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.077 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.077 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.077 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.726642+0000", 2024-04-20T01:48:38.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.078 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.078 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.078 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.078 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.726642+0000", 2024-04-20T01:48:38.078 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.078 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.078 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.078 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.078 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.726641+0000", 2024-04-20T01:48:38.078 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:38.078 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.078 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.079 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.726657+0000", 2024-04-20T01:48:38.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6019e-05 2024-04-20T01:48:38.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.079 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.726670+0000", 2024-04-20T01:48:38.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2473999999999999e-05 2024-04-20T01:48:38.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.079 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.726680+0000", 2024-04-20T01:48:38.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0334e-05 2024-04-20T01:48:38.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.080 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.080 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.726735+0000", 2024-04-20T01:48:38.080 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.4540000000000003e-05 2024-04-20T01:48:38.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.080 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.080 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.726745+0000", 2024-04-20T01:48:38.080 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0614e-05 2024-04-20T01:48:38.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.080 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.080 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.734690+0000", 2024-04-20T01:48:38.080 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0079451160000000003 2024-04-20T01:48:38.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.081 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.081 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.734727+0000", 2024-04-20T01:48:38.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.6570999999999999e-05 2024-04-20T01:48:38.081 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.081 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.081 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.081 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s3 91/30 ECSubWrite(tid=99, reqid=osd.3.0:98, at_version=0'0, trim_to=0'0, roll_forward_to=91'73))", 2024-04-20T01:48:38.081 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:32.728894+0000", 2024-04-20T01:48:38.081 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.2753550999999996, 2024-04-20T01:48:38.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0086564109999999993, 2024-04-20T01:48:38.081 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.082 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.082 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.082 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.082 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.082 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.728894+0000", 2024-04-20T01:48:38.082 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.082 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.082 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.082 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.728894+0000", 2024-04-20T01:48:38.082 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.083 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.728893+0000", 2024-04-20T01:48:38.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:38.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.083 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.728907+0000", 2024-04-20T01:48:38.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3371e-05 2024-04-20T01:48:38.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.083 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.728923+0000", 2024-04-20T01:48:38.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6631999999999999e-05 2024-04-20T01:48:38.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.084 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.728933+0000", 2024-04-20T01:48:38.084 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.9389999999999992e-06 2024-04-20T01:48:38.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.084 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.728978+0000", 2024-04-20T01:48:38.084 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.4716999999999999e-05 2024-04-20T01:48:38.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.084 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.728988+0000", 2024-04-20T01:48:38.085 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0190999999999999e-05 2024-04-20T01:48:38.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.085 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.085 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.085 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.737515+0000", 2024-04-20T01:48:38.085 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0085266349999999994 2024-04-20T01:48:38.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.085 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.085 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.085 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.737551+0000", 2024-04-20T01:48:38.085 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.5858000000000001e-05 2024-04-20T01:48:38.085 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.085 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.085 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.086 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.086 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.086 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:38.087 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:38.101+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:38.088 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-20T01:48:38.188 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:38.202+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:38.217 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:38.217 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:38.217 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 0 2024-04-20T01:48:38.217 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:38.227 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-20T01:48:38.289 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:38.303+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:38.310 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-20T01:48:38.355 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:38.355 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:38.355 INFO:teuthology.orchestra.run.smithi121.stdout: "complaint_time": 30, 2024-04-20T01:48:38.355 INFO:teuthology.orchestra.run.smithi121.stdout: "num_blocked_ops": 0 2024-04-20T01:48:38.355 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:38.365 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-20T01:48:38.390 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:38.404+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:38.477 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:48:38.491 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:38.506+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:38.498 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:48:38.518 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:48:38.538 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:38.538 INFO:teuthology.orchestra.run.smithi121.stdout: "size": 20, 2024-04-20T01:48:38.538 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 600, 2024-04-20T01:48:38.538 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-04-20T01:48:38.538 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.538 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:307 3.8s0 3:16aac79f:::smithi16579500-14:head [copy-from ver 2] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:38.539 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.823600+0000", 2024-04-20T01:48:38.539 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 54.724860419000002, 2024-04-20T01:48:38.539 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.016644029000000001, 2024-04-20T01:48:38.539 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.539 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:38.539 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:38.539 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:38.539 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:38.539 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 307 2024-04-20T01:48:38.540 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.540 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.540 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.540 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.540 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823600+0000", 2024-04-20T01:48:38.540 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.540 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.541 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.541 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.541 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823600+0000", 2024-04-20T01:48:38.541 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.541 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.541 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.541 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.541 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823598+0000", 2024-04-20T01:48:38.541 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:38.541 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.541 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.541 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.541 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823606+0000", 2024-04-20T01:48:38.541 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.2629999999999997e-06 2024-04-20T01:48:38.542 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.542 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.542 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.542 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823621+0000", 2024-04-20T01:48:38.542 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5373999999999999e-05 2024-04-20T01:48:38.542 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.542 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.542 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.543 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823635+0000", 2024-04-20T01:48:38.543 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3888e-05 2024-04-20T01:48:38.543 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.543 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.543 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.543 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823683+0000", 2024-04-20T01:48:38.543 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.8455000000000003e-05 2024-04-20T01:48:38.543 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.543 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.543 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:38.543 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823792+0000", 2024-04-20T01:48:38.543 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000108156 2024-04-20T01:48:38.544 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.544 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.544 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.544 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.830048+0000", 2024-04-20T01:48:38.544 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0062565729999999996 2024-04-20T01:48:38.544 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.544 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.544 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.544 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.831286+0000", 2024-04-20T01:48:38.544 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0012377829999999999 2024-04-20T01:48:38.544 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.545 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.545 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:38.545 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.840196+0000", 2024-04-20T01:48:38.545 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0089103129999999996 2024-04-20T01:48:38.545 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.545 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.545 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.545 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.840244+0000", 2024-04-20T01:48:38.545 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.8154e-05 2024-04-20T01:48:38.545 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.545 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.545 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.545 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.546 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.546 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:311 3.8s0 3:1d169b1d:::smithi16579500-9:head [append 0~376832] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:38.546 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.920919+0000", 2024-04-20T01:48:38.546 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 54.627541340000001, 2024-04-20T01:48:38.546 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.009305476, 2024-04-20T01:48:38.546 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.546 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:38.546 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:38.546 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:38.546 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:38.546 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 311 2024-04-20T01:48:38.546 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.546 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.546 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.547 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.547 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.920919+0000", 2024-04-20T01:48:38.547 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.547 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.547 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.547 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.547 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.920919+0000", 2024-04-20T01:48:38.547 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.547 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.547 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.547 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.547 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.920918+0000", 2024-04-20T01:48:38.547 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:38.548 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.548 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.548 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.548 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921528+0000", 2024-04-20T01:48:38.548 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00060985100000000001 2024-04-20T01:48:38.548 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.548 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.548 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.548 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921543+0000", 2024-04-20T01:48:38.548 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4302e-05 2024-04-20T01:48:38.548 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.548 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.548 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.548 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921557+0000", 2024-04-20T01:48:38.548 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4416999999999999e-05 2024-04-20T01:48:38.549 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.549 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.549 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.549 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921979+0000", 2024-04-20T01:48:38.549 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00042189099999999998 2024-04-20T01:48:38.549 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.549 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.549 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:38.549 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.922074+0000", 2024-04-20T01:48:38.549 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4798999999999998e-05 2024-04-20T01:48:38.549 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.549 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.549 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.549 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.922634+0000", 2024-04-20T01:48:38.550 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00056005300000000003 2024-04-20T01:48:38.550 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.550 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.550 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.550 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923907+0000", 2024-04-20T01:48:38.550 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0012733239999999999 2024-04-20T01:48:38.550 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.550 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.550 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:38.550 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930149+0000", 2024-04-20T01:48:38.550 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00624202 2024-04-20T01:48:38.550 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.550 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.551 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.551 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930225+0000", 2024-04-20T01:48:38.551 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.5860999999999998e-05 2024-04-20T01:48:38.551 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.551 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.551 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.551 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.551 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.551 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:313 3.8s0 3:1d169b1d:::smithi16579500-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:38.551 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.921593+0000", 2024-04-20T01:48:38.551 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 54.626867761, 2024-04-20T01:48:38.551 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0095502929999999996, 2024-04-20T01:48:38.551 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.551 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:38.551 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:38.552 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:38.552 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:38.552 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 313 2024-04-20T01:48:38.552 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.552 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.552 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.552 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.552 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921593+0000", 2024-04-20T01:48:38.552 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.552 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.552 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.552 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.552 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921593+0000", 2024-04-20T01:48:38.552 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.553 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.553 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.553 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.553 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921593+0000", 2024-04-20T01:48:38.553 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:38.553 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.553 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.553 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.553 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921596+0000", 2024-04-20T01:48:38.553 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.089e-06 2024-04-20T01:48:38.553 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.553 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.553 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.553 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921601+0000", 2024-04-20T01:48:38.553 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.8799999999999999e-06 2024-04-20T01:48:38.554 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.554 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.554 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.554 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921604+0000", 2024-04-20T01:48:38.554 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.36e-06 2024-04-20T01:48:38.554 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.554 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.554 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.554 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923831+0000", 2024-04-20T01:48:38.554 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0022266479999999999 2024-04-20T01:48:38.554 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.554 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.555 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:38.555 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923898+0000", 2024-04-20T01:48:38.555 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.7391000000000006e-05 2024-04-20T01:48:38.555 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.555 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.555 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.555 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930413+0000", 2024-04-20T01:48:38.555 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.006514382 2024-04-20T01:48:38.555 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.555 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.555 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:38.555 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930468+0000", 2024-04-20T01:48:38.555 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.5923e-05 2024-04-20T01:48:38.555 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.555 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.556 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.556 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.931143+0000", 2024-04-20T01:48:38.556 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00067489999999999998 2024-04-20T01:48:38.556 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.556 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.556 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.556 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.556 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.556 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:314 3.8s0 3:1746945e:::smithi16579500-24:head [copy-from ver 4] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:38.556 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.921607+0000", 2024-04-20T01:48:38.556 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 54.626854166000001, 2024-04-20T01:48:38.556 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013830053, 2024-04-20T01:48:38.556 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.556 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:38.556 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:38.557 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:38.557 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:38.557 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 314 2024-04-20T01:48:38.557 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.557 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.557 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.557 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.557 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921607+0000", 2024-04-20T01:48:38.557 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.557 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.557 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.557 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.557 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921607+0000", 2024-04-20T01:48:38.557 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.557 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.558 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.558 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.558 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921606+0000", 2024-04-20T01:48:38.558 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:38.558 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.558 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.558 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.558 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921609+0000", 2024-04-20T01:48:38.558 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0999999999999998e-06 2024-04-20T01:48:38.558 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.558 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.558 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.558 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921612+0000", 2024-04-20T01:48:38.558 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.4089999999999999e-06 2024-04-20T01:48:38.559 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.559 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.559 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.559 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921614+0000", 2024-04-20T01:48:38.559 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5610000000000001e-06 2024-04-20T01:48:38.559 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.559 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.559 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.559 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923919+0000", 2024-04-20T01:48:38.559 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0023049809999999998 2024-04-20T01:48:38.559 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.559 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.559 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:38.559 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923973+0000", 2024-04-20T01:48:38.560 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3532999999999997e-05 2024-04-20T01:48:38.560 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.560 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.560 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.560 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.928709+0000", 2024-04-20T01:48:38.560 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0047359979999999999 2024-04-20T01:48:38.560 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.560 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.560 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.560 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930393+0000", 2024-04-20T01:48:38.560 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0016838459999999999 2024-04-20T01:48:38.560 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.560 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.560 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:38.561 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.935394+0000", 2024-04-20T01:48:38.561 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0050015779999999996 2024-04-20T01:48:38.561 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.561 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.561 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.561 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.935437+0000", 2024-04-20T01:48:38.561 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2289000000000001e-05 2024-04-20T01:48:38.561 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.561 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.561 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.561 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.561 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.561 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:336 3.8s0 3:1746945e:::smithi16579500-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:38.561 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.973134+0000", 2024-04-20T01:48:38.562 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 54.575327197999997, 2024-04-20T01:48:38.562 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010536856000000001, 2024-04-20T01:48:38.562 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.562 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:38.562 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:38.562 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:38.562 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:38.562 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 336 2024-04-20T01:48:38.562 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.562 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.562 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.562 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.562 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973134+0000", 2024-04-20T01:48:38.562 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.563 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.563 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.563 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.563 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973134+0000", 2024-04-20T01:48:38.563 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.563 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.563 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.563 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.563 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973133+0000", 2024-04-20T01:48:38.563 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:38.563 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.563 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.563 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.563 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973135+0000", 2024-04-20T01:48:38.563 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5269999999999999e-06 2024-04-20T01:48:38.564 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.564 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.564 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.564 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973139+0000", 2024-04-20T01:48:38.564 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.3749999999999996e-06 2024-04-20T01:48:38.564 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.564 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.564 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.564 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973142+0000", 2024-04-20T01:48:38.564 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7800000000000001e-06 2024-04-20T01:48:38.564 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.564 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.564 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.564 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.978050+0000", 2024-04-20T01:48:38.564 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0049085049999999996 2024-04-20T01:48:38.565 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.565 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.565 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:38.565 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.978106+0000", 2024-04-20T01:48:38.565 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.5648e-05 2024-04-20T01:48:38.565 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.565 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.565 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.565 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.980649+0000", 2024-04-20T01:48:38.565 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0025426630000000001 2024-04-20T01:48:38.565 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.565 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.565 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:38.565 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.980701+0000", 2024-04-20T01:48:38.565 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.1907999999999998e-05 2024-04-20T01:48:38.566 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.566 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.566 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.566 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.983670+0000", 2024-04-20T01:48:38.566 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0029697590000000002 2024-04-20T01:48:38.566 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.566 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.566 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.566 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.566 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.566 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:487 3.8s0 3:16aac79f:::smithi16579500-14:head [append 0~425984] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:38.566 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:59.760614+0000", 2024-04-20T01:48:38.567 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 38.787847182, 2024-04-20T01:48:38.567 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010465621, 2024-04-20T01:48:38.567 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.567 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:38.567 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:38.567 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:38.567 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:38.567 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 487 2024-04-20T01:48:38.567 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.567 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.567 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.567 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.567 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.760614+0000", 2024-04-20T01:48:38.567 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.568 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.568 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.568 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.568 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.760614+0000", 2024-04-20T01:48:38.568 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.568 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.568 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.568 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.568 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.760611+0000", 2024-04-20T01:48:38.568 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999971 2024-04-20T01:48:38.568 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.568 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.568 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.568 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.761151+0000", 2024-04-20T01:48:38.568 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00054006400000000004 2024-04-20T01:48:38.569 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.569 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.569 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.569 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.761173+0000", 2024-04-20T01:48:38.569 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1926999999999999e-05 2024-04-20T01:48:38.569 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.569 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.569 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.569 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.761202+0000", 2024-04-20T01:48:38.569 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9286e-05 2024-04-20T01:48:38.569 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.569 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.569 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.569 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.761276+0000", 2024-04-20T01:48:38.570 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.4319000000000005e-05 2024-04-20T01:48:38.570 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.570 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.570 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op must wait for map", 2024-04-20T01:48:38.570 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.761289+0000", 2024-04-20T01:48:38.570 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.239e-05 2024-04-20T01:48:38.570 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.570 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.570 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.570 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.763247+0000", 2024-04-20T01:48:38.570 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0019584989999999998 2024-04-20T01:48:38.570 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.570 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.571 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:38.571 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.763331+0000", 2024-04-20T01:48:38.571 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.3320999999999997e-05 2024-04-20T01:48:38.571 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.571 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.571 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.571 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.764297+0000", 2024-04-20T01:48:38.571 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00096590700000000001 2024-04-20T01:48:38.571 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.571 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.571 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.571 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768005+0000", 2024-04-20T01:48:38.571 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0037084940000000001 2024-04-20T01:48:38.571 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.571 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.572 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:38.572 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.770983+0000", 2024-04-20T01:48:38.572 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0029780129999999998 2024-04-20T01:48:38.572 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.572 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.572 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.572 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.771079+0000", 2024-04-20T01:48:38.572 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6250999999999997e-05 2024-04-20T01:48:38.572 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.572 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.572 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.572 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.572 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.572 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:489 3.8s0 3:16aac79f:::smithi16579500-14:head [append 0~376832] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:38.573 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:59.767481+0000", 2024-04-20T01:48:38.573 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 38.780979510999998, 2024-04-20T01:48:38.573 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011576091, 2024-04-20T01:48:38.573 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.573 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:38.573 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:38.573 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:38.573 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:38.573 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 489 2024-04-20T01:48:38.573 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.573 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.573 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.573 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.573 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.767481+0000", 2024-04-20T01:48:38.573 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.574 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.574 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.574 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.574 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.767481+0000", 2024-04-20T01:48:38.574 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.574 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.574 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.574 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.574 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.767480+0000", 2024-04-20T01:48:38.574 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:38.574 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.574 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.574 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.574 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768144+0000", 2024-04-20T01:48:38.574 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00066418499999999997 2024-04-20T01:48:38.575 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.575 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.575 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.575 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768157+0000", 2024-04-20T01:48:38.575 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2281e-05 2024-04-20T01:48:38.575 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.575 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.575 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.575 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768166+0000", 2024-04-20T01:48:38.575 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2629999999999992e-06 2024-04-20T01:48:38.575 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.575 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.575 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.575 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768215+0000", 2024-04-20T01:48:38.575 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.9292000000000002e-05 2024-04-20T01:48:38.576 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.576 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.576 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:38.576 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768322+0000", 2024-04-20T01:48:38.576 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000106461 2024-04-20T01:48:38.576 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.576 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.576 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.576 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.769186+0000", 2024-04-20T01:48:38.576 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00086420900000000001 2024-04-20T01:48:38.576 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.576 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.576 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.576 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.771140+0000", 2024-04-20T01:48:38.577 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0019545890000000001 2024-04-20T01:48:38.577 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.577 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.577 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:38.577 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779011+0000", 2024-04-20T01:48:38.577 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0078710150000000003 2024-04-20T01:48:38.577 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.577 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.577 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.577 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779057+0000", 2024-04-20T01:48:38.577 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5918000000000003e-05 2024-04-20T01:48:38.577 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.577 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.577 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.577 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.578 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.578 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:490 3.8s0 3:16aac79f:::smithi16579500-14:head [setxattr _header (57) in=7b] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:38.578 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:59.768179+0000", 2024-04-20T01:48:38.578 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 38.780281617999997, 2024-04-20T01:48:38.578 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011076167, 2024-04-20T01:48:38.578 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.578 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:38.578 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:38.578 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:38.578 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:38.578 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 490 2024-04-20T01:48:38.578 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.578 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.578 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.579 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.579 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768179+0000", 2024-04-20T01:48:38.579 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.579 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.579 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.579 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.579 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768179+0000", 2024-04-20T01:48:38.579 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.579 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.579 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.579 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.579 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768178+0000", 2024-04-20T01:48:38.579 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:38.579 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.579 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.580 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.580 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768183+0000", 2024-04-20T01:48:38.580 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.288e-06 2024-04-20T01:48:38.580 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.580 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.580 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.580 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768191+0000", 2024-04-20T01:48:38.580 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.5850000000000002e-06 2024-04-20T01:48:38.580 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.580 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.580 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.580 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768197+0000", 2024-04-20T01:48:38.580 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.9270000000000002e-06 2024-04-20T01:48:38.580 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.581 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.581 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.581 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.769809+0000", 2024-04-20T01:48:38.581 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001611913 2024-04-20T01:48:38.581 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.581 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.581 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:38.581 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.769930+0000", 2024-04-20T01:48:38.581 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000121304 2024-04-20T01:48:38.581 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.581 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.581 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.581 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.770462+0000", 2024-04-20T01:48:38.581 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00053196700000000005 2024-04-20T01:48:38.582 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.582 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.582 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.582 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.771310+0000", 2024-04-20T01:48:38.582 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00084813200000000003 2024-04-20T01:48:38.582 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.582 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.582 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:38.582 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779156+0000", 2024-04-20T01:48:38.582 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0078461580000000006 2024-04-20T01:48:38.582 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.582 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.582 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.582 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779255+0000", 2024-04-20T01:48:38.583 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.8944999999999995e-05 2024-04-20T01:48:38.583 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.583 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.583 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.583 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.583 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.583 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:491 3.8s0 3:16aac79f:::smithi16579500-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:38.583 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:59.768204+0000", 2024-04-20T01:48:38.583 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 38.780256528999999, 2024-04-20T01:48:38.583 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.012786562, 2024-04-20T01:48:38.583 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.583 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:38.583 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:38.583 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:38.584 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:38.584 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 491 2024-04-20T01:48:38.584 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.584 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.584 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.584 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.584 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768204+0000", 2024-04-20T01:48:38.584 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.584 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.584 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.584 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.584 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768204+0000", 2024-04-20T01:48:38.584 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.584 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.585 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.585 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.585 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768204+0000", 2024-04-20T01:48:38.585 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:38.585 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.585 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.585 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.585 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768207+0000", 2024-04-20T01:48:38.585 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.4180000000000001e-06 2024-04-20T01:48:38.585 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.585 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.585 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.585 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768215+0000", 2024-04-20T01:48:38.585 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.6669999999999996e-06 2024-04-20T01:48:38.585 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.586 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.586 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.586 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768220+0000", 2024-04-20T01:48:38.586 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3149999999999999e-06 2024-04-20T01:48:38.586 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.586 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.586 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.586 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.771203+0000", 2024-04-20T01:48:38.586 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.002982434 2024-04-20T01:48:38.586 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.586 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.586 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:38.586 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.771299+0000", 2024-04-20T01:48:38.586 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6111000000000002e-05 2024-04-20T01:48:38.586 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.587 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.587 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.587 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779629+0000", 2024-04-20T01:48:38.587 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0083302529999999993 2024-04-20T01:48:38.587 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.587 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.587 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:38.587 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779693+0000", 2024-04-20T01:48:38.587 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.4424999999999994e-05 2024-04-20T01:48:38.587 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.587 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.587 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.587 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.780991+0000", 2024-04-20T01:48:38.587 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0012974390000000001 2024-04-20T01:48:38.588 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.588 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.588 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.588 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.588 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.588 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s4 75/30 ECSubWrite(tid=55, reqid=client.4643.0:781, at_version=75'41, trim_to=0'0, roll_forward_to=66'39))", 2024-04-20T01:48:38.588 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.735057+0000", 2024-04-20T01:48:38.588 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.813403745999999, 2024-04-20T01:48:38.588 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.012883867, 2024-04-20T01:48:38.588 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.588 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.588 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.588 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.588 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.588 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735057+0000", 2024-04-20T01:48:38.589 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.589 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.589 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.589 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.589 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735057+0000", 2024-04-20T01:48:38.589 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.589 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.589 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.589 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.589 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735056+0000", 2024-04-20T01:48:38.589 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:38.589 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.589 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.589 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.590 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735367+0000", 2024-04-20T01:48:38.590 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00031039199999999999 2024-04-20T01:48:38.590 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.590 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.590 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.590 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735378+0000", 2024-04-20T01:48:38.590 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1389000000000001e-05 2024-04-20T01:48:38.590 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.590 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.590 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.591 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735385+0000", 2024-04-20T01:48:38.591 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.9759999999999998e-06 2024-04-20T01:48:38.591 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.591 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.591 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.591 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735429+0000", 2024-04-20T01:48:38.591 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.3699000000000003e-05 2024-04-20T01:48:38.591 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.591 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.591 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.591 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735437+0000", 2024-04-20T01:48:38.591 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.7440000000000007e-06 2024-04-20T01:48:38.592 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.592 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.592 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.592 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747914+0000", 2024-04-20T01:48:38.592 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.012477162 2024-04-20T01:48:38.592 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.592 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.592 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.592 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747941+0000", 2024-04-20T01:48:38.592 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7107e-05 2024-04-20T01:48:38.592 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.592 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.592 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.593 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.593 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.593 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.1s3 75/30 ECSubWrite(tid=257, reqid=osd.4.0:253, at_version=0'0, trim_to=0'0, roll_forward_to=75'37))", 2024-04-20T01:48:38.593 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.737287+0000", 2024-04-20T01:48:38.593 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.811173362000002, 2024-04-20T01:48:38.593 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00931126, 2024-04-20T01:48:38.593 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.593 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.593 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.593 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.593 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.593 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737287+0000", 2024-04-20T01:48:38.593 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.593 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.594 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.594 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.594 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737287+0000", 2024-04-20T01:48:38.594 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.594 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.594 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.594 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.594 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737287+0000", 2024-04-20T01:48:38.594 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:38.594 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.594 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.594 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.594 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737307+0000", 2024-04-20T01:48:38.594 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0547000000000002e-05 2024-04-20T01:48:38.594 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.595 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.595 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.595 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737320+0000", 2024-04-20T01:48:38.595 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3112000000000001e-05 2024-04-20T01:48:38.595 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.595 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.595 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.595 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737339+0000", 2024-04-20T01:48:38.595 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8289999999999999e-05 2024-04-20T01:48:38.595 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.595 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.595 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.595 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737420+0000", 2024-04-20T01:48:38.595 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.106e-05 2024-04-20T01:48:38.596 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.596 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.596 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.596 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737431+0000", 2024-04-20T01:48:38.596 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1289e-05 2024-04-20T01:48:38.596 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.596 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.596 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.596 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.746568+0000", 2024-04-20T01:48:38.596 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0091374909999999993 2024-04-20T01:48:38.596 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.596 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.596 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.597 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.746599+0000", 2024-04-20T01:48:38.597 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0335999999999999e-05 2024-04-20T01:48:38.597 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.597 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.597 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.597 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.597 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.597 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.cs6 75/30 ECSubWrite(tid=114, reqid=osd.14.0:113, at_version=0'0, trim_to=0'0, roll_forward_to=75'93))", 2024-04-20T01:48:38.597 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.738324+0000", 2024-04-20T01:48:38.597 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.810136332999999, 2024-04-20T01:48:38.597 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0095662660000000004, 2024-04-20T01:48:38.597 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.597 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.597 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.598 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.598 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.598 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738324+0000", 2024-04-20T01:48:38.598 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.598 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.598 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.598 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.598 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738324+0000", 2024-04-20T01:48:38.598 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.598 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.598 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.598 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.599 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738324+0000", 2024-04-20T01:48:38.599 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:38.599 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.599 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.599 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.599 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738337+0000", 2024-04-20T01:48:38.599 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3596e-05 2024-04-20T01:48:38.599 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.599 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.599 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.599 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738349+0000", 2024-04-20T01:48:38.599 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1857e-05 2024-04-20T01:48:38.599 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.599 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.599 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.600 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738358+0000", 2024-04-20T01:48:38.600 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4060000000000005e-06 2024-04-20T01:48:38.600 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.600 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.600 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.600 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738425+0000", 2024-04-20T01:48:38.600 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.7916000000000005e-05 2024-04-20T01:48:38.600 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.600 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.600 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.600 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738442+0000", 2024-04-20T01:48:38.600 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6872999999999999e-05 2024-04-20T01:48:38.600 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.600 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.600 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.601 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747865+0000", 2024-04-20T01:48:38.601 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0094223810000000005 2024-04-20T01:48:38.601 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.601 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.601 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.601 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747891+0000", 2024-04-20T01:48:38.601 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6061999999999999e-05 2024-04-20T01:48:38.601 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.601 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.601 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.601 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.601 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.601 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s5 75/30 ECSubWrite(tid=258, reqid=osd.4.0:258, at_version=75'21, trim_to=0'0, roll_forward_to=68'19))", 2024-04-20T01:48:38.601 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.738907+0000", 2024-04-20T01:48:38.602 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.809553545, 2024-04-20T01:48:38.602 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010750793, 2024-04-20T01:48:38.602 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.602 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.602 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.602 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.602 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.602 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738907+0000", 2024-04-20T01:48:38.602 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.602 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.602 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.602 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.602 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738907+0000", 2024-04-20T01:48:38.603 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.603 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.603 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.603 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.603 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738907+0000", 2024-04-20T01:48:38.603 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:38.603 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.603 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.603 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.603 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738926+0000", 2024-04-20T01:48:38.603 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9032e-05 2024-04-20T01:48:38.603 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.603 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.604 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.604 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738938+0000", 2024-04-20T01:48:38.604 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2802999999999999e-05 2024-04-20T01:48:38.604 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.604 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.604 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.604 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738948+0000", 2024-04-20T01:48:38.604 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.0650000000000005e-06 2024-04-20T01:48:38.604 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.604 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.604 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.604 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739017+0000", 2024-04-20T01:48:38.604 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.9621000000000003e-05 2024-04-20T01:48:38.604 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.604 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.605 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.605 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739028+0000", 2024-04-20T01:48:38.605 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0957e-05 2024-04-20T01:48:38.605 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.605 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.605 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.605 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.749625+0000", 2024-04-20T01:48:38.605 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010597031999999999 2024-04-20T01:48:38.605 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.605 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.605 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.605 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.749658+0000", 2024-04-20T01:48:38.605 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.2947999999999997e-05 2024-04-20T01:48:38.605 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.605 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.606 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.606 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.606 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.606 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.cs6 75/30 ECSubWrite(tid=115, reqid=osd.14.0:115, at_version=75'95, trim_to=0'0, roll_forward_to=75'93))", 2024-04-20T01:48:38.606 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.739080+0000", 2024-04-20T01:48:38.606 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.809381301999998, 2024-04-20T01:48:38.606 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011132998, 2024-04-20T01:48:38.606 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.606 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.606 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.606 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.606 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.606 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739080+0000", 2024-04-20T01:48:38.606 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.607 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.607 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.607 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.607 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739080+0000", 2024-04-20T01:48:38.607 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.607 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.607 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.607 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.607 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739079+0000", 2024-04-20T01:48:38.607 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:38.607 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.607 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.607 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.607 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739096+0000", 2024-04-20T01:48:38.607 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7300999999999999e-05 2024-04-20T01:48:38.608 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.608 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.608 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.608 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739123+0000", 2024-04-20T01:48:38.608 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7257999999999999e-05 2024-04-20T01:48:38.608 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.608 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.608 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.608 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739132+0000", 2024-04-20T01:48:38.608 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8859999999999993e-06 2024-04-20T01:48:38.608 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.608 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.608 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.608 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739598+0000", 2024-04-20T01:48:38.608 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00046589800000000002 2024-04-20T01:48:38.609 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.609 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.609 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.609 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739609+0000", 2024-04-20T01:48:38.609 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0737000000000001e-05 2024-04-20T01:48:38.609 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.609 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.609 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.609 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.750185+0000", 2024-04-20T01:48:38.609 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010576370999999999 2024-04-20T01:48:38.609 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.609 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.609 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.609 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.750213+0000", 2024-04-20T01:48:38.610 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7304000000000002e-05 2024-04-20T01:48:38.610 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.610 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.610 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.610 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.610 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.610 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s5 75/30 ECSubWrite(tid=259, reqid=osd.4.0:259, at_version=75'23, trim_to=0'0, roll_forward_to=68'19))", 2024-04-20T01:48:38.610 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.740102+0000", 2024-04-20T01:48:38.610 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 21.808359063000001, 2024-04-20T01:48:38.610 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011279176, 2024-04-20T01:48:38.610 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.610 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.611 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.611 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.611 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.611 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740102+0000", 2024-04-20T01:48:38.611 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.611 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.611 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.611 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.611 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740102+0000", 2024-04-20T01:48:38.611 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.611 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.611 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.611 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.611 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740101+0000", 2024-04-20T01:48:38.612 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:38.612 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.612 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.612 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.612 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740123+0000", 2024-04-20T01:48:38.612 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1708e-05 2024-04-20T01:48:38.612 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.612 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.612 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.612 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740135+0000", 2024-04-20T01:48:38.612 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1776999999999999e-05 2024-04-20T01:48:38.612 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.612 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.612 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.613 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740142+0000", 2024-04-20T01:48:38.613 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.956e-06 2024-04-20T01:48:38.613 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.613 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.613 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.613 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742932+0000", 2024-04-20T01:48:38.613 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0027900440000000002 2024-04-20T01:48:38.613 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.613 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.613 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.613 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742942+0000", 2024-04-20T01:48:38.613 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0491e-05 2024-04-20T01:48:38.613 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.613 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.614 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.614 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.751351+0000", 2024-04-20T01:48:38.614 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0084087229999999999 2024-04-20T01:48:38.614 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.614 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.614 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.614 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.751381+0000", 2024-04-20T01:48:38.614 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0173999999999999e-05 2024-04-20T01:48:38.614 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.614 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.614 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.614 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.614 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.614 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s5 80/30 ECSubWrite(tid=309, reqid=osd.4.0:309, at_version=80'32, trim_to=0'0, roll_forward_to=79'28))", 2024-04-20T01:48:38.615 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:21.742879+0000", 2024-04-20T01:48:38.615 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 16.805582287, 2024-04-20T01:48:38.615 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0092419359999999992, 2024-04-20T01:48:38.615 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.615 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:38.615 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.615 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.615 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.615 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.742879+0000", 2024-04-20T01:48:38.615 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.615 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.615 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.615 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.615 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.742879+0000", 2024-04-20T01:48:38.615 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.616 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.616 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.616 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.616 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.742878+0000", 2024-04-20T01:48:38.616 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:38.616 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.616 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.616 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.616 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.742903+0000", 2024-04-20T01:48:38.616 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5403000000000001e-05 2024-04-20T01:48:38.616 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.616 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.616 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.616 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.742913+0000", 2024-04-20T01:48:38.616 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6390000000000004e-06 2024-04-20T01:48:38.617 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.617 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.617 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.617 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.742919+0000", 2024-04-20T01:48:38.617 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.1770000000000003e-06 2024-04-20T01:48:38.617 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.617 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.617 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.617 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744284+0000", 2024-04-20T01:48:38.617 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0013651189999999999 2024-04-20T01:48:38.617 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.617 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.617 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:38.617 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744295+0000", 2024-04-20T01:48:38.617 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0796e-05 2024-04-20T01:48:38.618 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.618 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.618 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:38.618 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.752095+0000", 2024-04-20T01:48:38.618 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0077997589999999999 2024-04-20T01:48:38.618 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.618 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.618 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.618 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.752120+0000", 2024-04-20T01:48:38.618 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5584000000000001e-05 2024-04-20T01:48:38.618 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.618 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.618 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.618 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.619 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.619 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:998 3.8s0 3:16aac79f:::smithi16579500-14:head [read 0~1982464,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:38.619 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:31.792901+0000", 2024-04-20T01:48:38.619 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 6.7555596199999997, 2024-04-20T01:48:38.619 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.94456701399999998, 2024-04-20T01:48:38.619 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.619 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:38.619 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:38.619 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:38.619 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:38.619 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 998 2024-04-20T01:48:38.619 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.619 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.619 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.620 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.620 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792901+0000", 2024-04-20T01:48:38.620 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.620 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.620 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.620 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.620 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792901+0000", 2024-04-20T01:48:38.620 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.620 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.620 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.620 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.620 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792900+0000", 2024-04-20T01:48:38.620 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:38.620 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.620 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.621 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.621 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792908+0000", 2024-04-20T01:48:38.621 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.2949999999999994e-06 2024-04-20T01:48:38.621 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.621 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.621 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.621 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792921+0000", 2024-04-20T01:48:38.621 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2738999999999999e-05 2024-04-20T01:48:38.621 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.621 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.621 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.621 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792931+0000", 2024-04-20T01:48:38.621 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0781000000000001e-05 2024-04-20T01:48:38.621 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.621 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.622 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.622 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.793020+0000", 2024-04-20T01:48:38.622 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8127000000000005e-05 2024-04-20T01:48:38.622 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.622 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.622 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for peered", 2024-04-20T01:48:38.622 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.793023+0000", 2024-04-20T01:48:38.622 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.5870000000000001e-06 2024-04-20T01:48:38.622 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.622 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.622 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.622 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.729818+0000", 2024-04-20T01:48:38.622 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.93679480800000003 2024-04-20T01:48:38.622 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.623 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.623 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:38.623 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.730070+0000", 2024-04-20T01:48:38.623 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00025204900000000001 2024-04-20T01:48:38.623 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.623 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.623 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.623 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.737468+0000", 2024-04-20T01:48:38.623 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0073981969999999996 2024-04-20T01:48:38.623 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.623 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.623 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.623 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.623 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.624 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:999 3.8s0 3:16aac79f:::smithi16579500-14:head [read 0~1982464,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:38.624 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:31.792943+0000", 2024-04-20T01:48:38.624 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 6.7555173850000001, 2024-04-20T01:48:38.624 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.94611857099999996, 2024-04-20T01:48:38.624 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.624 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:38.624 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:38.624 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:38.624 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:38.624 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 999 2024-04-20T01:48:38.624 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.624 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.624 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.624 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.625 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792943+0000", 2024-04-20T01:48:38.625 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.625 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.625 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.625 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.625 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792943+0000", 2024-04-20T01:48:38.625 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.625 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.625 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.625 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.625 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792943+0000", 2024-04-20T01:48:38.625 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:38.625 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.625 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.625 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.626 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792947+0000", 2024-04-20T01:48:38.626 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7290000000000002e-06 2024-04-20T01:48:38.626 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.626 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.626 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.626 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792953+0000", 2024-04-20T01:48:38.626 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.8029999999999999e-06 2024-04-20T01:48:38.626 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.626 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.626 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.626 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792959+0000", 2024-04-20T01:48:38.626 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.614e-06 2024-04-20T01:48:38.626 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.626 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.627 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.627 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.793056+0000", 2024-04-20T01:48:38.627 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7196999999999998e-05 2024-04-20T01:48:38.627 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.627 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.627 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for peered", 2024-04-20T01:48:38.627 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.793058+0000", 2024-04-20T01:48:38.627 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.967e-06 2024-04-20T01:48:38.627 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.627 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.627 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.627 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.732554+0000", 2024-04-20T01:48:38.627 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.93949528900000001 2024-04-20T01:48:38.627 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.627 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.628 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:38.628 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.732656+0000", 2024-04-20T01:48:38.628 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000102125 2024-04-20T01:48:38.628 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.628 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.628 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.628 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.739062+0000", 2024-04-20T01:48:38.628 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0064063519999999997 2024-04-20T01:48:38.628 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.628 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.628 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.628 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.628 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.628 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:1000 3.8s0 3:16aac79f:::smithi16579500-14:head [read 0~1982464,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:38.629 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:31.792964+0000", 2024-04-20T01:48:38.629 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 6.7554970399999998, 2024-04-20T01:48:38.629 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.94733045199999999, 2024-04-20T01:48:38.629 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.629 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:38.629 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:38.629 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:38.629 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:38.629 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 1000 2024-04-20T01:48:38.629 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.629 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.629 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.629 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.629 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792964+0000", 2024-04-20T01:48:38.629 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.630 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.630 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.630 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.630 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792964+0000", 2024-04-20T01:48:38.630 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.630 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.630 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.630 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.630 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792963+0000", 2024-04-20T01:48:38.630 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:38.630 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.630 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.630 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.630 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792974+0000", 2024-04-20T01:48:38.631 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1066999999999999e-05 2024-04-20T01:48:38.631 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.631 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.631 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.631 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792981+0000", 2024-04-20T01:48:38.631 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.3940000000000001e-06 2024-04-20T01:48:38.631 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.631 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.631 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.631 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792986+0000", 2024-04-20T01:48:38.631 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6169999999999998e-06 2024-04-20T01:48:38.631 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.631 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.631 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.631 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.793087+0000", 2024-04-20T01:48:38.632 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00010083 2024-04-20T01:48:38.632 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.632 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.632 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for peered", 2024-04-20T01:48:38.632 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.793088+0000", 2024-04-20T01:48:38.632 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2299999999999999e-07 2024-04-20T01:48:38.632 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.632 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.632 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.632 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.733410+0000", 2024-04-20T01:48:38.632 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.94032182600000003 2024-04-20T01:48:38.632 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.632 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.632 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:38.632 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.733498+0000", 2024-04-20T01:48:38.633 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.7819999999999996e-05 2024-04-20T01:48:38.633 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.633 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.633 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.633 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.740294+0000", 2024-04-20T01:48:38.633 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0067963729999999997 2024-04-20T01:48:38.633 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.633 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.633 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.633 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.633 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.633 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:1011 3.8s0 3:1746945e:::smithi16579500-24:head [rollback 31] snapc 38=[35,31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-04-20T01:48:38.633 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:32.727422+0000", 2024-04-20T01:48:38.633 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 5.8210392750000004, 2024-04-20T01:48:38.634 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.014864550000000001, 2024-04-20T01:48:38.634 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:38.634 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:38.634 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:38.634 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:38.634 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:38.634 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 1011 2024-04-20T01:48:38.634 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.634 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:38.634 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.634 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:38.634 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.727422+0000", 2024-04-20T01:48:38.634 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.634 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.635 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.635 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:38.635 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.727422+0000", 2024-04-20T01:48:38.635 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:38.635 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.635 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.635 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:38.635 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.727420+0000", 2024-04-20T01:48:38.635 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:38.635 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.635 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.635 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:38.635 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.727426+0000", 2024-04-20T01:48:38.635 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.494e-06 2024-04-20T01:48:38.635 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.636 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.636 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:38.636 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.727442+0000", 2024-04-20T01:48:38.636 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5877999999999999e-05 2024-04-20T01:48:38.636 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.636 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.636 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:38.636 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.727453+0000", 2024-04-20T01:48:38.636 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0839000000000001e-05 2024-04-20T01:48:38.636 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.636 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.636 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.637 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.727512+0000", 2024-04-20T01:48:38.637 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.9338e-05 2024-04-20T01:48:38.637 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.637 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.637 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for peered", 2024-04-20T01:48:38.637 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.727516+0000", 2024-04-20T01:48:38.637 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7850000000000002e-06 2024-04-20T01:48:38.637 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.637 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.637 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:38.637 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.733732+0000", 2024-04-20T01:48:38.637 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0062156850000000003 2024-04-20T01:48:38.637 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.637 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.637 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:38.638 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.733867+0000", 2024-04-20T01:48:38.638 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00013472399999999999 2024-04-20T01:48:38.638 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:38.638 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:38.638 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:38.638 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.742286+0000", 2024-04-20T01:48:38.638 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0084195599999999995 2024-04-20T01:48:38.638 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.638 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.638 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.638 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:38.638 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:38.638 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:38.640 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:48:38.640 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:48:38.641 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:48:38.641 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:48:38.641 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:48:38.641 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:38.654+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:38.641 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-20T01:48:38.642 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:48:38.663 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:48:38.683 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:48:38.704 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:48:38.724 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:48:38.742 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:38.757+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:38.744 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:48:38.765 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:48:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:48:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:48:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:48:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:48:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:48:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:48:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:48:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:48:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:48:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:48:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:48:38.766 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:48:38.767 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:48:38.767 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:48:38.767 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.14 2024-04-20T01:48:38.767 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:48:38.768 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-20T01:48:38.769 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-20T01:48:38.771 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:38.771 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:38.782 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-20T01:48:38.794 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is unset 2024-04-20T01:48:38.848 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:38.863+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:38.910 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:38.910 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:38.922 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-20T01:48:38.946 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:38.961+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:39.047 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:39.061+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:39.049 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:39.049 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:39.059 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-20T01:48:39.150 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:39.164+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:39.188 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:39.188 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:39.198 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-20T01:48:39.251 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:39.265+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:39.325 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:39.325 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:39.335 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-20T01:48:39.351 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:39.366+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:39.452 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:39.467+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:39.503 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:39.503 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:39.513 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-20T01:48:39.553 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:39.567+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:39.653 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:39.668+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:39.661 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:39.661 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:39.672 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-20T01:48:39.755 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:39.769+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:39.801 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:39.801 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:39.812 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-20T01:48:39.856 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:39.871+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:39.941 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:39.941 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:39.951 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-20T01:48:39.956 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:39.971+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:40.057 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:40.071+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:40.119 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:40.119 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:40.129 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-20T01:48:40.158 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:40.172+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:40.256 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:40.256 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:40.259 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:40.273+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:40.267 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-20T01:48:40.359 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:40.374+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:40.394 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:40.394 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:40.405 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-20T01:48:40.460 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:40.475+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:40.533 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:40.533 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:40.543 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-20T01:48:40.561 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:40.576+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:40.662 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:40.677+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:40.671 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:40.671 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:40.682 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-20T01:48:40.767 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:40.782+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:40.770 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-20T01:48:40.807 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-20T01:48:40.851 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:40.851 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:40.861 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-20T01:48:40.865 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:40.879+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:40.943 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:48:40.965 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:40.980+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:40.966 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:48:41.004 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:48:41.005 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:41.005 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:41.016 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-20T01:48:41.025 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:48:41.045 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:48:41.066 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:48:41.066 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:41.080+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:41.086 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:48:41.106 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:48:41.126 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:48:41.144 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:41.144 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:41.144 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:48:41.144 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:48:41.144 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:41.146 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:48:41.155 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-20T01:48:41.167 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:48:41.187 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:48:41.199 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:41.213+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:41.207 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:48:41.228 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:48:41.249 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:48:41.249 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:48:41.250 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:48:41.250 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:48:41.250 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:48:41.250 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:48:41.250 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:48:41.250 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:48:41.250 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:48:41.250 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:48:41.250 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:48:41.250 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:48:41.250 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:48:41.250 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:48:41.250 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:48:41.251 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.14 2024-04-20T01:48:41.251 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:48:41.252 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-20T01:48:41.253 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-20T01:48:41.267 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:41.281+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:41.287 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:41.287 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:48:41.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:48:41.287 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:48:41.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.287 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:234 3.0s0 3:0c1ed9f1:::smithi16579500-46:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:41.287 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.787453+0000", 2024-04-20T01:48:41.287 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.513065265000002, 2024-04-20T01:48:41.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016795529, 2024-04-20T01:48:41.287 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.287 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:41.287 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:41.288 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:41.288 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:41.288 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 234 2024-04-20T01:48:41.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.288 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787453+0000", 2024-04-20T01:48:41.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787453+0000", 2024-04-20T01:48:41.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787451+0000", 2024-04-20T01:48:41.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:41.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787462+0000", 2024-04-20T01:48:41.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1083e-05 2024-04-20T01:48:41.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787483+0000", 2024-04-20T01:48:41.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1656999999999999e-05 2024-04-20T01:48:41.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787503+0000", 2024-04-20T01:48:41.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9623000000000002e-05 2024-04-20T01:48:41.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787595+0000", 2024-04-20T01:48:41.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2417000000000006e-05 2024-04-20T01:48:41.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:41.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787604+0000", 2024-04-20T01:48:41.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.952e-06 2024-04-20T01:48:41.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.791584+0000", 2024-04-20T01:48:41.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039792250000000003 2024-04-20T01:48:41.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:41.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.791699+0000", 2024-04-20T01:48:41.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115692 2024-04-20T01:48:41.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.792530+0000", 2024-04-20T01:48:41.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083077900000000002 2024-04-20T01:48:41.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.798777+0000", 2024-04-20T01:48:41.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062472379999999996 2024-04-20T01:48:41.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:41.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.804175+0000", 2024-04-20T01:48:41.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005397158 2024-04-20T01:48:41.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.804248+0000", 2024-04-20T01:48:41.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3682000000000004e-05 2024-04-20T01:48:41.293 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.293 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.293 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.293 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:235 3.0s0 3:0c1ed9f1:::smithi16579500-46:head [rollback 5] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:41.293 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.787517+0000", 2024-04-20T01:48:41.294 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.513001436000003, 2024-04-20T01:48:41.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019202061999999999, 2024-04-20T01:48:41.294 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.294 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:41.294 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:41.294 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:41.294 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:41.294 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 235 2024-04-20T01:48:41.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.294 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787517+0000", 2024-04-20T01:48:41.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787517+0000", 2024-04-20T01:48:41.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787516+0000", 2024-04-20T01:48:41.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:41.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787521+0000", 2024-04-20T01:48:41.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2599999999999996e-06 2024-04-20T01:48:41.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787534+0000", 2024-04-20T01:48:41.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2717e-05 2024-04-20T01:48:41.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787538+0000", 2024-04-20T01:48:41.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3540000000000002e-06 2024-04-20T01:48:41.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787673+0000", 2024-04-20T01:48:41.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013480799999999999 2024-04-20T01:48:41.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:41.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787678+0000", 2024-04-20T01:48:41.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5229999999999997e-06 2024-04-20T01:48:41.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793075+0000", 2024-04-20T01:48:41.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053977549999999997 2024-04-20T01:48:41.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:41.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793190+0000", 2024-04-20T01:48:41.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114775 2024-04-20T01:48:41.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793760+0000", 2024-04-20T01:48:41.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00056945599999999998 2024-04-20T01:48:41.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799198+0000", 2024-04-20T01:48:41.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0054385589999999999 2024-04-20T01:48:41.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:41.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806653+0000", 2024-04-20T01:48:41.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0074547739999999999 2024-04-20T01:48:41.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806719+0000", 2024-04-20T01:48:41.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5732e-05 2024-04-20T01:48:41.299 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.299 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.299 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.299 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:236 3.0s0 3:0c1ed9f1:::smithi16579500-46:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:41.299 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:48:41.299 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.512941822000002, 2024-04-20T01:48:41.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019273109, 2024-04-20T01:48:41.299 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.300 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:41.300 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:41.300 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:41.300 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:41.300 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 236 2024-04-20T01:48:41.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.300 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:48:41.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:48:41.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:48:41.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:41.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787579+0000", 2024-04-20T01:48:41.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6380000000000002e-06 2024-04-20T01:48:41.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787586+0000", 2024-04-20T01:48:41.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.81e-06 2024-04-20T01:48:41.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787592+0000", 2024-04-20T01:48:41.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5180000000000004e-06 2024-04-20T01:48:41.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787705+0000", 2024-04-20T01:48:41.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011387000000000001 2024-04-20T01:48:41.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:41.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787712+0000", 2024-04-20T01:48:41.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6229999999999999e-06 2024-04-20T01:48:41.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794765+0000", 2024-04-20T01:48:41.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0070530519999999998 2024-04-20T01:48:41.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:41.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794873+0000", 2024-04-20T01:48:41.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107861 2024-04-20T01:48:41.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795365+0000", 2024-04-20T01:48:41.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049192300000000001 2024-04-20T01:48:41.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799201+0000", 2024-04-20T01:48:41.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003836454 2024-04-20T01:48:41.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:41.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806808+0000", 2024-04-20T01:48:41.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076070119999999998 2024-04-20T01:48:41.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806849+0000", 2024-04-20T01:48:41.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0852999999999997e-05 2024-04-20T01:48:41.305 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.305 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.305 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.305 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:443 3.0s0 3:06dd530f:::smithi16579500-30:head [append 0~720896] snapc 15=[15,14,13,b] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:41.305 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.714463+0000", 2024-04-20T01:48:41.305 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 46.586054713999999, 2024-04-20T01:48:41.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014549549, 2024-04-20T01:48:41.306 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.306 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:41.306 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:41.306 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:41.306 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:41.306 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 443 2024-04-20T01:48:41.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.306 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714463+0000", 2024-04-20T01:48:41.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714463+0000", 2024-04-20T01:48:41.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714461+0000", 2024-04-20T01:48:41.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:41.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714916+0000", 2024-04-20T01:48:41.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00045487499999999998 2024-04-20T01:48:41.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714934+0000", 2024-04-20T01:48:41.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7586000000000001e-05 2024-04-20T01:48:41.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714962+0000", 2024-04-20T01:48:41.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7976000000000001e-05 2024-04-20T01:48:41.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.715010+0000", 2024-04-20T01:48:41.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8535999999999997e-05 2024-04-20T01:48:41.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:41.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.715016+0000", 2024-04-20T01:48:41.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8309999999999997e-06 2024-04-20T01:48:41.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717469+0000", 2024-04-20T01:48:41.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024527529999999998 2024-04-20T01:48:41.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:41.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717568+0000", 2024-04-20T01:48:41.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8708000000000004e-05 2024-04-20T01:48:41.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718717+0000", 2024-04-20T01:48:41.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0011493510000000001 2024-04-20T01:48:41.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724443+0000", 2024-04-20T01:48:41.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057254280000000003 2024-04-20T01:48:41.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:41.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.728929+0000", 2024-04-20T01:48:41.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044864919999999999 2024-04-20T01:48:41.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.729013+0000", 2024-04-20T01:48:41.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3770999999999995e-05 2024-04-20T01:48:41.311 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.311 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.311 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.311 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:444 3.0s0 3:06dd530f:::smithi16579500-30:head [append 0~442368] snapc 15=[15,14,13,b] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:41.311 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.716986+0000", 2024-04-20T01:48:41.311 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 46.58353245, 2024-04-20T01:48:41.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014745457999999999, 2024-04-20T01:48:41.311 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.311 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:41.312 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:41.312 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:41.312 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:41.312 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 444 2024-04-20T01:48:41.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.312 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.716986+0000", 2024-04-20T01:48:41.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.716986+0000", 2024-04-20T01:48:41.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.716984+0000", 2024-04-20T01:48:41.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:41.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717309+0000", 2024-04-20T01:48:41.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032479500000000001 2024-04-20T01:48:41.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717321+0000", 2024-04-20T01:48:41.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2177e-05 2024-04-20T01:48:41.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717330+0000", 2024-04-20T01:48:41.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0399999999999998e-06 2024-04-20T01:48:41.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.720236+0000", 2024-04-20T01:48:41.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002905992 2024-04-20T01:48:41.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:41.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.720367+0000", 2024-04-20T01:48:41.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000130488 2024-04-20T01:48:41.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721328+0000", 2024-04-20T01:48:41.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00096118699999999996 2024-04-20T01:48:41.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.729125+0000", 2024-04-20T01:48:41.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077972429999999997 2024-04-20T01:48:41.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:41.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731687+0000", 2024-04-20T01:48:41.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00256201 2024-04-20T01:48:41.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731731+0000", 2024-04-20T01:48:41.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4005999999999998e-05 2024-04-20T01:48:41.316 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.316 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.316 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.316 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:445 3.0s0 3:06dd530f:::smithi16579500-30:head [append 0~327680] snapc 15=[15,14,13,b] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:41.316 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.718952+0000", 2024-04-20T01:48:41.317 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 46.581565785999999, 2024-04-20T01:48:41.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012930084, 2024-04-20T01:48:41.317 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.317 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:41.317 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:41.317 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:41.317 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:41.317 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 445 2024-04-20T01:48:41.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.317 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718952+0000", 2024-04-20T01:48:41.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718952+0000", 2024-04-20T01:48:41.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718950+0000", 2024-04-20T01:48:41.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:41.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719225+0000", 2024-04-20T01:48:41.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00027498300000000002 2024-04-20T01:48:41.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719241+0000", 2024-04-20T01:48:41.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6036999999999999e-05 2024-04-20T01:48:41.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719251+0000", 2024-04-20T01:48:41.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.027e-05 2024-04-20T01:48:41.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722012+0000", 2024-04-20T01:48:41.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027604309999999998 2024-04-20T01:48:41.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:41.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722121+0000", 2024-04-20T01:48:41.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109207 2024-04-20T01:48:41.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722804+0000", 2024-04-20T01:48:41.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00068250299999999999 2024-04-20T01:48:41.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.729128+0000", 2024-04-20T01:48:41.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0063243079999999998 2024-04-20T01:48:41.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:41.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731817+0000", 2024-04-20T01:48:41.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0026888239999999998 2024-04-20T01:48:41.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731882+0000", 2024-04-20T01:48:41.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5499000000000004e-05 2024-04-20T01:48:41.321 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.321 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.321 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.321 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:446 3.0s0 3:06dd530f:::smithi16579500-30:head [setxattr _header (57) in=7b] snapc 15=[15,14,13,b] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:41.321 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.719261+0000", 2024-04-20T01:48:41.322 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 46.581257448999999, 2024-04-20T01:48:41.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012992831, 2024-04-20T01:48:41.322 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.322 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:41.322 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:41.322 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:41.322 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:41.322 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 446 2024-04-20T01:48:41.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.322 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719261+0000", 2024-04-20T01:48:41.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719261+0000", 2024-04-20T01:48:41.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719260+0000", 2024-04-20T01:48:41.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:41.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719264+0000", 2024-04-20T01:48:41.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6179999999999997e-06 2024-04-20T01:48:41.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719271+0000", 2024-04-20T01:48:41.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9949999999999999e-06 2024-04-20T01:48:41.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719277+0000", 2024-04-20T01:48:41.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6999999999999996e-06 2024-04-20T01:48:41.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723396+0000", 2024-04-20T01:48:41.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0041186599999999997 2024-04-20T01:48:41.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:41.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723458+0000", 2024-04-20T01:48:41.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2786000000000005e-05 2024-04-20T01:48:41.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723867+0000", 2024-04-20T01:48:41.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040805300000000002 2024-04-20T01:48:41.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731913+0000", 2024-04-20T01:48:41.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080461330000000005 2024-04-20T01:48:41.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:41.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731951+0000", 2024-04-20T01:48:41.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8707000000000001e-05 2024-04-20T01:48:41.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.732253+0000", 2024-04-20T01:48:41.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030198999999999997 2024-04-20T01:48:41.326 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.326 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.326 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.327 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:447 3.0s0 3:06dd530f:::smithi16579500-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:41.327 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.719282+0000", 2024-04-20T01:48:41.327 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 46.581235882000001, 2024-04-20T01:48:41.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01483317, 2024-04-20T01:48:41.327 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.327 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:41.327 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:41.327 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:41.327 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:41.327 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 447 2024-04-20T01:48:41.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.327 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719282+0000", 2024-04-20T01:48:41.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719282+0000", 2024-04-20T01:48:41.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719282+0000", 2024-04-20T01:48:41.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:41.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719294+0000", 2024-04-20T01:48:41.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2747e-05 2024-04-20T01:48:41.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719302+0000", 2024-04-20T01:48:41.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0960000000000001e-06 2024-04-20T01:48:41.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719308+0000", 2024-04-20T01:48:41.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.066e-06 2024-04-20T01:48:41.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724217+0000", 2024-04-20T01:48:41.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0049083549999999997 2024-04-20T01:48:41.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:41.330 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724333+0000", 2024-04-20T01:48:41.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011662099999999999 2024-04-20T01:48:41.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.330 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.732307+0000", 2024-04-20T01:48:41.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0079733059999999994 2024-04-20T01:48:41.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:41.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.732409+0000", 2024-04-20T01:48:41.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010193599999999999 2024-04-20T01:48:41.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.734115+0000", 2024-04-20T01:48:41.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0017065520000000001 2024-04-20T01:48:41.331 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.331 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.331 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.331 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:781 3.0s0 3:06dd530f:::smithi16579500-30:head [copy-from ver 10] snapc 2a=[2a,28] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-04-20T01:48:41.331 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.717982+0000", 2024-04-20T01:48:41.331 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.582536121, 2024-04-20T01:48:41.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.036464862000000001, 2024-04-20T01:48:41.332 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.332 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:41.332 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:41.332 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:41.332 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:41.332 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 781 2024-04-20T01:48:41.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.332 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.717982+0000", 2024-04-20T01:48:41.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.717982+0000", 2024-04-20T01:48:41.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.717980+0000", 2024-04-20T01:48:41.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:41.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.717990+0000", 2024-04-20T01:48:41.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9790000000000005e-06 2024-04-20T01:48:41.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718003+0000", 2024-04-20T01:48:41.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.342e-05 2024-04-20T01:48:41.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718014+0000", 2024-04-20T01:48:41.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0258000000000001e-05 2024-04-20T01:48:41.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718122+0000", 2024-04-20T01:48:41.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108208 2024-04-20T01:48:41.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:41.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718130+0000", 2024-04-20T01:48:41.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2120000000000003e-06 2024-04-20T01:48:41.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722371+0000", 2024-04-20T01:48:41.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042406049999999997 2024-04-20T01:48:41.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:41.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722507+0000", 2024-04-20T01:48:41.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000136136 2024-04-20T01:48:41.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.734773+0000", 2024-04-20T01:48:41.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012266325999999999 2024-04-20T01:48:41.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.742440+0000", 2024-04-20T01:48:41.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076668489999999999 2024-04-20T01:48:41.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:41.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.754352+0000", 2024-04-20T01:48:41.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011911880999999999 2024-04-20T01:48:41.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.754447+0000", 2024-04-20T01:48:41.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4827999999999994e-05 2024-04-20T01:48:41.337 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.337 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.337 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.337 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:782 3.0s0 3:06dd530f:::smithi16579500-30:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e75)", 2024-04-20T01:48:41.338 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.718032+0000", 2024-04-20T01:48:41.338 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.582486124999999, 2024-04-20T01:48:41.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020311961999999999, 2024-04-20T01:48:41.338 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.338 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:41.338 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:41.338 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:41.338 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:41.338 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 782 2024-04-20T01:48:41.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.338 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718032+0000", 2024-04-20T01:48:41.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718032+0000", 2024-04-20T01:48:41.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718031+0000", 2024-04-20T01:48:41.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:41.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718038+0000", 2024-04-20T01:48:41.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0990000000000001e-06 2024-04-20T01:48:41.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718047+0000", 2024-04-20T01:48:41.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2520000000000005e-06 2024-04-20T01:48:41.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718059+0000", 2024-04-20T01:48:41.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1965e-05 2024-04-20T01:48:41.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718210+0000", 2024-04-20T01:48:41.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150519 2024-04-20T01:48:41.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:41.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718216+0000", 2024-04-20T01:48:41.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3539999999999996e-06 2024-04-20T01:48:41.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722660+0000", 2024-04-20T01:48:41.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.004443687 2024-04-20T01:48:41.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:41.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722692+0000", 2024-04-20T01:48:41.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1761000000000003e-05 2024-04-20T01:48:41.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.738085+0000", 2024-04-20T01:48:41.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015393333 2024-04-20T01:48:41.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:41.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.738203+0000", 2024-04-20T01:48:41.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011827100000000001 2024-04-20T01:48:41.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.738344+0000", 2024-04-20T01:48:41.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014056600000000001 2024-04-20T01:48:41.343 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.343 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.343 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.343 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs1 75/30 ECSubWrite(tid=65, reqid=osd.2.0:65, at_version=75'50, trim_to=0'0, roll_forward_to=73'46))", 2024-04-20T01:48:41.343 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.720810+0000", 2024-04-20T01:48:41.343 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.579707786, 2024-04-20T01:48:41.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013232499, 2024-04-20T01:48:41.343 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.343 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.344 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720810+0000", 2024-04-20T01:48:41.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720810+0000", 2024-04-20T01:48:41.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720809+0000", 2024-04-20T01:48:41.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:41.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720840+0000", 2024-04-20T01:48:41.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0954999999999998e-05 2024-04-20T01:48:41.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720853+0000", 2024-04-20T01:48:41.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2748000000000001e-05 2024-04-20T01:48:41.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720863+0000", 2024-04-20T01:48:41.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4409999999999999e-06 2024-04-20T01:48:41.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722832+0000", 2024-04-20T01:48:41.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019689899999999999 2024-04-20T01:48:41.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722841+0000", 2024-04-20T01:48:41.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8320000000000001e-06 2024-04-20T01:48:41.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.734017+0000", 2024-04-20T01:48:41.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011175314 2024-04-20T01:48:41.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.734043+0000", 2024-04-20T01:48:41.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5962999999999999e-05 2024-04-20T01:48:41.347 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.347 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.347 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.347 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s1 75/30 ECSubWrite(tid=351, reqid=osd.13.0:351, at_version=75'121, trim_to=0'0, roll_forward_to=73'119))", 2024-04-20T01:48:41.347 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.721483+0000", 2024-04-20T01:48:41.347 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.579035275999999, 2024-04-20T01:48:41.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012358233, 2024-04-20T01:48:41.347 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.348 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.348 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721483+0000", 2024-04-20T01:48:41.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721483+0000", 2024-04-20T01:48:41.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721482+0000", 2024-04-20T01:48:41.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:41.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721517+0000", 2024-04-20T01:48:41.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4520999999999997e-05 2024-04-20T01:48:41.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721530+0000", 2024-04-20T01:48:41.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2928000000000001e-05 2024-04-20T01:48:41.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721540+0000", 2024-04-20T01:48:41.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0276e-05 2024-04-20T01:48:41.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721654+0000", 2024-04-20T01:48:41.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113496 2024-04-20T01:48:41.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721665+0000", 2024-04-20T01:48:41.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1796e-05 2024-04-20T01:48:41.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.733806+0000", 2024-04-20T01:48:41.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012140656999999999 2024-04-20T01:48:41.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.733841+0000", 2024-04-20T01:48:41.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4820999999999997e-05 2024-04-20T01:48:41.351 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.351 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.351 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.351 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s1 75/30 ECSubWrite(tid=352, reqid=osd.13.0:352, at_version=75'123, trim_to=0'0, roll_forward_to=73'119))", 2024-04-20T01:48:41.352 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.723743+0000", 2024-04-20T01:48:41.352 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.576774615000001, 2024-04-20T01:48:41.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013134636999999999, 2024-04-20T01:48:41.352 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.352 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.352 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723743+0000", 2024-04-20T01:48:41.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723743+0000", 2024-04-20T01:48:41.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723742+0000", 2024-04-20T01:48:41.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:41.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723769+0000", 2024-04-20T01:48:41.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6086999999999999e-05 2024-04-20T01:48:41.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723782+0000", 2024-04-20T01:48:41.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3675e-05 2024-04-20T01:48:41.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723794+0000", 2024-04-20T01:48:41.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1965e-05 2024-04-20T01:48:41.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725734+0000", 2024-04-20T01:48:41.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019396439999999999 2024-04-20T01:48:41.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725743+0000", 2024-04-20T01:48:41.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8659999999999995e-06 2024-04-20T01:48:41.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736860+0000", 2024-04-20T01:48:41.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011117311 2024-04-20T01:48:41.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736878+0000", 2024-04-20T01:48:41.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7938999999999999e-05 2024-04-20T01:48:41.356 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.356 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.356 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.356 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs3 75/30 ECSubWrite(tid=112, reqid=osd.14.0:112, at_version=75'91, trim_to=0'0, roll_forward_to=74'89))", 2024-04-20T01:48:41.356 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.724406+0000", 2024-04-20T01:48:41.356 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.576111926999999, 2024-04-20T01:48:41.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012721514999999999, 2024-04-20T01:48:41.356 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.356 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.356 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724406+0000", 2024-04-20T01:48:41.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724406+0000", 2024-04-20T01:48:41.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724406+0000", 2024-04-20T01:48:41.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:41.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724444+0000", 2024-04-20T01:48:41.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8659999999999999e-05 2024-04-20T01:48:41.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724455+0000", 2024-04-20T01:48:41.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0468999999999999e-05 2024-04-20T01:48:41.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724469+0000", 2024-04-20T01:48:41.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4498000000000001e-05 2024-04-20T01:48:41.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724894+0000", 2024-04-20T01:48:41.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00042428799999999999 2024-04-20T01:48:41.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724905+0000", 2024-04-20T01:48:41.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1415e-05 2024-04-20T01:48:41.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.737103+0000", 2024-04-20T01:48:41.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012198293000000001 2024-04-20T01:48:41.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.737128+0000", 2024-04-20T01:48:41.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4178000000000001e-05 2024-04-20T01:48:41.360 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.360 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.360 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.360 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs3 75/30 ECSubWrite(tid=113, reqid=osd.14.0:113, at_version=75'93, trim_to=0'0, roll_forward_to=74'89))", 2024-04-20T01:48:41.360 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.724486+0000", 2024-04-20T01:48:41.360 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.576032233999999, 2024-04-20T01:48:41.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012679546999999999, 2024-04-20T01:48:41.361 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.361 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.361 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724486+0000", 2024-04-20T01:48:41.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724486+0000", 2024-04-20T01:48:41.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724485+0000", 2024-04-20T01:48:41.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:41.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724545+0000", 2024-04-20T01:48:41.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9892000000000002e-05 2024-04-20T01:48:41.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724556+0000", 2024-04-20T01:48:41.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0706999999999999e-05 2024-04-20T01:48:41.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724564+0000", 2024-04-20T01:48:41.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5129999999999996e-06 2024-04-20T01:48:41.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725568+0000", 2024-04-20T01:48:41.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001003547 2024-04-20T01:48:41.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725593+0000", 2024-04-20T01:48:41.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4604e-05 2024-04-20T01:48:41.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.737155+0000", 2024-04-20T01:48:41.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011562915 2024-04-20T01:48:41.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.737165+0000", 2024-04-20T01:48:41.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7909999999999999e-06 2024-04-20T01:48:41.364 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.364 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.364 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.365 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s3 75/30 ECSubWrite(tid=101, reqid=osd.5.0:101, at_version=75'82, trim_to=0'0, roll_forward_to=68'78))", 2024-04-20T01:48:41.365 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.724669+0000", 2024-04-20T01:48:41.365 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.575849300000002, 2024-04-20T01:48:41.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012107346, 2024-04-20T01:48:41.365 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.365 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.365 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724669+0000", 2024-04-20T01:48:41.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724669+0000", 2024-04-20T01:48:41.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724668+0000", 2024-04-20T01:48:41.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:41.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724693+0000", 2024-04-20T01:48:41.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4289000000000001e-05 2024-04-20T01:48:41.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724707+0000", 2024-04-20T01:48:41.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4489e-05 2024-04-20T01:48:41.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724715+0000", 2024-04-20T01:48:41.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.174e-06 2024-04-20T01:48:41.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725729+0000", 2024-04-20T01:48:41.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0010136870000000001 2024-04-20T01:48:41.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725739+0000", 2024-04-20T01:48:41.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0064000000000001e-05 2024-04-20T01:48:41.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736749+0000", 2024-04-20T01:48:41.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011009778 2024-04-20T01:48:41.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736776+0000", 2024-04-20T01:48:41.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7220999999999999e-05 2024-04-20T01:48:41.368 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.369 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.369 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.369 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs3 80/30 ECSubWrite(tid=132, reqid=osd.14.0:132, at_version=80'111, trim_to=0'0, roll_forward_to=77'109))", 2024-04-20T01:48:41.369 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:21.729572+0000", 2024-04-20T01:48:41.369 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 19.570945709, 2024-04-20T01:48:41.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020941090999999998, 2024-04-20T01:48:41.369 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.369 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.369 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729572+0000", 2024-04-20T01:48:41.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729572+0000", 2024-04-20T01:48:41.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729572+0000", 2024-04-20T01:48:41.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:41.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729602+0000", 2024-04-20T01:48:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0369e-05 2024-04-20T01:48:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729615+0000", 2024-04-20T01:48:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3158e-05 2024-04-20T01:48:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729626+0000", 2024-04-20T01:48:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0056e-05 2024-04-20T01:48:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729699+0000", 2024-04-20T01:48:41.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3857999999999997e-05 2024-04-20T01:48:41.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729712+0000", 2024-04-20T01:48:41.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3088000000000001e-05 2024-04-20T01:48:41.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.750484+0000", 2024-04-20T01:48:41.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020771511999999999 2024-04-20T01:48:41.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.750513+0000", 2024-04-20T01:48:41.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9363e-05 2024-04-20T01:48:41.373 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.373 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.373 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.373 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.es5 80/30 ECSubWrite(tid=112, reqid=osd.11.0:112, at_version=80'88, trim_to=0'0, roll_forward_to=77'86))", 2024-04-20T01:48:41.373 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:21.733917+0000", 2024-04-20T01:48:41.373 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 19.566600805, 2024-04-20T01:48:41.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015930045, 2024-04-20T01:48:41.373 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.373 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.373 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.733917+0000", 2024-04-20T01:48:41.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.733917+0000", 2024-04-20T01:48:41.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.733917+0000", 2024-04-20T01:48:41.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:41.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.733943+0000", 2024-04-20T01:48:41.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6764e-05 2024-04-20T01:48:41.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.733956+0000", 2024-04-20T01:48:41.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2537999999999999e-05 2024-04-20T01:48:41.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.733966+0000", 2024-04-20T01:48:41.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0145e-05 2024-04-20T01:48:41.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.734023+0000", 2024-04-20T01:48:41.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7231000000000001e-05 2024-04-20T01:48:41.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.734035+0000", 2024-04-20T01:48:41.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1163e-05 2024-04-20T01:48:41.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.749818+0000", 2024-04-20T01:48:41.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015783452 2024-04-20T01:48:41.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.749847+0000", 2024-04-20T01:48:41.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9235e-05 2024-04-20T01:48:41.377 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.377 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.377 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.377 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s5 80/57 ECSubWrite(tid=205, reqid=osd.8.0:205, at_version=80'147, trim_to=0'0, roll_forward_to=79'145))", 2024-04-20T01:48:41.377 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:21.737407+0000", 2024-04-20T01:48:41.377 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 19.563111191000001, 2024-04-20T01:48:41.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013126919000000001, 2024-04-20T01:48:41.377 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.377 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.377 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737407+0000", 2024-04-20T01:48:41.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737407+0000", 2024-04-20T01:48:41.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737406+0000", 2024-04-20T01:48:41.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:41.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737434+0000", 2024-04-20T01:48:41.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7623999999999998e-05 2024-04-20T01:48:41.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737448+0000", 2024-04-20T01:48:41.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3811e-05 2024-04-20T01:48:41.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737458+0000", 2024-04-20T01:48:41.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0426e-05 2024-04-20T01:48:41.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737568+0000", 2024-04-20T01:48:41.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109221 2024-04-20T01:48:41.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737579+0000", 2024-04-20T01:48:41.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1895e-05 2024-04-20T01:48:41.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.750521+0000", 2024-04-20T01:48:41.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012941998 2024-04-20T01:48:41.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.750534+0000", 2024-04-20T01:48:41.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2267e-05 2024-04-20T01:48:41.381 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.381 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.381 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.381 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s5 80/57 ECSubWrite(tid=206, reqid=osd.8.0:206, at_version=80'149, trim_to=0'0, roll_forward_to=79'145))", 2024-04-20T01:48:41.382 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:21.738507+0000", 2024-04-20T01:48:41.382 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 19.562010990000001, 2024-04-20T01:48:41.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014548627999999999, 2024-04-20T01:48:41.382 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.382 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.382 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.738507+0000", 2024-04-20T01:48:41.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.738507+0000", 2024-04-20T01:48:41.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.738506+0000", 2024-04-20T01:48:41.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:41.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.738540+0000", 2024-04-20T01:48:41.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4316000000000001e-05 2024-04-20T01:48:41.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.738553+0000", 2024-04-20T01:48:41.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2755000000000001e-05 2024-04-20T01:48:41.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.738564+0000", 2024-04-20T01:48:41.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.066e-05 2024-04-20T01:48:41.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.739602+0000", 2024-04-20T01:48:41.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001038186 2024-04-20T01:48:41.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.739612+0000", 2024-04-20T01:48:41.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0145e-05 2024-04-20T01:48:41.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.753027+0000", 2024-04-20T01:48:41.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013415152 2024-04-20T01:48:41.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.753056+0000", 2024-04-20T01:48:41.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8393000000000001e-05 2024-04-20T01:48:41.385 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.385 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.385 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.386 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.386 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.386 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:41.387 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-20T01:48:41.388 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:41.401+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:41.487 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:41.502+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:41.556 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:41.556 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:41.556 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-04-20T01:48:41.556 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:41.566 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-20T01:48:41.588 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:41.603+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:41.689 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:41.705+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:41.695 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:41.695 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:41.695 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:48:41.695 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:48:41.695 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:41.705 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-20T01:48:41.790 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:41.805+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:41.809 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is set 2024-04-20T01:48:41.835 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:41.835 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:48:41.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:48:41.835 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:48:41.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.835 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:289 3.9s0 3:94399c02:::smithi16579500-3:head [copy-from ver 6] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:41.835 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:43.807269+0000", 2024-04-20T01:48:41.836 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 58.041509617000003, 2024-04-20T01:48:41.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016170202000000002, 2024-04-20T01:48:41.836 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.836 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:41.836 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:41.836 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:41.836 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:41.836 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 289 2024-04-20T01:48:41.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.836 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807269+0000", 2024-04-20T01:48:41.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807269+0000", 2024-04-20T01:48:41.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807267+0000", 2024-04-20T01:48:41.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:41.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807280+0000", 2024-04-20T01:48:41.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3227999999999999e-05 2024-04-20T01:48:41.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807303+0000", 2024-04-20T01:48:41.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2991000000000001e-05 2024-04-20T01:48:41.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807318+0000", 2024-04-20T01:48:41.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4321999999999999e-05 2024-04-20T01:48:41.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807385+0000", 2024-04-20T01:48:41.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7403000000000005e-05 2024-04-20T01:48:41.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:41.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807394+0000", 2024-04-20T01:48:41.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7010000000000002e-06 2024-04-20T01:48:41.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.809246+0000", 2024-04-20T01:48:41.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0018521 2024-04-20T01:48:41.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:41.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.809314+0000", 2024-04-20T01:48:41.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7818999999999996e-05 2024-04-20T01:48:41.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.817661+0000", 2024-04-20T01:48:41.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0083471710000000005 2024-04-20T01:48:41.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.819085+0000", 2024-04-20T01:48:41.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0014239739999999999 2024-04-20T01:48:41.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:41.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.823350+0000", 2024-04-20T01:48:41.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042652990000000002 2024-04-20T01:48:41.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.823439+0000", 2024-04-20T01:48:41.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9424000000000003e-05 2024-04-20T01:48:41.841 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.841 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.841 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.842 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:290 3.9s0 3:94399c02:::smithi16579500-3:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:41.842 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:43.807329+0000", 2024-04-20T01:48:41.842 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 58.041449696000001, 2024-04-20T01:48:41.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011459461000000001, 2024-04-20T01:48:41.842 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.842 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:41.842 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:41.842 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:41.842 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:41.842 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 290 2024-04-20T01:48:41.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.843 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807329+0000", 2024-04-20T01:48:41.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807329+0000", 2024-04-20T01:48:41.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807329+0000", 2024-04-20T01:48:41.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:41.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807332+0000", 2024-04-20T01:48:41.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.349e-06 2024-04-20T01:48:41.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807343+0000", 2024-04-20T01:48:41.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1270999999999999e-05 2024-04-20T01:48:41.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807347+0000", 2024-04-20T01:48:41.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-04-20T01:48:41.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807453+0000", 2024-04-20T01:48:41.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106162 2024-04-20T01:48:41.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:41.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807460+0000", 2024-04-20T01:48:41.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9809999999999997e-06 2024-04-20T01:48:41.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.809441+0000", 2024-04-20T01:48:41.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019806060000000002 2024-04-20T01:48:41.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:41.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.809484+0000", 2024-04-20T01:48:41.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3019000000000003e-05 2024-04-20T01:48:41.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.818614+0000", 2024-04-20T01:48:41.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091302699999999994 2024-04-20T01:48:41.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:41.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.818713+0000", 2024-04-20T01:48:41.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8776000000000004e-05 2024-04-20T01:48:41.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.818789+0000", 2024-04-20T01:48:41.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5944000000000003e-05 2024-04-20T01:48:41.847 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.847 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.847 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.847 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 47/30 ECSubWrite(tid=38, reqid=osd.14.0:38, at_version=47'31, trim_to=0'0, roll_forward_to=46'29))", 2024-04-20T01:48:41.847 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.711869+0000", 2024-04-20T01:48:41.848 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 53.136909807999999, 2024-04-20T01:48:41.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016896644999999998, 2024-04-20T01:48:41.848 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.848 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.848 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711869+0000", 2024-04-20T01:48:41.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711869+0000", 2024-04-20T01:48:41.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711868+0000", 2024-04-20T01:48:41.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:41.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711907+0000", 2024-04-20T01:48:41.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8828000000000002e-05 2024-04-20T01:48:41.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711927+0000", 2024-04-20T01:48:41.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9656e-05 2024-04-20T01:48:41.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711937+0000", 2024-04-20T01:48:41.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0277e-05 2024-04-20T01:48:41.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711991+0000", 2024-04-20T01:48:41.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3980000000000002e-05 2024-04-20T01:48:41.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.712002+0000", 2024-04-20T01:48:41.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719e-05 2024-04-20T01:48:41.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.728747+0000", 2024-04-20T01:48:41.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016745466000000001 2024-04-20T01:48:41.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.728766+0000", 2024-04-20T01:48:41.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8607999999999999e-05 2024-04-20T01:48:41.852 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.852 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.852 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.852 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 47/30 ECSubWrite(tid=39, reqid=osd.14.0:39, at_version=47'33, trim_to=0'0, roll_forward_to=46'29))", 2024-04-20T01:48:41.852 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.713021+0000", 2024-04-20T01:48:41.852 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 53.135757601000002, 2024-04-20T01:48:41.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020945089, 2024-04-20T01:48:41.852 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.852 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.852 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713021+0000", 2024-04-20T01:48:41.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713021+0000", 2024-04-20T01:48:41.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713020+0000", 2024-04-20T01:48:41.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:41.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713046+0000", 2024-04-20T01:48:41.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5554e-05 2024-04-20T01:48:41.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713064+0000", 2024-04-20T01:48:41.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7635999999999999e-05 2024-04-20T01:48:41.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713074+0000", 2024-04-20T01:48:41.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9529999999999994e-06 2024-04-20T01:48:41.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713140+0000", 2024-04-20T01:48:41.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6235999999999994e-05 2024-04-20T01:48:41.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713155+0000", 2024-04-20T01:48:41.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5647000000000001e-05 2024-04-20T01:48:41.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733953+0000", 2024-04-20T01:48:41.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020797936 2024-04-20T01:48:41.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733966+0000", 2024-04-20T01:48:41.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3067999999999999e-05 2024-04-20T01:48:41.857 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.857 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.857 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.857 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s1 47/30 ECSubWrite(tid=30, reqid=osd.3.0:30, at_version=47'19, trim_to=0'0, roll_forward_to=44'17))", 2024-04-20T01:48:41.857 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.713663+0000", 2024-04-20T01:48:41.857 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 53.135115841000001, 2024-04-20T01:48:41.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0202712, 2024-04-20T01:48:41.857 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.857 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.858 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713663+0000", 2024-04-20T01:48:41.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713663+0000", 2024-04-20T01:48:41.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713662+0000", 2024-04-20T01:48:41.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:41.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713693+0000", 2024-04-20T01:48:41.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1031999999999997e-05 2024-04-20T01:48:41.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713712+0000", 2024-04-20T01:48:41.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8615999999999999e-05 2024-04-20T01:48:41.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713726+0000", 2024-04-20T01:48:41.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4477e-05 2024-04-20T01:48:41.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713779+0000", 2024-04-20T01:48:41.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3090999999999997e-05 2024-04-20T01:48:41.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713791+0000", 2024-04-20T01:48:41.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1921e-05 2024-04-20T01:48:41.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733906+0000", 2024-04-20T01:48:41.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020114522999999999 2024-04-20T01:48:41.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733934+0000", 2024-04-20T01:48:41.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8357999999999999e-05 2024-04-20T01:48:41.863 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.863 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.863 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.863 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 47/30 ECSubWrite(tid=39, reqid=osd.5.0:39, at_version=47'27, trim_to=0'0, roll_forward_to=46'25))", 2024-04-20T01:48:41.863 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.718432+0000", 2024-04-20T01:48:41.863 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 53.130347184000001, 2024-04-20T01:48:41.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015468671, 2024-04-20T01:48:41.863 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.863 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.863 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718432+0000", 2024-04-20T01:48:41.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718432+0000", 2024-04-20T01:48:41.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718431+0000", 2024-04-20T01:48:41.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:41.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718455+0000", 2024-04-20T01:48:41.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3085e-05 2024-04-20T01:48:41.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718471+0000", 2024-04-20T01:48:41.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6172999999999999e-05 2024-04-20T01:48:41.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718480+0000", 2024-04-20T01:48:41.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5719999999999992e-06 2024-04-20T01:48:41.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718592+0000", 2024-04-20T01:48:41.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111517 2024-04-20T01:48:41.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718602+0000", 2024-04-20T01:48:41.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.064e-05 2024-04-20T01:48:41.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733869+0000", 2024-04-20T01:48:41.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015266798 2024-04-20T01:48:41.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733900+0000", 2024-04-20T01:48:41.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.116e-05 2024-04-20T01:48:41.867 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.867 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.867 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.867 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 47/30 ECSubWrite(tid=40, reqid=osd.5.0:40, at_version=47'29, trim_to=0'0, roll_forward_to=46'25))", 2024-04-20T01:48:41.867 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.719277+0000", 2024-04-20T01:48:41.867 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 53.129501488000002, 2024-04-20T01:48:41.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017279452000000001, 2024-04-20T01:48:41.867 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.868 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.868 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719277+0000", 2024-04-20T01:48:41.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719277+0000", 2024-04-20T01:48:41.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719277+0000", 2024-04-20T01:48:41.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:41.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719306+0000", 2024-04-20T01:48:41.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9771999999999999e-05 2024-04-20T01:48:41.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719322+0000", 2024-04-20T01:48:41.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6035000000000001e-05 2024-04-20T01:48:41.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719331+0000", 2024-04-20T01:48:41.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1130000000000007e-06 2024-04-20T01:48:41.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.728810+0000", 2024-04-20T01:48:41.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094789090000000006 2024-04-20T01:48:41.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.728819+0000", 2024-04-20T01:48:41.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8890000000000008e-06 2024-04-20T01:48:41.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.736530+0000", 2024-04-20T01:48:41.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077106129999999998 2024-04-20T01:48:41.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.736557+0000", 2024-04-20T01:48:41.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6974999999999998e-05 2024-04-20T01:48:41.871 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.871 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.871 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.871 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 47/30 ECSubWrite(tid=240, reqid=osd.13.0:240, at_version=47'33, trim_to=0'0, roll_forward_to=44'31))", 2024-04-20T01:48:41.871 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.722777+0000", 2024-04-20T01:48:41.871 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 53.126001774000002, 2024-04-20T01:48:41.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008649894, 2024-04-20T01:48:41.872 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.872 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.872 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722777+0000", 2024-04-20T01:48:41.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722777+0000", 2024-04-20T01:48:41.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722777+0000", 2024-04-20T01:48:41.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:41.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722805+0000", 2024-04-20T01:48:41.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7881e-05 2024-04-20T01:48:41.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722820+0000", 2024-04-20T01:48:41.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5387e-05 2024-04-20T01:48:41.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722831+0000", 2024-04-20T01:48:41.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813e-05 2024-04-20T01:48:41.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722915+0000", 2024-04-20T01:48:41.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3578000000000005e-05 2024-04-20T01:48:41.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722929+0000", 2024-04-20T01:48:41.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4256999999999999e-05 2024-04-20T01:48:41.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.731402+0000", 2024-04-20T01:48:41.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008472785 2024-04-20T01:48:41.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.731427+0000", 2024-04-20T01:48:41.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5355e-05 2024-04-20T01:48:41.875 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.875 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.875 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.875 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 47/30 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=47'35, trim_to=0'0, roll_forward_to=44'31))", 2024-04-20T01:48:41.875 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.723066+0000", 2024-04-20T01:48:41.876 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 53.125713028, 2024-04-20T01:48:41.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010874728, 2024-04-20T01:48:41.876 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.876 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.876 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723066+0000", 2024-04-20T01:48:41.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723066+0000", 2024-04-20T01:48:41.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723065+0000", 2024-04-20T01:48:41.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:41.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723089+0000", 2024-04-20T01:48:41.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3529000000000001e-05 2024-04-20T01:48:41.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723101+0000", 2024-04-20T01:48:41.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2143e-05 2024-04-20T01:48:41.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723110+0000", 2024-04-20T01:48:41.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9809999999999992e-06 2024-04-20T01:48:41.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.725642+0000", 2024-04-20T01:48:41.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002532337 2024-04-20T01:48:41.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.725655+0000", 2024-04-20T01:48:41.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2707e-05 2024-04-20T01:48:41.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733911+0000", 2024-04-20T01:48:41.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082559369999999997 2024-04-20T01:48:41.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733941+0000", 2024-04-20T01:48:41.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9798000000000002e-05 2024-04-20T01:48:41.879 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.879 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.879 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.880 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:396 3.9s0 3:9a2fbd15:::smithi16579500-1:head [copy-from ver 32] snapc d=[c,b] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-20T01:48:41.880 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.746088+0000", 2024-04-20T01:48:41.880 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 53.102690850000002, 2024-04-20T01:48:41.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01355685, 2024-04-20T01:48:41.880 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.880 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:41.880 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:41.880 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:41.880 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:41.880 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 396 2024-04-20T01:48:41.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.880 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746088+0000", 2024-04-20T01:48:41.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746088+0000", 2024-04-20T01:48:41.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746086+0000", 2024-04-20T01:48:41.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:41.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746094+0000", 2024-04-20T01:48:41.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.272e-06 2024-04-20T01:48:41.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746109+0000", 2024-04-20T01:48:41.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5563999999999999e-05 2024-04-20T01:48:41.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746131+0000", 2024-04-20T01:48:41.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1515999999999999e-05 2024-04-20T01:48:41.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746178+0000", 2024-04-20T01:48:41.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7472000000000002e-05 2024-04-20T01:48:41.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:41.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746261+0000", 2024-04-20T01:48:41.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2412999999999999e-05 2024-04-20T01:48:41.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.752312+0000", 2024-04-20T01:48:41.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0060515120000000002 2024-04-20T01:48:41.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.754299+0000", 2024-04-20T01:48:41.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019870690000000002 2024-04-20T01:48:41.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:41.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.759594+0000", 2024-04-20T01:48:41.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0052944070000000001 2024-04-20T01:48:41.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.759645+0000", 2024-04-20T01:48:41.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1236000000000002e-05 2024-04-20T01:48:41.885 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.885 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.885 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.885 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 48/30 ECSubWrite(tid=44, reqid=osd.5.0:44, at_version=48'33, trim_to=0'0, roll_forward_to=47'31))", 2024-04-20T01:48:41.885 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.716311+0000", 2024-04-20T01:48:41.885 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.132467445000003, 2024-04-20T01:48:41.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093701210000000004, 2024-04-20T01:48:41.885 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.886 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.886 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716311+0000", 2024-04-20T01:48:41.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716311+0000", 2024-04-20T01:48:41.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716311+0000", 2024-04-20T01:48:41.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:41.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716336+0000", 2024-04-20T01:48:41.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5216e-05 2024-04-20T01:48:41.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716348+0000", 2024-04-20T01:48:41.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1369000000000001e-05 2024-04-20T01:48:41.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716358+0000", 2024-04-20T01:48:41.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9019999999999994e-06 2024-04-20T01:48:41.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716410+0000", 2024-04-20T01:48:41.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1974000000000001e-05 2024-04-20T01:48:41.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716419+0000", 2024-04-20T01:48:41.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4439999999999997e-06 2024-04-20T01:48:41.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.725652+0000", 2024-04-20T01:48:41.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092323909999999995 2024-04-20T01:48:41.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.725682+0000", 2024-04-20T01:48:41.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0049999999999999e-05 2024-04-20T01:48:41.889 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.889 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.889 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.889 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 48/30 ECSubWrite(tid=256, reqid=client.4643.0:403, at_version=48'46, trim_to=0'0, roll_forward_to=48'44))", 2024-04-20T01:48:41.890 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.789372+0000", 2024-04-20T01:48:41.890 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.059406869, 2024-04-20T01:48:41.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086922490000000008, 2024-04-20T01:48:41.890 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.890 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.890 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789372+0000", 2024-04-20T01:48:41.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789372+0000", 2024-04-20T01:48:41.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789371+0000", 2024-04-20T01:48:41.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:41.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789968+0000", 2024-04-20T01:48:41.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00059678300000000004 2024-04-20T01:48:41.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789979+0000", 2024-04-20T01:48:41.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1499e-05 2024-04-20T01:48:41.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789986+0000", 2024-04-20T01:48:41.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2179999999999996e-06 2024-04-20T01:48:41.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.790029+0000", 2024-04-20T01:48:41.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2876000000000001e-05 2024-04-20T01:48:41.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.790036+0000", 2024-04-20T01:48:41.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7870000000000002e-06 2024-04-20T01:48:41.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.798040+0000", 2024-04-20T01:48:41.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080038720000000004 2024-04-20T01:48:41.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.798064+0000", 2024-04-20T01:48:41.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4037999999999999e-05 2024-04-20T01:48:41.895 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.895 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.895 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.895 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 48/30 ECSubWrite(tid=257, reqid=client.4643.0:404, at_version=48'47, trim_to=0'0, roll_forward_to=48'44))", 2024-04-20T01:48:41.895 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.791622+0000", 2024-04-20T01:48:41.895 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.057156882999998, 2024-04-20T01:48:41.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0088453630000000002, 2024-04-20T01:48:41.895 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.895 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.896 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.791622+0000", 2024-04-20T01:48:41.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.791622+0000", 2024-04-20T01:48:41.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.791621+0000", 2024-04-20T01:48:41.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:41.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792215+0000", 2024-04-20T01:48:41.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000593689 2024-04-20T01:48:41.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792227+0000", 2024-04-20T01:48:41.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2037e-05 2024-04-20T01:48:41.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792236+0000", 2024-04-20T01:48:41.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8240000000000006e-06 2024-04-20T01:48:41.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792538+0000", 2024-04-20T01:48:41.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000301668 2024-04-20T01:48:41.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792549+0000", 2024-04-20T01:48:41.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0915e-05 2024-04-20T01:48:41.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.800443+0000", 2024-04-20T01:48:41.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0078945300000000003 2024-04-20T01:48:41.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.800467+0000", 2024-04-20T01:48:41.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4309000000000001e-05 2024-04-20T01:48:41.899 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.899 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.899 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.899 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s1 53/30 ECSubWrite(tid=33, reqid=client.4643.0:444, at_version=53'22, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:41.899 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.721415+0000", 2024-04-20T01:48:41.899 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.127364106000002, 2024-04-20T01:48:41.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097484879999999996, 2024-04-20T01:48:41.900 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.900 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.900 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721415+0000", 2024-04-20T01:48:41.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721415+0000", 2024-04-20T01:48:41.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721413+0000", 2024-04-20T01:48:41.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:41.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721522+0000", 2024-04-20T01:48:41.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108445 2024-04-20T01:48:41.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721540+0000", 2024-04-20T01:48:41.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8128e-05 2024-04-20T01:48:41.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721555+0000", 2024-04-20T01:48:41.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5228999999999999e-05 2024-04-20T01:48:41.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721639+0000", 2024-04-20T01:48:41.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3803999999999999e-05 2024-04-20T01:48:41.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721650+0000", 2024-04-20T01:48:41.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0702e-05 2024-04-20T01:48:41.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731131+0000", 2024-04-20T01:48:41.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094816929999999994 2024-04-20T01:48:41.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731163+0000", 2024-04-20T01:48:41.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1946999999999998e-05 2024-04-20T01:48:41.903 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.903 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.903 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.904 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 58/30 ECSubWrite(tid=52, reqid=client.4643.0:489, at_version=58'41, trim_to=0'0, roll_forward_to=55'37))", 2024-04-20T01:48:41.904 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:59.769343+0000", 2024-04-20T01:48:41.904 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 42.079435768000003, 2024-04-20T01:48:41.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094380780000000008, 2024-04-20T01:48:41.904 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.904 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.904 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769343+0000", 2024-04-20T01:48:41.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769343+0000", 2024-04-20T01:48:41.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769343+0000", 2024-04-20T01:48:41.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:41.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769462+0000", 2024-04-20T01:48:41.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011929 2024-04-20T01:48:41.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769474+0000", 2024-04-20T01:48:41.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1857e-05 2024-04-20T01:48:41.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769482+0000", 2024-04-20T01:48:41.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0809999999999994e-06 2024-04-20T01:48:41.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769531+0000", 2024-04-20T01:48:41.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9168000000000001e-05 2024-04-20T01:48:41.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769538+0000", 2024-04-20T01:48:41.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5930000000000002e-06 2024-04-20T01:48:41.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.778749+0000", 2024-04-20T01:48:41.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092115670000000004 2024-04-20T01:48:41.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.778781+0000", 2024-04-20T01:48:41.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2140000000000001e-05 2024-04-20T01:48:41.907 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.907 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.907 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.908 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:733 3.9s0 3:94399c02:::smithi16579500-3:head [rmxattr Pp[amim]nQkdfb][ca]Rhonh_mr`cpapUabYUfb\\ in=40b,rmxattr QP`igcZgYZ`X]j_`ObrUNcR_gQZW^mQs_T`XhhQp in=40b,rmxattr QXqRngrniZSrdNdQaffOsm_g_pYQoedrXoNqT[mn in=40b,rmxattr V[thp]nUrl^Xata^al[oh[d`ddg`p[siljQWZ^\\X in=40b,rmxattr XgcaedbaNamUmoa^d[dhg[arrjeikfUlToSjsbY` in=40b,rmxattr ZcXlR_O_^fXfTQrt`cZfOgVlmhepod\\^\\`socsW` in=40b,rmxattr ][ifNPWgaZd[^RnfpRhWbfS\\^a]m[\\[ienetlUUZ in=40b,rmxattr `U`PSXddYml`lYXX[sSNnPfg`VgWbXsYnftUQ`it in=40b,rmxattr aR in=2b,rmxattr eNsVcjeRcj\\m]Q[VhYdSqNcmPhhR[UPR^ORSk_Vd in=40b,rmxattr eRqQZNtlPRVjcgdjmbSmOsUSjSlU]tSoVjr`^oRN in=40b,rmxattr fVRh^qlag^rjpdaqVk`^r]eR`P[XUlr]]agZjWq[ in=40b,rmxattr frZXQX\\isoq]gmjs`rk]PiQ\\fV[RTNPNYN]YV[mq in=40b,rmxattr h\\bUdlZ]kknhkaaYeTcclaXatghTgbVrhNQblto_ in=40b,rmxattr hgq\\iqsggNPTRPQok in=17b,rmxattr jecnmafmfg__r^\\S_sU]bi]kYaplTcttSSX`\\prR in=40b,rmxattr jmliR in=5b,rmxattr mnhUacnNajpOrNecmPZT[OUj[XaWmU in=30b,rmxattr peUYXiNb_UsOaXnhlX_s_[QTW_SoUaN]Wal`rkao in=40b,rmxattr rZrXsckW]Y`iWQiRn in=17b] snapc 26=[26,25,1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-20T01:48:41.908 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:11.823678+0000", 2024-04-20T01:48:41.908 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.025101402000001, 2024-04-20T01:48:41.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0098521919999999992, 2024-04-20T01:48:41.908 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.908 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:41.908 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:41.908 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:41.908 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:41.908 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 733 2024-04-20T01:48:41.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.908 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823678+0000", 2024-04-20T01:48:41.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823678+0000", 2024-04-20T01:48:41.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823675+0000", 2024-04-20T01:48:41.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:41.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823691+0000", 2024-04-20T01:48:41.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5954999999999999e-05 2024-04-20T01:48:41.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823715+0000", 2024-04-20T01:48:41.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4113000000000001e-05 2024-04-20T01:48:41.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823730+0000", 2024-04-20T01:48:41.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4694e-05 2024-04-20T01:48:41.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823786+0000", 2024-04-20T01:48:41.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6419000000000003e-05 2024-04-20T01:48:41.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:41.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823913+0000", 2024-04-20T01:48:41.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012677500000000001 2024-04-20T01:48:41.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.824989+0000", 2024-04-20T01:48:41.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0010762199999999999 2024-04-20T01:48:41.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.828014+0000", 2024-04-20T01:48:41.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0030246190000000001 2024-04-20T01:48:41.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:41.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.833442+0000", 2024-04-20T01:48:41.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0054285490000000004 2024-04-20T01:48:41.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.833530+0000", 2024-04-20T01:48:41.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7462999999999999e-05 2024-04-20T01:48:41.912 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.913 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.913 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.913 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 75/30 ECSubWrite(tid=100, reqid=osd.5.0:100, at_version=75'80, trim_to=0'0, roll_forward_to=68'78))", 2024-04-20T01:48:41.913 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.720432+0000", 2024-04-20T01:48:41.913 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.128346656000002, 2024-04-20T01:48:41.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010779983, 2024-04-20T01:48:41.913 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.913 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.913 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720432+0000", 2024-04-20T01:48:41.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720432+0000", 2024-04-20T01:48:41.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720432+0000", 2024-04-20T01:48:41.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:41.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720457+0000", 2024-04-20T01:48:41.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5272000000000001e-05 2024-04-20T01:48:41.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720469+0000", 2024-04-20T01:48:41.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2193e-05 2024-04-20T01:48:41.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720480+0000", 2024-04-20T01:48:41.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0291e-05 2024-04-20T01:48:41.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723263+0000", 2024-04-20T01:48:41.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002783003 2024-04-20T01:48:41.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723273+0000", 2024-04-20T01:48:41.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.01e-05 2024-04-20T01:48:41.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731180+0000", 2024-04-20T01:48:41.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0079069929999999993 2024-04-20T01:48:41.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731212+0000", 2024-04-20T01:48:41.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2421e-05 2024-04-20T01:48:41.917 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.917 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.917 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.917 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 75/30 ECSubWrite(tid=112, reqid=osd.14.0:112, at_version=75'91, trim_to=0'0, roll_forward_to=74'89))", 2024-04-20T01:48:41.917 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.720882+0000", 2024-04-20T01:48:41.917 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.127897127000001, 2024-04-20T01:48:41.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010362481999999999, 2024-04-20T01:48:41.917 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.917 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.917 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720882+0000", 2024-04-20T01:48:41.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720882+0000", 2024-04-20T01:48:41.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720881+0000", 2024-04-20T01:48:41.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:41.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720905+0000", 2024-04-20T01:48:41.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3683e-05 2024-04-20T01:48:41.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720917+0000", 2024-04-20T01:48:41.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1651000000000001e-05 2024-04-20T01:48:41.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720926+0000", 2024-04-20T01:48:41.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2040000000000003e-06 2024-04-20T01:48:41.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724280+0000", 2024-04-20T01:48:41.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003354363 2024-04-20T01:48:41.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724291+0000", 2024-04-20T01:48:41.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1004e-05 2024-04-20T01:48:41.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731213+0000", 2024-04-20T01:48:41.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069213670000000003 2024-04-20T01:48:41.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731244+0000", 2024-04-20T01:48:41.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1563000000000003e-05 2024-04-20T01:48:41.921 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.921 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.921 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.921 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 75/30 ECSubWrite(tid=101, reqid=osd.5.0:101, at_version=75'82, trim_to=0'0, roll_forward_to=68'78))", 2024-04-20T01:48:41.921 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.723236+0000", 2024-04-20T01:48:41.921 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.125542755000001, 2024-04-20T01:48:41.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010789544, 2024-04-20T01:48:41.921 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.921 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.922 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723236+0000", 2024-04-20T01:48:41.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723236+0000", 2024-04-20T01:48:41.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723235+0000", 2024-04-20T01:48:41.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:41.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723261+0000", 2024-04-20T01:48:41.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5381e-05 2024-04-20T01:48:41.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723279+0000", 2024-04-20T01:48:41.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7796e-05 2024-04-20T01:48:41.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723290+0000", 2024-04-20T01:48:41.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1739000000000001e-05 2024-04-20T01:48:41.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726374+0000", 2024-04-20T01:48:41.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003084023 2024-04-20T01:48:41.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726389+0000", 2024-04-20T01:48:41.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5041e-05 2024-04-20T01:48:41.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.733997+0000", 2024-04-20T01:48:41.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076079329999999999 2024-04-20T01:48:41.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.734026+0000", 2024-04-20T01:48:41.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8394e-05 2024-04-20T01:48:41.925 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.925 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.925 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.925 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 75/30 ECSubWrite(tid=113, reqid=osd.14.0:113, at_version=75'93, trim_to=0'0, roll_forward_to=74'89))", 2024-04-20T01:48:41.925 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.724246+0000", 2024-04-20T01:48:41.925 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.124532910999999, 2024-04-20T01:48:41.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097217950000000001, 2024-04-20T01:48:41.926 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:41.926 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:41.926 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:41.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:41.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724246+0000", 2024-04-20T01:48:41.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:41.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724246+0000", 2024-04-20T01:48:41.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:41.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:41.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724245+0000", 2024-04-20T01:48:41.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:41.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:41.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724270+0000", 2024-04-20T01:48:41.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4536999999999999e-05 2024-04-20T01:48:41.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:41.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724284+0000", 2024-04-20T01:48:41.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4421e-05 2024-04-20T01:48:41.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:41.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724292+0000", 2024-04-20T01:48:41.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6009999999999997e-06 2024-04-20T01:48:41.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:41.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726433+0000", 2024-04-20T01:48:41.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021418399999999999 2024-04-20T01:48:41.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:41.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726444+0000", 2024-04-20T01:48:41.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0671e-05 2024-04-20T01:48:41.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:41.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.733939+0000", 2024-04-20T01:48:41.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0074953249999999997 2024-04-20T01:48:41.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:41.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:41.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:41.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.733968+0000", 2024-04-20T01:48:41.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8368e-05 2024-04-20T01:48:41.929 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.929 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.929 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.929 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:41.929 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:41.929 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:41.931 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-20T01:48:41.931 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:48:41.945+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:48:42.031 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:42.046+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:42.101 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:42.101 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:42.101 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-04-20T01:48:42.101 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:42.113 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-20T01:48:42.133 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:42.147+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:42.233 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:42.247+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:42.242 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:42.242 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:42.242 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:48:42.243 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:48:42.243 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:42.253 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-20T01:48:42.334 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:42.349+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:42.425 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:48:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:48:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:48:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:104 3.es0 3:78f9000f:::smithi16579500-35:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:48:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.47585728, 2024-04-20T01:48:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012120811, 2024-04-20T01:48:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 104 2024-04-20T01:48:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:48:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:48:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:48:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962445+0000", 2024-04-20T01:48:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3210000000000001e-06 2024-04-20T01:48:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962451+0000", 2024-04-20T01:48:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.31e-06 2024-04-20T01:48:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962457+0000", 2024-04-20T01:48:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4819999999999999e-06 2024-04-20T01:48:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966434+0000", 2024-04-20T01:48:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039772239999999997 2024-04-20T01:48:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966501+0000", 2024-04-20T01:48:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6947999999999997e-05 2024-04-20T01:48:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966766+0000", 2024-04-20T01:48:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00026468999999999999 2024-04-20T01:48:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970614+0000", 2024-04-20T01:48:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0038481330000000001 2024-04-20T01:48:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974486+0000", 2024-04-20T01:48:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003871799 2024-04-20T01:48:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974563+0000", 2024-04-20T01:48:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7416999999999994e-05 2024-04-20T01:48:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:105 3.es0 3:78f9000f:::smithi16579500-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962461+0000", 2024-04-20T01:48:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.475839070000006, 2024-04-20T01:48:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014134643000000001, 2024-04-20T01:48:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 105 2024-04-20T01:48:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962461+0000", 2024-04-20T01:48:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962461+0000", 2024-04-20T01:48:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962460+0000", 2024-04-20T01:48:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962462+0000", 2024-04-20T01:48:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.897e-06 2024-04-20T01:48:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962466+0000", 2024-04-20T01:48:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4520000000000002e-06 2024-04-20T01:48:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962468+0000", 2024-04-20T01:48:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6570000000000001e-06 2024-04-20T01:48:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966956+0000", 2024-04-20T01:48:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044879730000000001 2024-04-20T01:48:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.967029+0000", 2024-04-20T01:48:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2657999999999995e-05 2024-04-20T01:48:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974602+0000", 2024-04-20T01:48:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075729370000000001 2024-04-20T01:48:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974650+0000", 2024-04-20T01:48:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8000000000000001e-05 2024-04-20T01:48:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976595+0000", 2024-04-20T01:48:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019453789999999999 2024-04-20T01:48:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:109 3.es0 3:7f925225:::smithi16579500-37:head [append 0~606208,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:48:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.475828638999999, 2024-04-20T01:48:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012453573000000001, 2024-04-20T01:48:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 109 2024-04-20T01:48:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:48:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:48:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:48:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962732+0000", 2024-04-20T01:48:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00026073200000000001 2024-04-20T01:48:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962742+0000", 2024-04-20T01:48:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0735e-05 2024-04-20T01:48:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962749+0000", 2024-04-20T01:48:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.365e-06 2024-04-20T01:48:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.967197+0000", 2024-04-20T01:48:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044487119999999996 2024-04-20T01:48:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.967324+0000", 2024-04-20T01:48:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126575 2024-04-20T01:48:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968044+0000", 2024-04-20T01:48:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00072015399999999998 2024-04-20T01:48:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970795+0000", 2024-04-20T01:48:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027507650000000001 2024-04-20T01:48:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974884+0000", 2024-04-20T01:48:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040886469999999999 2024-04-20T01:48:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974925+0000", 2024-04-20T01:48:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1094000000000003e-05 2024-04-20T01:48:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:110 3.es0 3:7f925225:::smithi16579500-37:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962755+0000", 2024-04-20T01:48:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.475544768999995, 2024-04-20T01:48:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012356723999999999, 2024-04-20T01:48:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 110 2024-04-20T01:48:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962755+0000", 2024-04-20T01:48:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962755+0000", 2024-04-20T01:48:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962754+0000", 2024-04-20T01:48:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962758+0000", 2024-04-20T01:48:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8750000000000002e-06 2024-04-20T01:48:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962765+0000", 2024-04-20T01:48:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2370000000000003e-06 2024-04-20T01:48:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962770+0000", 2024-04-20T01:48:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9030000000000003e-06 2024-04-20T01:48:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968614+0000", 2024-04-20T01:48:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0058437619999999997 2024-04-20T01:48:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968676+0000", 2024-04-20T01:48:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1592000000000003e-05 2024-04-20T01:48:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968935+0000", 2024-04-20T01:48:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025953599999999999 2024-04-20T01:48:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970828+0000", 2024-04-20T01:48:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0018928510000000001 2024-04-20T01:48:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975036+0000", 2024-04-20T01:48:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042079680000000003 2024-04-20T01:48:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975112+0000", 2024-04-20T01:48:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5558999999999999e-05 2024-04-20T01:48:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:111 3.es0 3:7f925225:::smithi16579500-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962775+0000", 2024-04-20T01:48:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.475525013999999, 2024-04-20T01:48:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014337756, 2024-04-20T01:48:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 111 2024-04-20T01:48:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962775+0000", 2024-04-20T01:48:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962775+0000", 2024-04-20T01:48:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962774+0000", 2024-04-20T01:48:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962777+0000", 2024-04-20T01:48:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7839999999999999e-06 2024-04-20T01:48:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962784+0000", 2024-04-20T01:48:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9809999999999997e-06 2024-04-20T01:48:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962790+0000", 2024-04-20T01:48:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5509999999999999e-06 2024-04-20T01:48:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970564+0000", 2024-04-20T01:48:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077740680000000003 2024-04-20T01:48:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970609+0000", 2024-04-20T01:48:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5059999999999999e-05 2024-04-20T01:48:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975165+0000", 2024-04-20T01:48:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0045564999999999998 2024-04-20T01:48:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975263+0000", 2024-04-20T01:48:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7689000000000006e-05 2024-04-20T01:48:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977112+0000", 2024-04-20T01:48:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001849552 2024-04-20T01:48:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:142 3.es0 3:797661d0:::smithi16579500-48:head [append 0~425984,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962794+0000", 2024-04-20T01:48:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.475505812999998, 2024-04-20T01:48:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013969364, 2024-04-20T01:48:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 142 2024-04-20T01:48:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962794+0000", 2024-04-20T01:48:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962794+0000", 2024-04-20T01:48:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962793+0000", 2024-04-20T01:48:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963014+0000", 2024-04-20T01:48:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000221015 2024-04-20T01:48:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963025+0000", 2024-04-20T01:48:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0739e-05 2024-04-20T01:48:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963032+0000", 2024-04-20T01:48:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7580000000000003e-06 2024-04-20T01:48:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.971338+0000", 2024-04-20T01:48:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008305659 2024-04-20T01:48:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.971477+0000", 2024-04-20T01:48:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013920100000000001 2024-04-20T01:48:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.972169+0000", 2024-04-20T01:48:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00069257800000000001 2024-04-20T01:48:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974577+0000", 2024-04-20T01:48:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024072820000000002 2024-04-20T01:48:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976694+0000", 2024-04-20T01:48:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021172629999999999 2024-04-20T01:48:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976763+0000", 2024-04-20T01:48:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9259999999999998e-05 2024-04-20T01:48:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:143 3.es0 3:797661d0:::smithi16579500-48:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.963039+0000", 2024-04-20T01:48:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.475261058000001, 2024-04-20T01:48:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013923471999999999, 2024-04-20T01:48:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 143 2024-04-20T01:48:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963039+0000", 2024-04-20T01:48:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963039+0000", 2024-04-20T01:48:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963038+0000", 2024-04-20T01:48:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963210+0000", 2024-04-20T01:48:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017155800000000001 2024-04-20T01:48:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963217+0000", 2024-04-20T01:48:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7309999999999993e-06 2024-04-20T01:48:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963230+0000", 2024-04-20T01:48:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.295e-05 2024-04-20T01:48:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.972654+0000", 2024-04-20T01:48:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094237369999999997 2024-04-20T01:48:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.972720+0000", 2024-04-20T01:48:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5958999999999996e-05 2024-04-20T01:48:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.973225+0000", 2024-04-20T01:48:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00050500399999999998 2024-04-20T01:48:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974578+0000", 2024-04-20T01:48:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001353295 2024-04-20T01:48:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976919+0000", 2024-04-20T01:48:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023405840000000002 2024-04-20T01:48:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976962+0000", 2024-04-20T01:48:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3226000000000003e-05 2024-04-20T01:48:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:144 3.es0 3:797661d0:::smithi16579500-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:48:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.475062424000001, 2024-04-20T01:48:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014456992, 2024-04-20T01:48:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 144 2024-04-20T01:48:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:48:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:48:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:48:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963240+0000", 2024-04-20T01:48:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6459999999999999e-06 2024-04-20T01:48:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963247+0000", 2024-04-20T01:48:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9650000000000002e-06 2024-04-20T01:48:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963253+0000", 2024-04-20T01:48:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1900000000000003e-06 2024-04-20T01:48:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.973649+0000", 2024-04-20T01:48:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010396865999999999 2024-04-20T01:48:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.973745+0000", 2024-04-20T01:48:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5345000000000003e-05 2024-04-20T01:48:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974151+0000", 2024-04-20T01:48:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000405893 2024-04-20T01:48:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976606+0000", 2024-04-20T01:48:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024551730000000002 2024-04-20T01:48:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977622+0000", 2024-04-20T01:48:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0010159570000000001 2024-04-20T01:48:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977694+0000", 2024-04-20T01:48:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2469e-05 2024-04-20T01:48:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:145 3.es0 3:797661d0:::smithi16579500-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:48:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.475040602999997, 2024-04-20T01:48:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016339408999999999, 2024-04-20T01:48:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 145 2024-04-20T01:48:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:48:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:48:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:48:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963267+0000", 2024-04-20T01:48:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2490000000000001e-06 2024-04-20T01:48:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963274+0000", 2024-04-20T01:48:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7599999999999997e-06 2024-04-20T01:48:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963279+0000", 2024-04-20T01:48:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5400000000000003e-06 2024-04-20T01:48:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977300+0000", 2024-04-20T01:48:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014020608 2024-04-20T01:48:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977365+0000", 2024-04-20T01:48:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5617999999999994e-05 2024-04-20T01:48:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.978057+0000", 2024-04-20T01:48:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00069179099999999998 2024-04-20T01:48:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.978136+0000", 2024-04-20T01:48:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9132000000000001e-05 2024-04-20T01:48:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.979598+0000", 2024-04-20T01:48:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0014621599999999999 2024-04-20T01:48:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:245 3.es0 3:78f9000f:::smithi16579500-35:head [copy-from ver 10] snapc 5=[2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785668+0000", 2024-04-20T01:48:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.652631417000002, 2024-04-20T01:48:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025155522, 2024-04-20T01:48:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 245 2024-04-20T01:48:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785668+0000", 2024-04-20T01:48:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785668+0000", 2024-04-20T01:48:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785666+0000", 2024-04-20T01:48:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785676+0000", 2024-04-20T01:48:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0139999999999999e-05 2024-04-20T01:48:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785692+0000", 2024-04-20T01:48:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5894999999999999e-05 2024-04-20T01:48:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785715+0000", 2024-04-20T01:48:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2654999999999999e-05 2024-04-20T01:48:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785771+0000", 2024-04-20T01:48:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6331e-05 2024-04-20T01:48:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785779+0000", 2024-04-20T01:48:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4789999999999999e-06 2024-04-20T01:48:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791776+0000", 2024-04-20T01:48:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059978840000000002 2024-04-20T01:48:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791905+0000", 2024-04-20T01:48:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000128147 2024-04-20T01:48:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.805603+0000", 2024-04-20T01:48:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013698824 2024-04-20T01:48:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.808612+0000", 2024-04-20T01:48:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003008541 2024-04-20T01:48:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.810754+0000", 2024-04-20T01:48:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021421840000000001 2024-04-20T01:48:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.810824+0000", 2024-04-20T01:48:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9627000000000003e-05 2024-04-20T01:48:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:246 3.es0 3:78f9000f:::smithi16579500-35:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:48:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.652571238, 2024-04-20T01:48:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020980167000000001, 2024-04-20T01:48:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 246 2024-04-20T01:48:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:48:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:48:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:48:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785731+0000", 2024-04-20T01:48:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3550000000000001e-06 2024-04-20T01:48:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785738+0000", 2024-04-20T01:48:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2080000000000003e-06 2024-04-20T01:48:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785747+0000", 2024-04-20T01:48:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.388e-06 2024-04-20T01:48:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785859+0000", 2024-04-20T01:48:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112432 2024-04-20T01:48:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785866+0000", 2024-04-20T01:48:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0890000000000002e-06 2024-04-20T01:48:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792104+0000", 2024-04-20T01:48:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062376970000000004 2024-04-20T01:48:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792157+0000", 2024-04-20T01:48:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3269000000000001e-05 2024-04-20T01:48:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.806521+0000", 2024-04-20T01:48:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014363651999999999 2024-04-20T01:48:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.806639+0000", 2024-04-20T01:48:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011755400000000001 2024-04-20T01:48:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.806709+0000", 2024-04-20T01:48:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0037999999999996e-05 2024-04-20T01:48:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:248 3.es0 3:79bd6b86:::smithi16579500-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:48:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.652515477999998, 2024-04-20T01:48:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011602988, 2024-04-20T01:48:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 248 2024-04-20T01:48:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:48:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:48:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:48:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785787+0000", 2024-04-20T01:48:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4079999999999998e-06 2024-04-20T01:48:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785791+0000", 2024-04-20T01:48:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9929999999999997e-06 2024-04-20T01:48:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785795+0000", 2024-04-20T01:48:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1559999999999997e-06 2024-04-20T01:48:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785936+0000", 2024-04-20T01:48:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014018 2024-04-20T01:48:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785939+0000", 2024-04-20T01:48:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-04-20T01:48:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794183+0000", 2024-04-20T01:48:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082434339999999991 2024-04-20T01:48:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794272+0000", 2024-04-20T01:48:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9246000000000006e-05 2024-04-20T01:48:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.797387+0000", 2024-04-20T01:48:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0031150269999999998 2024-04-20T01:48:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:249 3.es0 3:79bd6b86:::smithi16579500-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:48:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.652501180999998, 2024-04-20T01:48:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011999312, 2024-04-20T01:48:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 249 2024-04-20T01:48:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:48:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:48:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:48:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785800+0000", 2024-04-20T01:48:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.995e-06 2024-04-20T01:48:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785804+0000", 2024-04-20T01:48:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2990000000000001e-06 2024-04-20T01:48:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785807+0000", 2024-04-20T01:48:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.715e-06 2024-04-20T01:48:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785955+0000", 2024-04-20T01:48:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000147923 2024-04-20T01:48:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785958+0000", 2024-04-20T01:48:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8940000000000002e-06 2024-04-20T01:48:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794609+0000", 2024-04-20T01:48:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086509289999999999 2024-04-20T01:48:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.795125+0000", 2024-04-20T01:48:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00051564999999999998 2024-04-20T01:48:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.797798+0000", 2024-04-20T01:48:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0026731250000000002 2024-04-20T01:48:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:586 3.es0 3:79bd6b86:::smithi16579500-12:head [delete] snapc 20=[20,1f,1a,b] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e65)", 2024-04-20T01:48:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:06.762289+0000", 2024-04-20T01:48:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 35.676011101999997, 2024-04-20T01:48:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011713963000000001, 2024-04-20T01:48:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 586 2024-04-20T01:48:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762289+0000", 2024-04-20T01:48:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762289+0000", 2024-04-20T01:48:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762287+0000", 2024-04-20T01:48:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762294+0000", 2024-04-20T01:48:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6450000000000005e-06 2024-04-20T01:48:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762308+0000", 2024-04-20T01:48:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3652000000000001e-05 2024-04-20T01:48:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762318+0000", 2024-04-20T01:48:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6679999999999996e-06 2024-04-20T01:48:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762391+0000", 2024-04-20T01:48:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3823000000000001e-05 2024-04-20T01:48:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762400+0000", 2024-04-20T01:48:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6750000000000008e-06 2024-04-20T01:48:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.767267+0000", 2024-04-20T01:48:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0048666300000000003 2024-04-20T01:48:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.767389+0000", 2024-04-20T01:48:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012267699999999999 2024-04-20T01:48:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.774003+0000", 2024-04-20T01:48:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066131640000000004 2024-04-20T01:48:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:727 3.es0 3:797661d0:::smithi16579500-48:head [append 0~622592] snapc 26=[26,25,1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-20T01:48:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:11.814331+0000", 2024-04-20T01:48:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.623969070000001, 2024-04-20T01:48:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013760674000000001, 2024-04-20T01:48:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 727 2024-04-20T01:48:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814331+0000", 2024-04-20T01:48:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814331+0000", 2024-04-20T01:48:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814329+0000", 2024-04-20T01:48:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814688+0000", 2024-04-20T01:48:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00035975800000000001 2024-04-20T01:48:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814706+0000", 2024-04-20T01:48:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7248000000000002e-05 2024-04-20T01:48:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814716+0000", 2024-04-20T01:48:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9620000000000005e-06 2024-04-20T01:48:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814789+0000", 2024-04-20T01:48:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3734999999999998e-05 2024-04-20T01:48:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814800+0000", 2024-04-20T01:48:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0898e-05 2024-04-20T01:48:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.816951+0000", 2024-04-20T01:48:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021503379999999999 2024-04-20T01:48:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.817082+0000", 2024-04-20T01:48:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013159599999999999 2024-04-20T01:48:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.818406+0000", 2024-04-20T01:48:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013237749999999999 2024-04-20T01:48:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.822671+0000", 2024-04-20T01:48:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042655460000000003 2024-04-20T01:48:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.828006+0000", 2024-04-20T01:48:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005334999 2024-04-20T01:48:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.828091+0000", 2024-04-20T01:48:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4857000000000004e-05 2024-04-20T01:48:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:732 3.es0 3:797661d0:::smithi16579500-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e70)", 2024-04-20T01:48:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:11.823728+0000", 2024-04-20T01:48:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.614571918999999, 2024-04-20T01:48:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012035161000000001, 2024-04-20T01:48:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 732 2024-04-20T01:48:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823728+0000", 2024-04-20T01:48:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823728+0000", 2024-04-20T01:48:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823727+0000", 2024-04-20T01:48:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823731+0000", 2024-04-20T01:48:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8349999999999997e-06 2024-04-20T01:48:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823745+0000", 2024-04-20T01:48:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4211e-05 2024-04-20T01:48:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823756+0000", 2024-04-20T01:48:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0705e-05 2024-04-20T01:48:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.827650+0000", 2024-04-20T01:48:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003894163 2024-04-20T01:48:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.827758+0000", 2024-04-20T01:48:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107759 2024-04-20T01:48:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.834824+0000", 2024-04-20T01:48:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.007066587 2024-04-20T01:48:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.834920+0000", 2024-04-20T01:48:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5639999999999999e-05 2024-04-20T01:48:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.835763+0000", 2024-04-20T01:48:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000842835 2024-04-20T01:48:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:1007 3.es0 3:797661d0:::smithi16579500-48:head [read 0~6258688,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:31.807916+0000", 2024-04-20T01:48:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.630384165000001, 2024-04-20T01:48:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011898517000000001, 2024-04-20T01:48:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1007 2024-04-20T01:48:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807916+0000", 2024-04-20T01:48:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807916+0000", 2024-04-20T01:48:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807913+0000", 2024-04-20T01:48:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807935+0000", 2024-04-20T01:48:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1943e-05 2024-04-20T01:48:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807962+0000", 2024-04-20T01:48:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6199000000000001e-05 2024-04-20T01:48:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807976+0000", 2024-04-20T01:48:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4029000000000001e-05 2024-04-20T01:48:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808105+0000", 2024-04-20T01:48:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012951299999999999 2024-04-20T01:48:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808234+0000", 2024-04-20T01:48:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000129068 2024-04-20T01:48:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.819814+0000", 2024-04-20T01:48:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011579789 2024-04-20T01:48:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:1008 3.es0 3:797661d0:::smithi16579500-48:head [read 0~6258688,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:31.807999+0000", 2024-04-20T01:48:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.630301117, 2024-04-20T01:48:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020846726999999999, 2024-04-20T01:48:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1008 2024-04-20T01:48:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807999+0000", 2024-04-20T01:48:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807999+0000", 2024-04-20T01:48:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807998+0000", 2024-04-20T01:48:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808003+0000", 2024-04-20T01:48:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0919999999999997e-06 2024-04-20T01:48:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808014+0000", 2024-04-20T01:48:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1331999999999999e-05 2024-04-20T01:48:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808021+0000", 2024-04-20T01:48:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0559999999999999e-06 2024-04-20T01:48:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808626+0000", 2024-04-20T01:48:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000604493 2024-04-20T01:48:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808697+0000", 2024-04-20T01:48:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1711000000000005e-05 2024-04-20T01:48:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.828845+0000", 2024-04-20T01:48:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020147805000000001 2024-04-20T01:48:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:1009 3.es0 3:797661d0:::smithi16579500-48:head [read 0~6258688,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:31.808028+0000", 2024-04-20T01:48:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.630271646000001, 2024-04-20T01:48:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.026288720000000002, 2024-04-20T01:48:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1009 2024-04-20T01:48:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808028+0000", 2024-04-20T01:48:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808028+0000", 2024-04-20T01:48:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808027+0000", 2024-04-20T01:48:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808032+0000", 2024-04-20T01:48:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1459999999999998e-06 2024-04-20T01:48:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808038+0000", 2024-04-20T01:48:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.494e-06 2024-04-20T01:48:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808044+0000", 2024-04-20T01:48:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2770000000000002e-06 2024-04-20T01:48:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808958+0000", 2024-04-20T01:48:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00091398300000000005 2024-04-20T01:48:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.809027+0000", 2024-04-20T01:48:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8675000000000002e-05 2024-04-20T01:48:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.834317+0000", 2024-04-20T01:48:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025289696 2024-04-20T01:48:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpReadReply(3.es0 90/30 ECSubReadReply(tid=129, attrs_read=0))", 2024-04-20T01:48:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:31.816185+0000", 2024-04-20T01:48:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.622114422999999, 2024-04-20T01:48:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012799764, 2024-04-20T01:48:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.816185+0000", 2024-04-20T01:48:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.816185+0000", 2024-04-20T01:48:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.816185+0000", 2024-04-20T01:48:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.824675+0000", 2024-04-20T01:48:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0084907639999999996 2024-04-20T01:48:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.824690+0000", 2024-04-20T01:48:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4503e-05 2024-04-20T01:48:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.824700+0000", 2024-04-20T01:48:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7820000000000005e-06 2024-04-20T01:48:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.824734+0000", 2024-04-20T01:48:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4072000000000002e-05 2024-04-20T01:48:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.828985+0000", 2024-04-20T01:48:42.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042512629999999999 2024-04-20T01:48:42.524 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.524 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.524 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.524 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:42.524 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:42.524 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:42.525 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-20T01:48:42.526 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:42.540+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:42.626 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:42.640+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:42.642 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [14] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 3, 7, 11, 15] 2024-04-20T01:48:42.642 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-20T01:48:42.642 INFO:tasks.thrashosds.thrasher:inject_pause on osd.13 2024-04-20T01:48:42.642 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-20T01:48:42.642 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-20T01:48:42.642 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok config set filestore_inject_stall 3 2024-04-20T01:48:42.655 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:42.655 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:42.655 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-04-20T01:48:42.655 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:42.665 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-20T01:48:42.750 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:42.764+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:42.784 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:42.784 INFO:teuthology.orchestra.run.smithi121.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-04-20T01:48:42.785 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:42.828 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:48:42.841+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:48:42.835 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:42.835 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:42.835 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:48:42.835 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:48:42.835 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:42.845 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-20T01:48:42.929 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:42.943+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:43.015 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:43.015 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:48:43.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:48:43.016 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:48:43.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.016 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 37/30 ECSubWrite(tid=43, reqid=client.4643.0:241, at_version=37'31, trim_to=0'0, roll_forward_to=36'28))", 2024-04-20T01:48:43.016 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.793501+0000", 2024-04-20T01:48:43.016 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 64.235585732999994, 2024-04-20T01:48:43.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005847517, 2024-04-20T01:48:43.016 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:43.016 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:43.016 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:43.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:43.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793501+0000", 2024-04-20T01:48:43.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:43.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793501+0000", 2024-04-20T01:48:43.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:43.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793500+0000", 2024-04-20T01:48:43.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:43.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:43.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793525+0000", 2024-04-20T01:48:43.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4641999999999999e-05 2024-04-20T01:48:43.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:43.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793537+0000", 2024-04-20T01:48:43.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2626000000000001e-05 2024-04-20T01:48:43.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:43.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793547+0000", 2024-04-20T01:48:43.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.842e-06 2024-04-20T01:48:43.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:43.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793599+0000", 2024-04-20T01:48:43.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2301e-05 2024-04-20T01:48:43.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:43.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793607+0000", 2024-04-20T01:48:43.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9070000000000007e-06 2024-04-20T01:48:43.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:43.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799321+0000", 2024-04-20T01:48:43.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005713358 2024-04-20T01:48:43.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:43.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799349+0000", 2024-04-20T01:48:43.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7832999999999999e-05 2024-04-20T01:48:43.020 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.020 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.020 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.020 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 37/30 ECSubWrite(tid=19, reqid=client.4643.0:235, at_version=37'9, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:48:43.020 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.793816+0000", 2024-04-20T01:48:43.020 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 64.235270258, 2024-04-20T01:48:43.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0055550790000000001, 2024-04-20T01:48:43.020 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:43.020 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:43.020 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:43.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:43.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793816+0000", 2024-04-20T01:48:43.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:43.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793816+0000", 2024-04-20T01:48:43.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:43.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793816+0000", 2024-04-20T01:48:43.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:43.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:43.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793838+0000", 2024-04-20T01:48:43.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2075000000000001e-05 2024-04-20T01:48:43.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:43.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793849+0000", 2024-04-20T01:48:43.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1433999999999999e-05 2024-04-20T01:48:43.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:43.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793859+0000", 2024-04-20T01:48:43.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4849999999999998e-06 2024-04-20T01:48:43.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:43.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793956+0000", 2024-04-20T01:48:43.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7114000000000006e-05 2024-04-20T01:48:43.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:43.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793965+0000", 2024-04-20T01:48:43.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0820000000000005e-06 2024-04-20T01:48:43.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:43.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799344+0000", 2024-04-20T01:48:43.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053794400000000001 2024-04-20T01:48:43.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:43.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799372+0000", 2024-04-20T01:48:43.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7231000000000001e-05 2024-04-20T01:48:43.024 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.024 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.024 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.025 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 38/30 ECSubWrite(tid=23, reqid=osd.3.0:22, at_version=0'0, trim_to=0'0, roll_forward_to=38'12))", 2024-04-20T01:48:43.025 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.788077+0000", 2024-04-20T01:48:43.025 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 63.241010203999998, 2024-04-20T01:48:43.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057844480000000002, 2024-04-20T01:48:43.025 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:43.025 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:43.025 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:43.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:43.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788077+0000", 2024-04-20T01:48:43.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:43.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788077+0000", 2024-04-20T01:48:43.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:43.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788076+0000", 2024-04-20T01:48:43.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:43.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:43.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788088+0000", 2024-04-20T01:48:43.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2672e-05 2024-04-20T01:48:43.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:43.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788101+0000", 2024-04-20T01:48:43.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2468e-05 2024-04-20T01:48:43.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:43.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788121+0000", 2024-04-20T01:48:43.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0315000000000001e-05 2024-04-20T01:48:43.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:43.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788181+0000", 2024-04-20T01:48:43.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0257999999999998e-05 2024-04-20T01:48:43.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:43.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788194+0000", 2024-04-20T01:48:43.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2714000000000001e-05 2024-04-20T01:48:43.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:43.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.793811+0000", 2024-04-20T01:48:43.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056169649999999998 2024-04-20T01:48:43.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:43.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.793861+0000", 2024-04-20T01:48:43.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9978000000000002e-05 2024-04-20T01:48:43.031 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.031 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.031 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.032 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 53/30 ECSubWrite(tid=32, reqid=client.4643.0:443, at_version=53'21, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:43.032 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.718947+0000", 2024-04-20T01:48:43.032 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 48.310139235999998, 2024-04-20T01:48:43.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009785129, 2024-04-20T01:48:43.032 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:43.032 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:43.032 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:43.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:43.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718947+0000", 2024-04-20T01:48:43.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:43.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718947+0000", 2024-04-20T01:48:43.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:43.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718947+0000", 2024-04-20T01:48:43.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:43.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:43.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719111+0000", 2024-04-20T01:48:43.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000164227 2024-04-20T01:48:43.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:43.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719125+0000", 2024-04-20T01:48:43.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.454e-05 2024-04-20T01:48:43.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:43.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719137+0000", 2024-04-20T01:48:43.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1783999999999999e-05 2024-04-20T01:48:43.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:43.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719206+0000", 2024-04-20T01:48:43.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9123999999999998e-05 2024-04-20T01:48:43.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:43.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719221+0000", 2024-04-20T01:48:43.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4963999999999999e-05 2024-04-20T01:48:43.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:43.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.728704+0000", 2024-04-20T01:48:43.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094822859999999995 2024-04-20T01:48:43.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:43.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.728733+0000", 2024-04-20T01:48:43.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8972e-05 2024-04-20T01:48:43.036 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.036 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.036 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.036 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 53/30 ECSubWrite(tid=33, reqid=client.4643.0:444, at_version=53'22, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:43.036 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.721427+0000", 2024-04-20T01:48:43.036 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 48.307659985999997, 2024-04-20T01:48:43.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010045943999999999, 2024-04-20T01:48:43.036 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:43.036 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:43.036 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:43.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:43.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721427+0000", 2024-04-20T01:48:43.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:43.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721427+0000", 2024-04-20T01:48:43.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:43.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721424+0000", 2024-04-20T01:48:43.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999967 2024-04-20T01:48:43.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:43.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721541+0000", 2024-04-20T01:48:43.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117309 2024-04-20T01:48:43.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:43.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721560+0000", 2024-04-20T01:48:43.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8559999999999998e-05 2024-04-20T01:48:43.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:43.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721571+0000", 2024-04-20T01:48:43.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1612e-05 2024-04-20T01:48:43.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:43.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721648+0000", 2024-04-20T01:48:43.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6534999999999998e-05 2024-04-20T01:48:43.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:43.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721660+0000", 2024-04-20T01:48:43.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2396e-05 2024-04-20T01:48:43.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:43.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731447+0000", 2024-04-20T01:48:43.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097865039999999997 2024-04-20T01:48:43.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:43.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731473+0000", 2024-04-20T01:48:43.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6129999999999999e-05 2024-04-20T01:48:43.040 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.040 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.040 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.040 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 53/30 ECSubWrite(tid=34, reqid=client.4643.0:445, at_version=53'23, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:43.040 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.722888+0000", 2024-04-20T01:48:43.040 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 48.306198506000001, 2024-04-20T01:48:43.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086321880000000007, 2024-04-20T01:48:43.041 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:43.041 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:43.041 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:43.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:43.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722888+0000", 2024-04-20T01:48:43.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:43.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722888+0000", 2024-04-20T01:48:43.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:43.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722887+0000", 2024-04-20T01:48:43.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:43.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:43.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722958+0000", 2024-04-20T01:48:43.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0167999999999996e-05 2024-04-20T01:48:43.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:43.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722970+0000", 2024-04-20T01:48:43.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2393e-05 2024-04-20T01:48:43.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:43.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722979+0000", 2024-04-20T01:48:43.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1509999999999993e-06 2024-04-20T01:48:43.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:43.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723935+0000", 2024-04-20T01:48:43.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00095590900000000001 2024-04-20T01:48:43.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:43.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723947+0000", 2024-04-20T01:48:43.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2405999999999999e-05 2024-04-20T01:48:43.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:43.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731503+0000", 2024-04-20T01:48:43.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075554990000000002 2024-04-20T01:48:43.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:43.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731520+0000", 2024-04-20T01:48:43.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7490999999999999e-05 2024-04-20T01:48:43.044 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.044 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.044 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.045 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 53/30 ECSubWrite(tid=35, reqid=client.4643.0:446, at_version=53'24, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:43.045 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.724462+0000", 2024-04-20T01:48:43.045 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 48.304624427, 2024-04-20T01:48:43.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0070986110000000003, 2024-04-20T01:48:43.045 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:43.045 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:43.045 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:43.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:43.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724462+0000", 2024-04-20T01:48:43.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:43.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724462+0000", 2024-04-20T01:48:43.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:43.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724461+0000", 2024-04-20T01:48:43.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:43.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:43.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724482+0000", 2024-04-20T01:48:43.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0557999999999999e-05 2024-04-20T01:48:43.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:43.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724502+0000", 2024-04-20T01:48:43.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9704999999999999e-05 2024-04-20T01:48:43.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:43.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724511+0000", 2024-04-20T01:48:43.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8699999999999998e-06 2024-04-20T01:48:43.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:43.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724537+0000", 2024-04-20T01:48:43.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.688e-05 2024-04-20T01:48:43.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:43.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724543+0000", 2024-04-20T01:48:43.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9909999999999999e-06 2024-04-20T01:48:43.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:43.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731548+0000", 2024-04-20T01:48:43.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0070045990000000002 2024-04-20T01:48:43.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:43.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731561+0000", 2024-04-20T01:48:43.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2857000000000001e-05 2024-04-20T01:48:43.048 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.048 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.049 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.049 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs2 55/30 ECSubWrite(tid=42, reqid=osd.2.0:42, at_version=55'26, trim_to=0'0, roll_forward_to=52'21))", 2024-04-20T01:48:43.049 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:56.731259+0000", 2024-04-20T01:48:43.049 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 46.297827699999999, 2024-04-20T01:48:43.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0064248020000000003, 2024-04-20T01:48:43.049 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:43.049 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:43.049 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:43.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:43.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731259+0000", 2024-04-20T01:48:43.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:43.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731259+0000", 2024-04-20T01:48:43.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:43.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731259+0000", 2024-04-20T01:48:43.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:43.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:43.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731283+0000", 2024-04-20T01:48:43.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4437e-05 2024-04-20T01:48:43.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:43.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731292+0000", 2024-04-20T01:48:43.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0880000000000001e-06 2024-04-20T01:48:43.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:43.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731300+0000", 2024-04-20T01:48:43.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8599999999999993e-06 2024-04-20T01:48:43.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:43.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731742+0000", 2024-04-20T01:48:43.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044220700000000002 2024-04-20T01:48:43.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:43.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731753+0000", 2024-04-20T01:48:43.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0968000000000001e-05 2024-04-20T01:48:43.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:43.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.737642+0000", 2024-04-20T01:48:43.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0058890239999999996 2024-04-20T01:48:43.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:43.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.737684+0000", 2024-04-20T01:48:43.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1683000000000001e-05 2024-04-20T01:48:43.053 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.053 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.053 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.053 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 55/30 ECSubWrite(tid=41, reqid=osd.3.0:41, at_version=55'30, trim_to=0'0, roll_forward_to=54'26))", 2024-04-20T01:48:43.053 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:56.732670+0000", 2024-04-20T01:48:43.053 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 46.296416688999997, 2024-04-20T01:48:43.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080432620000000007, 2024-04-20T01:48:43.053 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:43.053 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:43.053 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:43.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:43.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732670+0000", 2024-04-20T01:48:43.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:43.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732670+0000", 2024-04-20T01:48:43.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:43.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732669+0000", 2024-04-20T01:48:43.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:43.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:43.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732698+0000", 2024-04-20T01:48:43.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8818000000000001e-05 2024-04-20T01:48:43.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:43.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732711+0000", 2024-04-20T01:48:43.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2696e-05 2024-04-20T01:48:43.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:43.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732721+0000", 2024-04-20T01:48:43.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0006e-05 2024-04-20T01:48:43.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:43.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732805+0000", 2024-04-20T01:48:43.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4152000000000003e-05 2024-04-20T01:48:43.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:43.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732814+0000", 2024-04-20T01:48:43.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7110000000000007e-06 2024-04-20T01:48:43.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:43.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.740690+0000", 2024-04-20T01:48:43.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0078757670000000005 2024-04-20T01:48:43.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:43.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.740713+0000", 2024-04-20T01:48:43.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3141000000000001e-05 2024-04-20T01:48:43.057 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.057 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.057 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.057 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 55/30 ECSubWrite(tid=42, reqid=client.4643.0:451, at_version=55'31, trim_to=0'0, roll_forward_to=55'28))", 2024-04-20T01:48:43.057 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:56.739643+0000", 2024-04-20T01:48:43.057 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 46.289443994999999, 2024-04-20T01:48:43.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056333429999999999, 2024-04-20T01:48:43.057 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:43.057 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:43.058 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:43.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:43.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739643+0000", 2024-04-20T01:48:43.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:43.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739643+0000", 2024-04-20T01:48:43.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:43.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739641+0000", 2024-04-20T01:48:43.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:43.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:43.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739813+0000", 2024-04-20T01:48:43.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017258700000000001 2024-04-20T01:48:43.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:43.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739833+0000", 2024-04-20T01:48:43.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9312000000000001e-05 2024-04-20T01:48:43.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:43.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739844+0000", 2024-04-20T01:48:43.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1303e-05 2024-04-20T01:48:43.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:43.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.740000+0000", 2024-04-20T01:48:43.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015592899999999999 2024-04-20T01:48:43.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:43.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.740013+0000", 2024-04-20T01:48:43.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2659e-05 2024-04-20T01:48:43.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:43.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.745246+0000", 2024-04-20T01:48:43.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0052337290000000003 2024-04-20T01:48:43.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:43.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.745276+0000", 2024-04-20T01:48:43.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9688000000000001e-05 2024-04-20T01:48:43.061 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.061 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.061 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.061 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 58/57 ECSubWrite(tid=121, reqid=client.4643.0:485, at_version=58'81, trim_to=0'0, roll_forward_to=58'78))", 2024-04-20T01:48:43.061 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:59.819732+0000", 2024-04-20T01:48:43.062 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 43.209355217999999, 2024-04-20T01:48:43.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053001919999999996, 2024-04-20T01:48:43.062 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:43.062 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:43.062 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:43.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:43.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819732+0000", 2024-04-20T01:48:43.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:43.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819732+0000", 2024-04-20T01:48:43.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:43.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819731+0000", 2024-04-20T01:48:43.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:43.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:43.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819761+0000", 2024-04-20T01:48:43.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0620999999999997e-05 2024-04-20T01:48:43.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:43.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819775+0000", 2024-04-20T01:48:43.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4042999999999999e-05 2024-04-20T01:48:43.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:43.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819785+0000", 2024-04-20T01:48:43.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9369999999999999e-06 2024-04-20T01:48:43.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:43.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819822+0000", 2024-04-20T01:48:43.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6430000000000002e-05 2024-04-20T01:48:43.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:43.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819829+0000", 2024-04-20T01:48:43.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0299999999999996e-06 2024-04-20T01:48:43.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:43.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.825010+0000", 2024-04-20T01:48:43.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0051810900000000002 2024-04-20T01:48:43.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:43.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.825032+0000", 2024-04-20T01:48:43.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1759e-05 2024-04-20T01:48:43.065 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.065 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.065 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.066 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 60/57 ECSubWrite(tid=139, reqid=osd.8.0:138, at_version=0'0, trim_to=0'0, roll_forward_to=60'83))", 2024-04-20T01:48:43.066 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:01.785138+0000", 2024-04-20T01:48:43.066 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 41.243948705000001, 2024-04-20T01:48:43.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0054622710000000003, 2024-04-20T01:48:43.066 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:43.066 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:43.066 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:43.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:43.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785138+0000", 2024-04-20T01:48:43.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:43.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785138+0000", 2024-04-20T01:48:43.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:43.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785137+0000", 2024-04-20T01:48:43.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:43.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:43.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785149+0000", 2024-04-20T01:48:43.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1198e-05 2024-04-20T01:48:43.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:43.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785160+0000", 2024-04-20T01:48:43.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066e-05 2024-04-20T01:48:43.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:43.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785168+0000", 2024-04-20T01:48:43.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4880000000000007e-06 2024-04-20T01:48:43.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:43.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785217+0000", 2024-04-20T01:48:43.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8952000000000002e-05 2024-04-20T01:48:43.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:43.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785228+0000", 2024-04-20T01:48:43.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0915e-05 2024-04-20T01:48:43.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:43.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.790582+0000", 2024-04-20T01:48:43.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053536649999999996 2024-04-20T01:48:43.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:43.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.790600+0000", 2024-04-20T01:48:43.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8677000000000001e-05 2024-04-20T01:48:43.069 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.070 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.070 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.070 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 65/57 ECSubWrite(tid=165, reqid=osd.8.0:165, at_version=65'111, trim_to=0'0, roll_forward_to=65'107))", 2024-04-20T01:48:43.070 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:06.783738+0000", 2024-04-20T01:48:43.070 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 36.245348802000002, 2024-04-20T01:48:43.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053461910000000001, 2024-04-20T01:48:43.070 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:43.070 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:43.070 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:43.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:43.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783738+0000", 2024-04-20T01:48:43.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:43.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783738+0000", 2024-04-20T01:48:43.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:43.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783737+0000", 2024-04-20T01:48:43.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:43.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:43.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783756+0000", 2024-04-20T01:48:43.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8454e-05 2024-04-20T01:48:43.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:43.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783768+0000", 2024-04-20T01:48:43.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1950000000000001e-05 2024-04-20T01:48:43.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:43.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783777+0000", 2024-04-20T01:48:43.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0850000000000003e-06 2024-04-20T01:48:43.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:43.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783815+0000", 2024-04-20T01:48:43.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7857e-05 2024-04-20T01:48:43.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:43.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783822+0000", 2024-04-20T01:48:43.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1130000000000003e-06 2024-04-20T01:48:43.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:43.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.789064+0000", 2024-04-20T01:48:43.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0052421009999999999 2024-04-20T01:48:43.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:43.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.789084+0000", 2024-04-20T01:48:43.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0395e-05 2024-04-20T01:48:43.074 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.074 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.074 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.074 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 75/57 ECSubWrite(tid=185, reqid=osd.8.0:185, at_version=75'128, trim_to=0'0, roll_forward_to=68'123))", 2024-04-20T01:48:43.074 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.718372+0000", 2024-04-20T01:48:43.074 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 26.310714259000001, 2024-04-20T01:48:43.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.006709179, 2024-04-20T01:48:43.074 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:43.074 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:43.074 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:43.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:43.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718372+0000", 2024-04-20T01:48:43.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:43.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718372+0000", 2024-04-20T01:48:43.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:43.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718372+0000", 2024-04-20T01:48:43.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:43.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:43.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718398+0000", 2024-04-20T01:48:43.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5945e-05 2024-04-20T01:48:43.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:43.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718408+0000", 2024-04-20T01:48:43.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5540000000000003e-06 2024-04-20T01:48:43.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:43.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718416+0000", 2024-04-20T01:48:43.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4030000000000007e-06 2024-04-20T01:48:43.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:43.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718988+0000", 2024-04-20T01:48:43.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00057154199999999997 2024-04-20T01:48:43.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:43.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718999+0000", 2024-04-20T01:48:43.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1499e-05 2024-04-20T01:48:43.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:43.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725052+0000", 2024-04-20T01:48:43.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0060528350000000003 2024-04-20T01:48:43.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:43.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725082+0000", 2024-04-20T01:48:43.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9816999999999999e-05 2024-04-20T01:48:43.078 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.078 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.078 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.078 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs2 75/30 ECSubWrite(tid=64, reqid=osd.2.0:64, at_version=75'48, trim_to=0'0, roll_forward_to=73'46))", 2024-04-20T01:48:43.078 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.720225+0000", 2024-04-20T01:48:43.078 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 26.308862213000001, 2024-04-20T01:48:43.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0058277789999999999, 2024-04-20T01:48:43.078 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:43.078 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:43.079 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:43.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:43.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720225+0000", 2024-04-20T01:48:43.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:43.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720225+0000", 2024-04-20T01:48:43.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:43.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720224+0000", 2024-04-20T01:48:43.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:43.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:43.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720253+0000", 2024-04-20T01:48:43.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9485e-05 2024-04-20T01:48:43.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:43.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720280+0000", 2024-04-20T01:48:43.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.69e-05 2024-04-20T01:48:43.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:43.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720291+0000", 2024-04-20T01:48:43.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0589e-05 2024-04-20T01:48:43.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:43.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720522+0000", 2024-04-20T01:48:43.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023163799999999999 2024-04-20T01:48:43.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:43.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720532+0000", 2024-04-20T01:48:43.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7499999999999998e-06 2024-04-20T01:48:43.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:43.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726006+0000", 2024-04-20T01:48:43.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0054741770000000002 2024-04-20T01:48:43.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:43.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726052+0000", 2024-04-20T01:48:43.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.613e-05 2024-04-20T01:48:43.082 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.082 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.082 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.082 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs2 75/30 ECSubWrite(tid=65, reqid=osd.2.0:65, at_version=75'50, trim_to=0'0, roll_forward_to=73'46))", 2024-04-20T01:48:43.082 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.720305+0000", 2024-04-20T01:48:43.082 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 26.308781577000001, 2024-04-20T01:48:43.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0060264209999999997, 2024-04-20T01:48:43.082 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:43.083 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:43.083 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:43.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:43.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720305+0000", 2024-04-20T01:48:43.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:43.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720305+0000", 2024-04-20T01:48:43.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:43.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720305+0000", 2024-04-20T01:48:43.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:43.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:43.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720327+0000", 2024-04-20T01:48:43.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2181e-05 2024-04-20T01:48:43.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:43.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720335+0000", 2024-04-20T01:48:43.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7710000000000006e-06 2024-04-20T01:48:43.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:43.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720341+0000", 2024-04-20T01:48:43.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4690000000000002e-06 2024-04-20T01:48:43.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:43.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722262+0000", 2024-04-20T01:48:43.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001920843 2024-04-20T01:48:43.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:43.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722272+0000", 2024-04-20T01:48:43.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0481e-05 2024-04-20T01:48:43.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:43.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726304+0000", 2024-04-20T01:48:43.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040320979999999996 2024-04-20T01:48:43.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:43.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726332+0000", 2024-04-20T01:48:43.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7148000000000002e-05 2024-04-20T01:48:43.086 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.086 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.086 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.086 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 75/57 ECSubWrite(tid=188, reqid=osd.8.0:188, at_version=75'132, trim_to=0'0, roll_forward_to=75'128))", 2024-04-20T01:48:43.086 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.730972+0000", 2024-04-20T01:48:43.086 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 26.298114868999999, 2024-04-20T01:48:43.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059973420000000001, 2024-04-20T01:48:43.087 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:43.087 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:43.087 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:43.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:43.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.730972+0000", 2024-04-20T01:48:43.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:43.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.730972+0000", 2024-04-20T01:48:43.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:43.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.730971+0000", 2024-04-20T01:48:43.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:43.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:43.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731000+0000", 2024-04-20T01:48:43.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8756e-05 2024-04-20T01:48:43.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:43.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731009+0000", 2024-04-20T01:48:43.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6780000000000006e-06 2024-04-20T01:48:43.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:43.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731017+0000", 2024-04-20T01:48:43.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9880000000000004e-06 2024-04-20T01:48:43.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:43.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.732017+0000", 2024-04-20T01:48:43.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00099972800000000007 2024-04-20T01:48:43.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:43.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.732026+0000", 2024-04-20T01:48:43.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0299999999999999e-06 2024-04-20T01:48:43.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:43.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736942+0000", 2024-04-20T01:48:43.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0049165370000000003 2024-04-20T01:48:43.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:43.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736969+0000", 2024-04-20T01:48:43.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7005e-05 2024-04-20T01:48:43.090 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.090 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.090 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.090 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 75/30 ECSubWrite(tid=55, reqid=client.4643.0:781, at_version=75'41, trim_to=0'0, roll_forward_to=66'39))", 2024-04-20T01:48:43.090 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.735309+0000", 2024-04-20T01:48:43.091 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 26.293777758000001, 2024-04-20T01:48:43.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005306926, 2024-04-20T01:48:43.091 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:43.091 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:43.091 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:43.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:43.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735309+0000", 2024-04-20T01:48:43.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:43.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735309+0000", 2024-04-20T01:48:43.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:43.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735309+0000", 2024-04-20T01:48:43.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:43.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:43.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735445+0000", 2024-04-20T01:48:43.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013668299999999999 2024-04-20T01:48:43.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:43.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735459+0000", 2024-04-20T01:48:43.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3370000000000001e-05 2024-04-20T01:48:43.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:43.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735469+0000", 2024-04-20T01:48:43.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8700000000000004e-06 2024-04-20T01:48:43.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:43.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735568+0000", 2024-04-20T01:48:43.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9122000000000004e-05 2024-04-20T01:48:43.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:43.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735579+0000", 2024-04-20T01:48:43.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1294999999999999e-05 2024-04-20T01:48:43.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:43.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.740581+0000", 2024-04-20T01:48:43.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0050022410000000002 2024-04-20T01:48:43.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:43.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.740616+0000", 2024-04-20T01:48:43.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4706000000000003e-05 2024-04-20T01:48:43.094 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.094 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.094 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.095 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 87/30 ECSubWrite(tid=83, reqid=osd.3.0:83, at_version=87'59, trim_to=0'0, roll_forward_to=85'57))", 2024-04-20T01:48:43.095 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:28.775123+0000", 2024-04-20T01:48:43.095 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.253963276, 2024-04-20T01:48:43.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075716259999999997, 2024-04-20T01:48:43.095 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:43.095 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:43.095 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:43.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:43.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.775123+0000", 2024-04-20T01:48:43.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:43.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.775123+0000", 2024-04-20T01:48:43.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:43.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.775123+0000", 2024-04-20T01:48:43.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:43.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:43.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.775159+0000", 2024-04-20T01:48:43.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6084999999999998e-05 2024-04-20T01:48:43.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:43.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.775174+0000", 2024-04-20T01:48:43.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4921e-05 2024-04-20T01:48:43.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:43.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.775185+0000", 2024-04-20T01:48:43.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1277999999999999e-05 2024-04-20T01:48:43.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:43.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.775257+0000", 2024-04-20T01:48:43.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1247000000000004e-05 2024-04-20T01:48:43.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:43.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.775268+0000", 2024-04-20T01:48:43.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.137e-05 2024-04-20T01:48:43.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:43.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.782664+0000", 2024-04-20T01:48:43.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0073960780000000004 2024-04-20T01:48:43.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:43.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.782695+0000", 2024-04-20T01:48:43.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0978000000000001e-05 2024-04-20T01:48:43.098 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.098 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.099 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.099 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 87/30 ECSubWrite(tid=84, reqid=osd.3.0:84, at_version=87'61, trim_to=0'0, roll_forward_to=85'57))", 2024-04-20T01:48:43.099 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:28.776211+0000", 2024-04-20T01:48:43.099 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.252876150000001, 2024-04-20T01:48:43.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0096891890000000008, 2024-04-20T01:48:43.099 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:43.099 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:43.099 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:43.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:43.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.776211+0000", 2024-04-20T01:48:43.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:43.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.776211+0000", 2024-04-20T01:48:43.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:43.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:43.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.776210+0000", 2024-04-20T01:48:43.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:43.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:43.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.776231+0000", 2024-04-20T01:48:43.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0735999999999999e-05 2024-04-20T01:48:43.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:43.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.776248+0000", 2024-04-20T01:48:43.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7459999999999999e-05 2024-04-20T01:48:43.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:43.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.776258+0000", 2024-04-20T01:48:43.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0044000000000001e-05 2024-04-20T01:48:43.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:43.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.776524+0000", 2024-04-20T01:48:43.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000265665 2024-04-20T01:48:43.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:43.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.776534+0000", 2024-04-20T01:48:43.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0152e-05 2024-04-20T01:48:43.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:43.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.785821+0000", 2024-04-20T01:48:43.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092868589999999997 2024-04-20T01:48:43.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:43.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:43.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:43.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.785900+0000", 2024-04-20T01:48:43.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9059000000000003e-05 2024-04-20T01:48:43.103 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.103 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.103 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.103 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:43.103 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:43.103 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:43.104 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-20T01:48:43.105 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:43.118+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:43.205 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:43.219+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:43.253 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-20T01:48:43.258 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:48:43.258 INFO:teuthology.orchestra.run.smithi044.stdout: "ops": [], 2024-04-20T01:48:43.258 INFO:teuthology.orchestra.run.smithi044.stdout: "num_ops": 0 2024-04-20T01:48:43.258 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:48:43.271 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-20T01:48:43.305 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-04-20T01:48:43.319+0000 7fa9375d0700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 74537) UID: 0 2024-04-20T01:48:43.407 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:48:43.421+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:48:43.421 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:48:43.442 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:48:43.459 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:43.460 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:43.463 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:48:43.472 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-20T01:48:43.485 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:48:43.507 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:48:43.508 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:43.522+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:43.528 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:48:43.550 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:48:43.571 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:48:43.592 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:48:43.608 INFO:tasks.ceph.osd.13.smithi121.stderr:2024-04-20T01:48:43.623+0000 7f589f484700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 74351) UID: 0 2024-04-20T01:48:43.613 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:48:43.617 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:43.617 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:43.629 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-20T01:48:43.635 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:48:43.656 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:48:43.679 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:48:43.700 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:48:43.709 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:43.724+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:43.722 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:48:43.723 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:48:43.723 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:48:43.723 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:48:43.723 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:48:43.723 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:48:43.723 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:48:43.723 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:48:43.723 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:48:43.723 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:48:43.723 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:48:43.723 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:48:43.723 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:48:43.723 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:48:43.723 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:48:43.724 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.14 2024-04-20T01:48:43.724 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:48:43.726 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-20T01:48:43.726 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-20T01:48:43.780 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:43.781 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:43.793 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-20T01:48:43.810 INFO:tasks.ceph.osd.11.smithi165.stderr:2024-04-20T01:48:43.824+0000 7f1b6b8f1700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 74539) UID: 0 2024-04-20T01:48:43.824 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-20T01:48:43.911 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:43.926+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:43.941 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:43.941 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:43.953 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-20T01:48:44.012 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:44.027+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:44.099 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:44.099 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:44.110 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-20T01:48:44.111 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T01:48:44.113 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:44.127+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:44.213 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:44.227+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:44.260 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:44.260 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:44.276 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-20T01:48:44.314 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:44.329+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:44.415 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:44.429+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:44.459 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:44.459 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:44.471 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-20T01:48:44.490 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:48:44.492 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T01:48:44.504 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-04-20T01:48:43.155047+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":126534176,"num_objects":69,"num_object_clones":21,"num_object_copies":473,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":2,"num_read":617,"num_read_kb":252350,"num_write":1223,"num_write_kb":171284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":45056000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1397,"ondisk_log_size":1397,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":682444,"kb_used_data":229640,"kb_used_omap":24,"kb_used_meta":452583,"kb_avail":1509266996,"statfs":{"total":1546188226560,"available":1545489403904,"internally_reserved":0,"allocated":235151360,"data_stored":284935796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25507,"internal_metadata":463445085},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001806"},"pg_stats":[{"pgid":"3.f","version":"91'66","reported_seq":287,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T01:48:37.931176+0000","last_change":"2024-04-20T01:48:37.931176+0000","last_active":"2024-04-20T01:48:37.931176+0000","last_peered":"2024-04-20T01:48:37.931176+0000","last_clean":"2024-04-20T01:48:37.931176+0000","last_became_active":"2024-04-20T01:47:32.867228+0000","last_became_peered":"2024-04-20T01:47:32.867228+0000","last_unstale":"2024-04-20T01:48:37.931176+0000","last_undegraded":"2024-04-20T01:48:37.931176+0000","last_fullsized":"2024-04-20T01:48:37.931176+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'66","last_scrub_stamp":"2024-04-20T01:48:37.930981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:48:37.930981+0000","objects_scrubbed":3,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:43.887124+0000","scrub_duration":0.0047362580000000001,"objects_trimmed":0,"snaptrim_duration":0.00059163100000000001,"stat_sum":{"num_bytes":9158656,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":31228,"num_write":71,"num_write_kb":9744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,15,13,8,11,0],"acting":[2,3,15,13,8,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"28","length":"1"},{"start":"33","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"}]},{"pgid":"3.e","version":"91'101","reported_seq":319,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T01:48:35.925815+0000","last_change":"2024-04-20T01:48:32.721589+0000","last_active":"2024-04-20T01:48:35.925815+0000","last_peered":"2024-04-20T01:48:35.925815+0000","last_clean":"2024-04-20T01:48:35.925815+0000","last_became_active":"2024-04-20T01:47:32.867409+0000","last_became_peered":"2024-04-20T01:47:32.867409+0000","last_unstale":"2024-04-20T01:48:35.925815+0000","last_undegraded":"2024-04-20T01:48:35.925815+0000","last_fullsized":"2024-04-20T01:48:35.925815+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:42.581217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001330403,"stat_sum":{"num_bytes":8486912,"num_objects":5,"num_object_clones":1,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":48,"num_read_kb":25936,"num_write":42,"num_write_kb":12240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,1,10,8,3,12],"acting":[11,9,1,10,8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"28","length":"1"},{"start":"33","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"}]},{"pgid":"3.d","version":"86'13","reported_seq":221,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T01:48:36.444757+0000","last_change":"2024-04-20T01:48:36.444757+0000","last_active":"2024-04-20T01:48:36.444757+0000","last_peered":"2024-04-20T01:48:36.444757+0000","last_clean":"2024-04-20T01:48:36.444757+0000","last_became_active":"2024-04-20T01:47:32.867954+0000","last_became_peered":"2024-04-20T01:47:32.867954+0000","last_unstale":"2024-04-20T01:48:36.444757+0000","last_undegraded":"2024-04-20T01:48:36.444757+0000","last_fullsized":"2024-04-20T01:48:36.444757+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"86'13","last_scrub_stamp":"2024-04-20T01:48:36.444597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:48:36.444597+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:59.750473+0000","scrub_duration":0.0020601930000000001,"objects_trimmed":0,"snaptrim_duration":0.00042964899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,11,15,7,8,4],"acting":[6,2,11,15,7,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"28","length":"1"},{"start":"33","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"}]},{"pgid":"3.c","version":"84'117","reported_seq":302,"reported_epoch":93,"state":"active+undersized+degraded","last_fresh":"2024-04-20T01:48:35.992093+0000","last_change":"2024-04-20T01:48:32.729454+0000","last_active":"2024-04-20T01:48:35.992093+0000","last_peered":"2024-04-20T01:48:35.992093+0000","last_clean":"2024-04-20T01:48:25.757240+0000","last_became_active":"2024-04-20T01:48:32.729454+0000","last_became_peered":"2024-04-20T01:48:32.729454+0000","last_unstale":"2024-04-20T01:48:35.992093+0000","last_undegraded":"2024-04-20T01:48:32.720475+0000","last_fullsized":"2024-04-20T01:48:32.720096+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:45.278307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039776630000000002,"stat_sum":{"num_bytes":24707072,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":23,"num_read_kb":8938,"num_write":82,"num_write_kb":22768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2147483647,8,6,3,7,12,5],"acting":[2147483647,8,6,3,7,12,5],"avail_no_missing":["8(1)","3(3)","5(6)","6(2)","7(4)","12(5)"],"object_location_counts":[{"shards":"3(3),5(6),6(2),7(4),8(1),12(5)","objects":6}],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"33","length":"1"}]},{"pgid":"3.b","version":"91'174","reported_seq":397,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T01:48:35.797408+0000","last_change":"2024-04-20T01:48:32.726960+0000","last_active":"2024-04-20T01:48:35.797408+0000","last_peered":"2024-04-20T01:48:35.797408+0000","last_clean":"2024-04-20T01:48:35.797408+0000","last_became_active":"2024-04-20T01:47:32.867309+0000","last_became_peered":"2024-04-20T01:47:32.867309+0000","last_unstale":"2024-04-20T01:48:35.797408+0000","last_undegraded":"2024-04-20T01:48:35.797408+0000","last_fullsized":"2024-04-20T01:48:35.797408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:01.227735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010435821,"stat_sum":{"num_bytes":24674304,"num_objects":7,"num_object_clones":1,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":72,"num_read_kb":68190,"num_write":184,"num_write_kb":26320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,11,10,9],"acting":[4,0,1,12,11,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"28","length":"1"},{"start":"33","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"}]},{"pgid":"3.a","version":"89'30","reported_seq":231,"reported_epoch":93,"state":"active+undersized+degraded","last_fresh":"2024-04-20T01:48:35.926946+0000","last_change":"2024-04-20T01:48:32.731010+0000","last_active":"2024-04-20T01:48:35.926946+0000","last_peered":"2024-04-20T01:48:35.926946+0000","last_clean":"2024-04-20T01:48:30.784754+0000","last_became_active":"2024-04-20T01:48:32.731010+0000","last_became_peered":"2024-04-20T01:48:32.731010+0000","last_unstale":"2024-04-20T01:48:35.926946+0000","last_undegraded":"2024-04-20T01:48:32.719054+0000","last_fullsized":"2024-04-20T01:48:32.718741+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:50.067770+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.00068918899999999999,"stat_sum":{"num_bytes":1720320,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":7013,"num_write":12,"num_write_kb":3376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,1,2147483647,3,15,2],"acting":[6,10,1,2147483647,3,15,2],"avail_no_missing":["6(0)","1(2)","2(6)","3(4)","10(1)","15(5)"],"object_location_counts":[{"shards":"1(2),2(6),3(4),6(0),10(1),15(5)","objects":1}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"33","length":"1"}]},{"pgid":"3.9","version":"89'41","reported_seq":239,"reported_epoch":93,"state":"active+undersized+degraded","last_fresh":"2024-04-20T01:48:35.798158+0000","last_change":"2024-04-20T01:48:32.730588+0000","last_active":"2024-04-20T01:48:35.798158+0000","last_peered":"2024-04-20T01:48:35.798158+0000","last_clean":"2024-04-20T01:48:30.794897+0000","last_became_active":"2024-04-20T01:48:32.730588+0000","last_became_peered":"2024-04-20T01:48:32.730588+0000","last_unstale":"2024-04-20T01:48:35.798158+0000","last_undegraded":"2024-04-20T01:48:32.723640+0000","last_fullsized":"2024-04-20T01:48:32.723235+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012668992,"stat_sum":{"num_bytes":4620288,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":3124,"num_write":50,"num_write_kb":5008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0,11,2147483647,8,3],"acting":[7,1,0,11,2147483647,8,3],"avail_no_missing":["7(0)","0(2)","1(1)","3(6)","8(5)","11(3)"],"object_location_counts":[{"shards":"0(2),1(1),3(6),7(0),8(5),11(3)","objects":4}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"33","length":"1"}]},{"pgid":"3.8","version":"91'97","reported_seq":309,"reported_epoch":93,"state":"active+undersized+degraded","last_fresh":"2024-04-20T01:48:35.929134+0000","last_change":"2024-04-20T01:48:32.729117+0000","last_active":"2024-04-20T01:48:35.929134+0000","last_peered":"2024-04-20T01:48:35.929134+0000","last_clean":"2024-04-20T01:48:30.796855+0000","last_became_active":"2024-04-20T01:48:32.729117+0000","last_became_peered":"2024-04-20T01:48:32.729117+0000","last_unstale":"2024-04-20T01:48:35.929134+0000","last_undegraded":"2024-04-20T01:48:32.717038+0000","last_fullsized":"2024-04-20T01:48:32.716659+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:58.562677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043849600000000002,"stat_sum":{"num_bytes":7716864,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":39,"num_read_kb":25841,"num_write":45,"num_write_kb":13232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,11,3,2147483647,2,7],"acting":[5,0,11,3,2147483647,2,7],"avail_no_missing":["5(0)","0(1)","2(5)","3(3)","7(6)","11(2)"],"object_location_counts":[{"shards":"0(1),2(5),3(3),5(0),7(6),11(2)","objects":4}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"33","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":77,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T01:48:35.797530+0000","last_change":"2024-04-20T01:47:20.071988+0000","last_active":"2024-04-20T01:48:35.797530+0000","last_peered":"2024-04-20T01:48:35.797530+0000","last_clean":"2024-04-20T01:48:35.797530+0000","last_became_active":"2024-04-20T01:47:20.071615+0000","last_became_peered":"2024-04-20T01:47:20.071615+0000","last_unstale":"2024-04-20T01:48:35.797530+0000","last_undegraded":"2024-04-20T01:48:35.797530+0000","last_fullsized":"2024-04-20T01:48:35.797530+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:40.106777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"89'35","reported_seq":240,"reported_epoch":93,"state":"active+undersized+degraded","last_fresh":"2024-04-20T01:48:35.927399+0000","last_change":"2024-04-20T01:48:32.729928+0000","last_active":"2024-04-20T01:48:35.927399+0000","last_peered":"2024-04-20T01:48:35.927399+0000","last_clean":"2024-04-20T01:48:30.781219+0000","last_became_active":"2024-04-20T01:48:32.729928+0000","last_became_peered":"2024-04-20T01:48:32.729928+0000","last_unstale":"2024-04-20T01:48:35.927399+0000","last_undegraded":"2024-04-20T01:48:32.715908+0000","last_fullsized":"2024-04-20T01:48:32.715658+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:58.175494+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.00049871900000000001,"stat_sum":{"num_bytes":4440064,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":14071,"num_write":18,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,4,2147483647,12,0,11],"acting":[10,8,4,2147483647,12,0,11],"avail_no_missing":["10(0)","0(5)","4(2)","8(1)","11(6)","12(4)"],"object_location_counts":[{"shards":"0(5),4(2),8(1),10(0),11(6),12(4)","objects":2}],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"33","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":87,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T01:48:36.889025+0000","last_change":"2024-04-20T01:48:36.889025+0000","last_active":"2024-04-20T01:48:36.889025+0000","last_peered":"2024-04-20T01:48:36.889025+0000","last_clean":"2024-04-20T01:48:36.889025+0000","last_became_active":"2024-04-20T01:47:20.075314+0000","last_became_peered":"2024-04-20T01:47:20.075314+0000","last_unstale":"2024-04-20T01:48:36.889025+0000","last_undegraded":"2024-04-20T01:48:36.889025+0000","last_fullsized":"2024-04-20T01:48:36.889025+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:36.888923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:36.888923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:45.012481+0000","scrub_duration":0.0018745420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,12],"acting":[2,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"91'51","reported_seq":251,"reported_epoch":94,"state":"active+undersized+degraded","last_fresh":"2024-04-20T01:48:38.969445+0000","last_change":"2024-04-20T01:48:32.727341+0000","last_active":"2024-04-20T01:48:38.969445+0000","last_peered":"2024-04-20T01:48:38.969445+0000","last_clean":"2024-04-20T01:48:30.786632+0000","last_became_active":"2024-04-20T01:48:32.727341+0000","last_became_peered":"2024-04-20T01:48:32.727341+0000","last_unstale":"2024-04-20T01:48:38.969445+0000","last_undegraded":"2024-04-20T01:48:32.718563+0000","last_fullsized":"2024-04-20T01:48:32.718172+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.015135822,"stat_sum":{"num_bytes":1540096,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":17,"num_read_kb":3173,"num_write":67,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,4,2147483647,11,5,10],"acting":[13,2,4,2147483647,11,5,10],"avail_no_missing":["13(0)","2(1)","4(2)","5(5)","10(6)","11(4)"],"object_location_counts":[{"shards":"2(1),4(2),5(5),10(6),11(4),13(0)","objects":5}],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"33","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":88,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-20T01:48:38.969389+0000","last_change":"2024-04-20T01:48:26.145238+0000","last_active":"2024-04-20T01:48:38.969389+0000","last_peered":"2024-04-20T01:48:38.969389+0000","last_clean":"2024-04-20T01:48:38.969389+0000","last_became_active":"2024-04-20T01:47:20.038972+0000","last_became_peered":"2024-04-20T01:47:20.038972+0000","last_unstale":"2024-04-20T01:48:38.969389+0000","last_undegraded":"2024-04-20T01:48:38.969389+0000","last_fullsized":"2024-04-20T01:48:38.969389+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:26.145095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:26.145095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:54.362058+0000","scrub_duration":0.0015014830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"91'75","reported_seq":287,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T01:48:35.925965+0000","last_change":"2024-04-20T01:48:32.729859+0000","last_active":"2024-04-20T01:48:35.925965+0000","last_peered":"2024-04-20T01:48:35.925965+0000","last_clean":"2024-04-20T01:48:35.925965+0000","last_became_active":"2024-04-20T01:47:32.868365+0000","last_became_peered":"2024-04-20T01:47:32.868365+0000","last_unstale":"2024-04-20T01:48:35.925965+0000","last_undegraded":"2024-04-20T01:48:35.925965+0000","last_fullsized":"2024-04-20T01:48:35.925965+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:41.198793+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0091780230000000004,"stat_sum":{"num_bytes":9158656,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":9566,"num_write":68,"num_write_kb":11056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,1,5,15,4],"acting":[3,7,6,1,5,15,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"28","length":"1"},{"start":"33","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"}]},{"pgid":"2.3","version":"0'0","reported_seq":88,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T01:48:35.928680+0000","last_change":"2024-04-20T01:48:26.810391+0000","last_active":"2024-04-20T01:48:35.928680+0000","last_peered":"2024-04-20T01:48:35.928680+0000","last_clean":"2024-04-20T01:48:35.928680+0000","last_became_active":"2024-04-20T01:47:20.038764+0000","last_became_peered":"2024-04-20T01:47:20.038764+0000","last_unstale":"2024-04-20T01:48:35.928680+0000","last_undegraded":"2024-04-20T01:48:35.928680+0000","last_fullsized":"2024-04-20T01:48:35.928680+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:26.810276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:26.810276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:27.490462+0000","scrub_duration":0.0038071630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"25'157","reported_seq":286,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-20T01:48:38.969389+0000","last_change":"2024-04-20T01:48:36.239045+0000","last_active":"2024-04-20T01:48:38.969389+0000","last_peered":"2024-04-20T01:48:38.969389+0000","last_clean":"2024-04-20T01:48:38.969389+0000","last_became_active":"2024-04-20T01:47:17.302105+0000","last_became_peered":"2024-04-20T01:47:17.302105+0000","last_unstale":"2024-04-20T01:48:38.969389+0000","last_undegraded":"2024-04-20T01:48:38.969389+0000","last_fullsized":"2024-04-20T01:48:38.969389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'157","last_scrub_stamp":"2024-04-20T01:48:36.238909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_clean_scrub_stamp":"2024-04-20T01:48:36.238909+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:57.382841+0000","scrub_duration":0.0025722610000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,15],"acting":[13,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.2","version":"89'146","reported_seq":360,"reported_epoch":94,"state":"active+undersized+degraded","last_fresh":"2024-04-20T01:48:38.969431+0000","last_change":"2024-04-20T01:48:32.730115+0000","last_active":"2024-04-20T01:48:38.969431+0000","last_peered":"2024-04-20T01:48:38.969431+0000","last_clean":"2024-04-20T01:48:30.787189+0000","last_became_active":"2024-04-20T01:48:32.730115+0000","last_became_peered":"2024-04-20T01:48:32.730115+0000","last_unstale":"2024-04-20T01:48:38.969431+0000","last_undegraded":"2024-04-20T01:48:32.719074+0000","last_fullsized":"2024-04-20T01:48:32.718683+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:46.684655+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.011785924,"stat_sum":{"num_bytes":13025280,"num_objects":8,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":62,"num_read_kb":21576,"num_write":105,"num_write_kb":23152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,6,10,2147483647,1,7],"acting":[13,3,6,10,2147483647,1,7],"avail_no_missing":["13(0)","1(5)","3(1)","6(2)","7(6)","10(3)"],"object_location_counts":[{"shards":"1(5),3(1),6(2),7(6),10(3),13(0)","objects":8}],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"33","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T01:48:37.682586+0000","last_change":"2024-04-20T01:48:37.682586+0000","last_active":"2024-04-20T01:48:37.682586+0000","last_peered":"2024-04-20T01:48:37.682586+0000","last_clean":"2024-04-20T01:48:37.682586+0000","last_became_active":"2024-04-20T01:48:19.876908+0000","last_became_peered":"2024-04-20T01:48:19.876908+0000","last_unstale":"2024-04-20T01:48:37.682586+0000","last_undegraded":"2024-04-20T01:48:37.682586+0000","last_fullsized":"2024-04-20T01:48:37.682586+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:37.682441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:37.682441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:48.439592+0000","scrub_duration":0.0018475430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"89'49","reported_seq":247,"reported_epoch":93,"state":"active+undersized+degraded","last_fresh":"2024-04-20T01:48:35.797397+0000","last_change":"2024-04-20T01:48:32.732259+0000","last_active":"2024-04-20T01:48:35.797397+0000","last_peered":"2024-04-20T01:48:35.797397+0000","last_clean":"2024-04-20T01:48:30.786309+0000","last_became_active":"2024-04-20T01:48:32.732259+0000","last_became_peered":"2024-04-20T01:48:32.732259+0000","last_unstale":"2024-04-20T01:48:35.797397+0000","last_undegraded":"2024-04-20T01:48:32.716381+0000","last_fullsized":"2024-04-20T01:48:32.715977+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:51.426907+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.00096742699999999998,"stat_sum":{"num_bytes":2588672,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":1012,"num_write":36,"num_write_kb":3424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2147483647,9,5,6,3,13],"acting":[4,2147483647,9,5,6,3,13],"avail_no_missing":["4(0)","3(5)","5(3)","6(4)","9(2)","13(6)"],"object_location_counts":[{"shards":"3(5),4(0),5(3),6(4),9(2),13(6)","objects":2}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"33","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":76,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T01:48:35.798081+0000","last_change":"2024-04-20T01:47:20.038409+0000","last_active":"2024-04-20T01:48:35.798081+0000","last_peered":"2024-04-20T01:48:35.798081+0000","last_clean":"2024-04-20T01:48:35.798081+0000","last_became_active":"2024-04-20T01:47:20.038107+0000","last_became_peered":"2024-04-20T01:47:20.038107+0000","last_unstale":"2024-04-20T01:48:35.798081+0000","last_undegraded":"2024-04-20T01:48:35.798081+0000","last_fullsized":"2024-04-20T01:48:35.798081+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:44.125577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"89'174","reported_seq":669,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T01:48:35.924085+0000","last_change":"2024-04-20T01:48:32.723552+0000","last_active":"2024-04-20T01:48:35.924085+0000","last_peered":"2024-04-20T01:48:35.924085+0000","last_clean":"2024-04-20T01:48:35.924085+0000","last_became_active":"2024-04-20T01:48:29.782732+0000","last_became_peered":"2024-04-20T01:48:29.782732+0000","last_unstale":"2024-04-20T01:48:35.924085+0000","last_undegraded":"2024-04-20T01:48:35.924085+0000","last_fullsized":"2024-04-20T01:48:35.924085+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:01.314314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073509699999999999,"stat_sum":{"num_bytes":9256960,"num_objects":9,"num_object_clones":2,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":55,"num_read_kb":16983,"num_write":126,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":45056000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,15,4,1,0,6,13],"acting":[8,15,4,1,0,6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"28","length":"1"},{"start":"33","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"}]},{"pgid":"3.7","version":"91'49","reported_seq":248,"reported_epoch":93,"state":"active+undersized+degraded","last_fresh":"2024-04-20T01:48:35.797604+0000","last_change":"2024-04-20T01:48:32.732175+0000","last_active":"2024-04-20T01:48:35.797604+0000","last_peered":"2024-04-20T01:48:35.797604+0000","last_clean":"2024-04-20T01:48:30.811499+0000","last_became_active":"2024-04-20T01:48:32.732175+0000","last_became_peered":"2024-04-20T01:48:32.732175+0000","last_unstale":"2024-04-20T01:48:35.797604+0000","last_undegraded":"2024-04-20T01:48:32.716539+0000","last_fullsized":"2024-04-20T01:48:32.716064+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:52.047610+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015133989,"stat_sum":{"num_bytes":4210688,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":13319,"num_write":56,"num_write_kb":4912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,0,8,5,2147483647],"acting":[4,11,9,0,8,5,2147483647],"avail_no_missing":["4(0)","0(3)","5(5)","8(4)","9(2)","11(1)"],"object_location_counts":[{"shards":"0(3),4(0),5(5),8(4),9(2),11(1)","objects":4}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"33","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":77,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T01:48:35.924235+0000","last_change":"2024-04-20T01:47:20.110600+0000","last_active":"2024-04-20T01:48:35.924235+0000","last_peered":"2024-04-20T01:48:35.924235+0000","last_clean":"2024-04-20T01:48:35.924235+0000","last_became_active":"2024-04-20T01:47:20.110260+0000","last_became_peered":"2024-04-20T01:47:20.110260+0000","last_unstale":"2024-04-20T01:48:35.924235+0000","last_undegraded":"2024-04-20T01:48:35.924235+0000","last_fullsized":"2024-04-20T01:48:35.924235+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:47:19.015601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:47.718290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":88,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T01:48:37.432504+0000","last_change":"2024-04-20T01:48:37.432504+0000","last_active":"2024-04-20T01:48:37.432504+0000","last_peered":"2024-04-20T01:48:37.432504+0000","last_clean":"2024-04-20T01:48:37.432504+0000","last_became_active":"2024-04-20T01:47:20.036328+0000","last_became_peered":"2024-04-20T01:47:20.036328+0000","last_unstale":"2024-04-20T01:48:37.432504+0000","last_undegraded":"2024-04-20T01:48:37.432504+0000","last_fullsized":"2024-04-20T01:48:37.432504+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:37.432331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:37.432331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:49.945535+0000","scrub_duration":0.0020345559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,13],"acting":[6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"89'22","reported_seq":227,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-20T01:48:35.925861+0000","last_change":"2024-04-20T01:48:32.725554+0000","last_active":"2024-04-20T01:48:35.925861+0000","last_peered":"2024-04-20T01:48:35.925861+0000","last_clean":"2024-04-20T01:48:35.925861+0000","last_became_active":"2024-04-20T01:47:32.866834+0000","last_became_peered":"2024-04-20T01:47:32.866834+0000","last_unstale":"2024-04-20T01:48:35.925861+0000","last_undegraded":"2024-04-20T01:48:35.925861+0000","last_fullsized":"2024-04-20T01:48:35.925861+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:31.699267+0000","last_clean_scrub_stamp":"2024-04-20T01:47:31.699267+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:48:54.424527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087751300000000001,"stat_sum":{"num_bytes":638976,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":2260,"num_write":52,"num_write_kb":2720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,3,6,2,9],"acting":[0,13,1,3,6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"28","length":"1"},{"start":"33","length":"1"},{"start":"36","length":"1"},{"start":"38","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":125943808,"num_objects":67,"num_object_clones":21,"num_object_copies":469,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":2,"num_read":481,"num_read_kb":252232,"num_write":1020,"num_write_kb":167824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":45056000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":212111360,"data_stored":271134720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1240,"ondisk_log_size":1240,"up":112,"acting":112,"num_store_stats":16},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280532,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41824,"kb_used_data":10708,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94330016,"statfs":{"total":96636764160,"available":96593936384,"internally_reserved":0,"allocated":10964992,"data_stored":14430347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.372}]},{"osd":2,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.56699999999999995}]},{"osd":3,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.436}]},{"osd":4,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":9.0129999999999999,"5min":9.0129999999999999,"15min":9.0129999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.63}]},{"osd":5,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.51800000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.504}]},{"osd":7,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.54800000000000004}]},{"osd":10,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.57899999999999996}]},{"osd":11,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.72399999999999998}]},{"osd":12,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.62},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":9.1389999999999993,"5min":9.1389999999999993,"15min":9.1389999999999993},"last":0.74399999999999999}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.52900000000000003}]},{"osd":15,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.8719999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.66600000000000004}]}]},{"osd":15,"up_from":21,"seq":90194313236,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36044,"kb_used_data":9024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335796,"statfs":{"total":96636764160,"available":96599855104,"internally_reserved":0,"allocated":9240576,"data_stored":11440870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7,8,9,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":4.5259999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":0.39800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":2,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.44900000000000001}]},{"osd":3,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.35499999999999998}]},{"osd":4,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.49099999999999999}]},{"osd":5,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.625}]},{"osd":6,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.71599999999999997}]},{"osd":8,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.751},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":4.5979999999999999,"5min":4.5979999999999999,"15min":4.5979999999999999},"last":0.58499999999999996}]},{"osd":9,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.78600000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.53600000000000003}]}]},{"osd":14,"up_from":21,"seq":90194313234,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43664,"kb_used_data":16640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328176,"statfs":{"total":96636764160,"available":96592052224,"internally_reserved":0,"allocated":17039360,"data_stored":23276351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,4,5,6,7,8,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.49199999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.44},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.56799999999999995}]},{"osd":5,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.629},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.61699999999999999}]},{"osd":6,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.42599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":9,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.65700000000000003}]},{"osd":12,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.67400000000000004}]},{"osd":13,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.75}]},{"osd":15,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.70999999999999996}]}]},{"osd":12,"up_from":21,"seq":90194313236,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42112,"kb_used_data":15092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94329728,"statfs":{"total":96636764160,"available":96593641472,"internally_reserved":0,"allocated":15454208,"data_stored":17289414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":1,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.504}]},{"osd":2,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.76500000000000001}]},{"osd":3,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.753},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.54300000000000004}]},{"osd":4,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.60299999999999998}]},{"osd":5,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.623},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.70999999999999996}]},{"osd":6,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.81299999999999994}]},{"osd":11,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":6.6779999999999999,"5min":6.6779999999999999,"15min":6.6779999999999999},"last":0.79100000000000004}]},{"osd":13,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.86399999999999999}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":1.135},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.99299999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.89100000000000001}]}]},{"osd":11,"up_from":21,"seq":90194313235,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48300,"kb_used_data":17184,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94323540,"statfs":{"total":96636764160,"available":96587304960,"internally_reserved":0,"allocated":17596416,"data_stored":20242630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,3,4,8,9,10,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.67900000000000005}]},{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.48499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":3,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.60799999999999998}]},{"osd":4,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":8.5969999999999995,"5min":8.5969999999999995,"15min":8.5969999999999995},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.627}]},{"osd":8,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.83599999999999997}]},{"osd":9,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.58999999999999997}]},{"osd":10,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.752}]},{"osd":12,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":12.385,"5min":12.385,"15min":12.385},"last":0.77200000000000002}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.61899999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.79300000000000004}]}]},{"osd":10,"up_from":21,"seq":90194313237,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45624,"kb_used_data":14508,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94326216,"statfs":{"total":96636764160,"available":96590045184,"internally_reserved":0,"allocated":14856192,"data_stored":16109766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,4,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.50700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.65000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":7,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.745}]},{"osd":8,"last update":"Sat Apr 20 01:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.67100000000000004}]},{"osd":9,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.63300000000000001}]},{"osd":11,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":6.0410000000000004,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":0.754}]},{"osd":12,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.331,"15min":1.331},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":9.7509999999999994,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":9.7759999999999998,"5min":9.7759999999999998,"15min":9.7759999999999998},"last":0.72499999999999998}]},{"osd":13,"last update":"Sat Apr 20 01:48:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.76400000000000001}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.64200000000000002}]},{"osd":15,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.80200000000000005}]}]},{"osd":9,"up_from":21,"seq":90194313237,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38284,"kb_used_data":11264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333556,"statfs":{"total":96636764160,"available":96597561344,"internally_reserved":0,"allocated":11534336,"data_stored":12030150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,2,4,6,7,8,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.54900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.34200000000000003}]},{"osd":7,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.60999999999999999}]},{"osd":8,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.3319999999999999,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.56599999999999995}]},{"osd":10,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48199999999999998}]},{"osd":11,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.69499999999999995}]},{"osd":12,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.71499999999999997}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.56000000000000005}]},{"osd":15,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.73299999999999998}]}]},{"osd":8,"up_from":21,"seq":90194313236,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42740,"kb_used_data":15720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94329100,"statfs":{"total":96636764160,"available":96592998400,"internally_reserved":0,"allocated":16097280,"data_stored":21381318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":1,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.52800000000000002}]},{"osd":3,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":0.73199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":6,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":0.69099999999999995}]},{"osd":7,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.624},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.76800000000000002}]},{"osd":9,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.65500000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":11,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.5110000000000001,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.82699999999999996}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":13,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.77800000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":15,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.81599999999999995}]}]},{"osd":7,"up_from":21,"seq":90194313237,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41360,"kb_used_data":14340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94330480,"statfs":{"total":96636764160,"available":96594411520,"internally_reserved":0,"allocated":14684160,"data_stored":18792646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,3,4,6,8,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.43099999999999999}]},{"osd":1,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.67800000000000005}]},{"osd":3,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56499999999999995}]},{"osd":4,"last update":"Sat Apr 20 01:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":4.1630000000000003,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":4.1920000000000002,"5min":4.1920000000000002,"15min":4.1920000000000002},"last":0.54200000000000004}]},{"osd":6,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.48899999999999999}]},{"osd":8,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":5.7679999999999998,"5min":5.7679999999999998,"15min":5.7679999999999998},"last":0.626},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":9.952,"5min":9.952,"15min":9.952},"last":0.70099999999999996}]},{"osd":9,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.751},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.76000000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":11,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.68899999999999995}]},{"osd":12,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":4.2720000000000002,"5min":4.2720000000000002,"15min":4.2720000000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.72299999999999998}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.85999999999999999}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":6,"up_from":21,"seq":90194313235,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42864,"kb_used_data":15844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328976,"statfs":{"total":96636764160,"available":96592871424,"internally_reserved":0,"allocated":16224256,"data_stored":19742918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,7,8,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.59499999999999997}]},{"osd":2,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.435},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.34000000000000002}]},{"osd":3,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.54600000000000004}]},{"osd":4,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.68700000000000006}]},{"osd":5,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.38300000000000001}]},{"osd":7,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.56999999999999995}]},{"osd":8,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.78300000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":10,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.42199999999999999}]},{"osd":11,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.629},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.64500000000000002}]},{"osd":12,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.753}]},{"osd":13,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.46000000000000002}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.65700000000000003}]},{"osd":15,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.48499999999999999}]}]},{"osd":1,"up_from":21,"seq":90194313236,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49944,"kb_used_data":18828,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94321896,"statfs":{"total":96636764160,"available":96585621504,"internally_reserved":0,"allocated":19279872,"data_stored":21917894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.56699999999999995}]},{"osd":2,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.497},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48499999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.58399999999999996}]},{"osd":4,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.63400000000000001}]},{"osd":5,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.39200000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.52300000000000002}]},{"osd":7,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.624},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70999999999999996}]},{"osd":8,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.67900000000000005}]},{"osd":9,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.51300000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.57699999999999996}]}]},{"osd":0,"up_from":21,"seq":90194313237,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44168,"kb_used_data":17148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94327672,"statfs":{"total":96636764160,"available":96591536128,"internally_reserved":0,"allocated":17559552,"data_stored":20860540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.624}]},{"osd":2,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":8.0619999999999994,"5min":8.0619999999999994,"15min":8.0619999999999994},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.441}]},{"osd":3,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":3.363,"5min":3.363,"15min":3.363},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":7.9710000000000001,"5min":7.9710000000000001,"15min":7.9710000000000001},"last":0.77900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":3.5339999999999998,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":0.51900000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":8.0239999999999991,"5min":8.0239999999999991,"15min":8.0239999999999991},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.495}]},{"osd":7,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":3.4020000000000001,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":7.8630000000000004,"5min":7.8630000000000004,"15min":7.8630000000000004},"last":0.75}]},{"osd":8,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":8.1189999999999998,"5min":8.1189999999999998,"15min":8.1189999999999998},"last":0.38100000000000001}]},{"osd":9,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":3.5699999999999998,"5min":3.5699999999999998,"15min":3.5699999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":0.73299999999999998}]},{"osd":13,"last update":"Sat Apr 20 01:48:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.69899999999999995}]},{"osd":15,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":3.4369999999999998,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":0.76200000000000001}]}]},{"osd":2,"up_from":21,"seq":90194313235,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33444,"kb_used_data":6424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338396,"statfs":{"total":96636764160,"available":96602517504,"internally_reserved":0,"allocated":6578176,"data_stored":7987398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65000000000000002}]},{"osd":1,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.44600000000000001}]},{"osd":3,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.76800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":7,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.67900000000000005}]},{"osd":8,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.755}]},{"osd":9,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.5}]},{"osd":11,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.88900000000000001}]},{"osd":12,"last update":"Sat Apr 20 01:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":3.1190000000000002,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":3.492,"5min":3.492,"15min":3.492},"last":0.82999999999999996}]},{"osd":13,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.72099999999999997}]},{"osd":15,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.78300000000000003}]}]},{"osd":3,"up_from":21,"seq":90194313238,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50972,"kb_used_data":19856,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94320868,"statfs":{"total":96636764160,"available":96584568832,"internally_reserved":0,"allocated":20332544,"data_stored":25882822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,4,5,6,7,8,9,10,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.42699999999999999}]},{"osd":2,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.73699999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.496},{"interface":"front","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":9.3650000000000002,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":0.54000000000000004}]},{"osd":5,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.46600000000000003}]},{"osd":6,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.58999999999999997}]},{"osd":7,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.36599999999999999}]},{"osd":8,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":9.4700000000000006,"5min":9.4700000000000006,"15min":9.4700000000000006},"last":0.66300000000000003}]},{"osd":9,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.51500000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":12,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":10.959,"5min":10.959,"15min":10.959},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.49399999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.504}]}]},{"osd":4,"up_from":21,"seq":90194313237,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42012,"kb_used_data":14992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94329828,"statfs":{"total":96636764160,"available":96593743872,"internally_reserved":0,"allocated":15351808,"data_stored":17461446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":5.0679999999999996,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":1.331,"5min":1.331,"15min":1.331},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":16.27,"5min":16.27,"15min":16.27},"last":0.65200000000000002}]},{"osd":1,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":10.507999999999999,"5min":10.507999999999999,"15min":10.507999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":11.019,"5min":11.019,"15min":11.019},"last":0.72099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":4.9180000000000001,"5min":4.9180000000000001,"15min":4.9180000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":10.619,"5min":10.619,"15min":10.619},"last":0.79600000000000004}]},{"osd":5,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":10.895,"5min":10.895,"15min":10.895},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":4.9589999999999996,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":0.67300000000000004}]},{"osd":6,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":11.051,"5min":11.051,"15min":11.051},"last":0.51500000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":10.589,"5min":10.589,"15min":10.589},"last":0.48599999999999999}]},{"osd":7,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":10.957000000000001,"5min":10.957000000000001,"15min":10.957000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":10.656000000000001,"5min":10.656000000000001,"15min":10.656000000000001},"last":0.83799999999999997}]},{"osd":8,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.79,"5min":3.79,"15min":3.79},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.89400000000000002}]},{"osd":9,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.385,"15min":1.385},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":10.929,"5min":10.929,"15min":10.929},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":10.555999999999999,"5min":10.555999999999999,"15min":10.555999999999999},"last":0.95599999999999996}]},{"osd":10,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":5.0229999999999997,"5min":5.0229999999999997,"15min":5.0229999999999997},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":10.69,"5min":10.69,"15min":10.69},"last":0.93300000000000005}]},{"osd":11,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":10.987,"5min":10.987,"15min":10.987},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":4.9900000000000002,"5min":4.9900000000000002,"15min":4.9900000000000002},"last":0.82599999999999996}]},{"osd":12,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":10.720000000000001,"5min":10.720000000000001,"15min":10.720000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.772,"5min":1.772,"15min":1.772},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":16.617000000000001,"5min":16.617000000000001,"15min":16.617000000000001},"last":0.88400000000000001}]},{"osd":13,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":3.8999999999999999,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":4.5019999999999998,"5min":4.5019999999999998,"15min":4.5019999999999998},"last":0.86799999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]},{"osd":5,"up_from":21,"seq":90194313236,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39088,"kb_used_data":12068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94332752,"statfs":{"total":96636764160,"available":96596738048,"internally_reserved":0,"allocated":12357632,"data_stored":16089286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,8,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":10.423999999999999,"5min":10.423999999999999,"15min":10.423999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":10.465,"5min":10.465,"15min":10.465},"last":0.374}]},{"osd":2,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.41599999999999998}]},{"osd":3,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.60399999999999998}]},{"osd":4,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":8.5269999999999992,"5min":8.5269999999999992,"15min":8.5269999999999992},"last":0.317},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.79,"5min":2.79,"15min":2.79},"last":0.34599999999999997}]},{"osd":6,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.46899999999999997}]},{"osd":7,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.64200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":10,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.45600000000000002}]},{"osd":11,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.69599999999999995}]},{"osd":12,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":10.616,"5min":10.616,"15min":10.616},"last":0.56200000000000006},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":10.629,"5min":10.629,"15min":10.629},"last":0.57199999999999995}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.84699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16179200,"data_stored":20058112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17895424,"data_stored":21114880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5193728,"data_stored":7184384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18948096,"data_stored":25079808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":13967360,"data_stored":16658432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10973184,"data_stored":15286272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":14839808,"data_stored":18939904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":13299712,"data_stored":17989632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":14712832,"data_stored":20578304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":10149888,"data_stored":11227136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":13471744,"data_stored":15306752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16211968,"data_stored":19439616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":14069760,"data_stored":16486400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":9052160,"data_stored":13099008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":15822848,"data_stored":22577152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":7323648,"data_stored":10108928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T01:48:44.509 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-04-20T01:48:44.510 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-04-20T01:48:44.516 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:44.531+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:44.617 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T01:48:44.632+0000 7f453918c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43100) UID: 0 2024-04-20T01:48:44.633 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:44.633 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:44.645 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-20T01:48:44.718 INFO:tasks.ceph.osd.4.smithi044.stderr:2024-04-20T01:48:44.732+0000 7fb5f1e24700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43484) UID: 0 2024-04-20T01:48:44.792 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:44.792 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:44.804 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-20T01:48:44.816 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is set 2024-04-20T01:48:44.830 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-20T01:48:44.844+0000 7f3bd0aeb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43483) UID: 0 2024-04-20T01:48:44.920 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T01:48:44.934+0000 7fd94969e700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 74344) UID: 0 2024-04-20T01:48:44.960 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:44.960 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:44.972 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-20T01:48:45.020 INFO:tasks.ceph.osd.15.smithi165.stderr:2024-04-20T01:48:45.035+0000 7f78529a2700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 74540) UID: 0 2024-04-20T01:48:45.117 INFO:teuthology.orchestra.run.smithi044.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:45.118 INFO:teuthology.orchestra.run.smithi044.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:45.121 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:45.135+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:45.129 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-20T01:48:45.222 INFO:tasks.ceph.osd.2.smithi123.stderr:2024-04-20T01:48:45.236+0000 7f01c044c700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43099) UID: 0 2024-04-20T01:48:45.269 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:45.269 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:45.280 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-20T01:48:45.323 INFO:tasks.ceph.osd.10.smithi123.stderr:2024-04-20T01:48:45.337+0000 7ff11d0bf700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 43101) UID: 0 2024-04-20T01:48:45.408 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:45.408 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:45.418 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-20T01:48:45.424 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-20T01:48:45.439+0000 7f0f58b1f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 74532) UID: 0 2024-04-20T01:48:45.525 INFO:tasks.ceph.osd.9.smithi121.stderr:2024-04-20T01:48:45.540+0000 7faff4e42700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 74350) UID: 0 2024-04-20T01:48:45.545 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:45.546 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:45.556 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-20T01:48:45.626 INFO:tasks.ceph.osd.8.smithi044.stderr:2024-04-20T01:48:45.641+0000 7ffafe710700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 43485) UID: 0 2024-04-20T01:48:45.724 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:45.724 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:45.726 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-20T01:48:45.727 INFO:tasks.ceph.osd.5.smithi121.stderr:2024-04-20T01:48:45.741+0000 7f986dfea700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 74346) UID: 0 2024-04-20T01:48:45.734 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-20T01:48:45.828 INFO:tasks.ceph.osd.12.smithi044.stderr:2024-04-20T01:48:45.842+0000 7feac00b5700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 43495) UID: 0 2024-04-20T01:48:45.838 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'unique_pool_0' removed 2024-04-20T01:48:45.856 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-04-20T01:48:45.868 INFO:tasks.thrashosds:joining thrashosds 2024-04-20T01:48:45.903 INFO:teuthology.orchestra.run.smithi121.stdout:ERROR: (22) Invalid argument 2024-04-20T01:48:45.904 INFO:teuthology.orchestra.run.smithi121.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-20T01:48:45.909 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0: {} 2024-04-20T01:48:45.914 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-20T01:48:45.930 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1: {} 2024-04-20T01:48:45.951 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2: {} 2024-04-20T01:48:45.973 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3: {} 2024-04-20T01:48:45.993 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4: {} 2024-04-20T01:48:46.014 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5: {} 2024-04-20T01:48:46.035 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6: {} 2024-04-20T01:48:46.044 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:46.044 INFO:teuthology.orchestra.run.smithi121.stdout: "size": 20, 2024-04-20T01:48:46.044 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 600, 2024-04-20T01:48:46.045 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-04-20T01:48:46.045 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.045 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:307 3.8s0 3:16aac79f:::smithi16579500-14:head [copy-from ver 2] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:46.045 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.823600+0000", 2024-04-20T01:48:46.045 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 62.234148937000001, 2024-04-20T01:48:46.045 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.016644029000000001, 2024-04-20T01:48:46.045 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.045 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:46.045 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:46.045 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:46.045 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:46.045 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 307 2024-04-20T01:48:46.045 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.046 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.046 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.046 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.046 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823600+0000", 2024-04-20T01:48:46.046 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.046 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.046 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.046 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.046 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823600+0000", 2024-04-20T01:48:46.046 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.047 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.047 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.047 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.047 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823598+0000", 2024-04-20T01:48:46.047 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:46.047 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.047 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.047 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.047 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823606+0000", 2024-04-20T01:48:46.047 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.2629999999999997e-06 2024-04-20T01:48:46.047 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.047 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.047 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.047 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823621+0000", 2024-04-20T01:48:46.047 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5373999999999999e-05 2024-04-20T01:48:46.048 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.048 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.048 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.048 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823635+0000", 2024-04-20T01:48:46.048 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3888e-05 2024-04-20T01:48:46.048 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.048 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.048 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.048 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823683+0000", 2024-04-20T01:48:46.048 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.8455000000000003e-05 2024-04-20T01:48:46.048 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.048 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.048 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:46.048 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823792+0000", 2024-04-20T01:48:46.049 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000108156 2024-04-20T01:48:46.049 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.049 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.049 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.049 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.830048+0000", 2024-04-20T01:48:46.049 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0062565729999999996 2024-04-20T01:48:46.049 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.049 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.049 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.049 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.831286+0000", 2024-04-20T01:48:46.049 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0012377829999999999 2024-04-20T01:48:46.049 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.049 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.049 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:46.049 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.840196+0000", 2024-04-20T01:48:46.050 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0089103129999999996 2024-04-20T01:48:46.050 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.050 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.050 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.050 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.840244+0000", 2024-04-20T01:48:46.050 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.8154e-05 2024-04-20T01:48:46.050 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.050 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.050 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.050 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.050 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.050 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:311 3.8s0 3:1d169b1d:::smithi16579500-9:head [append 0~376832] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:46.050 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.920919+0000", 2024-04-20T01:48:46.050 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 62.136829857999999, 2024-04-20T01:48:46.051 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.009305476, 2024-04-20T01:48:46.051 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.051 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:46.051 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:46.051 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:46.051 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:46.051 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 311 2024-04-20T01:48:46.051 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.051 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.051 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.051 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.051 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.920919+0000", 2024-04-20T01:48:46.051 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.051 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.051 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.052 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.052 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.920919+0000", 2024-04-20T01:48:46.052 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.052 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.052 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.052 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.052 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.920918+0000", 2024-04-20T01:48:46.052 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:46.052 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.052 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.052 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.052 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921528+0000", 2024-04-20T01:48:46.053 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00060985100000000001 2024-04-20T01:48:46.053 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.053 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.053 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.053 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921543+0000", 2024-04-20T01:48:46.053 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4302e-05 2024-04-20T01:48:46.053 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.053 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.053 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.053 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921557+0000", 2024-04-20T01:48:46.053 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4416999999999999e-05 2024-04-20T01:48:46.053 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.053 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.053 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.053 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921979+0000", 2024-04-20T01:48:46.054 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00042189099999999998 2024-04-20T01:48:46.054 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.054 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.054 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:46.054 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.922074+0000", 2024-04-20T01:48:46.054 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4798999999999998e-05 2024-04-20T01:48:46.054 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.054 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.054 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.054 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.922634+0000", 2024-04-20T01:48:46.054 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00056005300000000003 2024-04-20T01:48:46.054 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.054 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.054 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.054 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923907+0000", 2024-04-20T01:48:46.055 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0012733239999999999 2024-04-20T01:48:46.055 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.055 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.055 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:46.055 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930149+0000", 2024-04-20T01:48:46.055 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00624202 2024-04-20T01:48:46.055 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.055 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.055 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.055 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930225+0000", 2024-04-20T01:48:46.055 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.5860999999999998e-05 2024-04-20T01:48:46.055 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.055 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.055 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.055 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.056 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.056 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:313 3.8s0 3:1d169b1d:::smithi16579500-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:46.056 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.921593+0000", 2024-04-20T01:48:46.056 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 62.136156278999998, 2024-04-20T01:48:46.056 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0095502929999999996, 2024-04-20T01:48:46.056 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.056 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:46.056 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:46.057 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:46.057 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:46.057 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 313 2024-04-20T01:48:46.057 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.057 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.057 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.058 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.058 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921593+0000", 2024-04-20T01:48:46.058 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.058 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.058 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.058 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.058 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921593+0000", 2024-04-20T01:48:46.058 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.058 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.058 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.059 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.059 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921593+0000", 2024-04-20T01:48:46.059 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:46.059 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.059 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.059 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.059 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921596+0000", 2024-04-20T01:48:46.059 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.089e-06 2024-04-20T01:48:46.059 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.059 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.059 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.059 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921601+0000", 2024-04-20T01:48:46.060 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.8799999999999999e-06 2024-04-20T01:48:46.060 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.060 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.060 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.060 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921604+0000", 2024-04-20T01:48:46.060 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.36e-06 2024-04-20T01:48:46.060 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.060 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.060 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.060 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923831+0000", 2024-04-20T01:48:46.060 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0022266479999999999 2024-04-20T01:48:46.060 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.060 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.060 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923898+0000", 2024-04-20T01:48:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.7391000000000006e-05 2024-04-20T01:48:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930413+0000", 2024-04-20T01:48:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.006514382 2024-04-20T01:48:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930468+0000", 2024-04-20T01:48:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.5923e-05 2024-04-20T01:48:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.931143+0000", 2024-04-20T01:48:46.062 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00067489999999999998 2024-04-20T01:48:46.062 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.062 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.062 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.062 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.062 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.062 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:314 3.8s0 3:1746945e:::smithi16579500-24:head [copy-from ver 4] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:46.062 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.921607+0000", 2024-04-20T01:48:46.062 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 62.136142683999999, 2024-04-20T01:48:46.062 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013830053, 2024-04-20T01:48:46.062 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.062 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:46.062 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:46.062 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:46.063 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:46.063 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 314 2024-04-20T01:48:46.063 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.063 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.063 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.063 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.063 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921607+0000", 2024-04-20T01:48:46.063 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.063 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.063 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.063 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.063 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921607+0000", 2024-04-20T01:48:46.063 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.063 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.063 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921606+0000", 2024-04-20T01:48:46.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:46.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921609+0000", 2024-04-20T01:48:46.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0999999999999998e-06 2024-04-20T01:48:46.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921612+0000", 2024-04-20T01:48:46.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.4089999999999999e-06 2024-04-20T01:48:46.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.921614+0000", 2024-04-20T01:48:46.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5610000000000001e-06 2024-04-20T01:48:46.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923919+0000", 2024-04-20T01:48:46.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0023049809999999998 2024-04-20T01:48:46.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:46.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.923973+0000", 2024-04-20T01:48:46.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3532999999999997e-05 2024-04-20T01:48:46.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.066 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.928709+0000", 2024-04-20T01:48:46.066 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0047359979999999999 2024-04-20T01:48:46.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.066 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.066 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.066 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.930393+0000", 2024-04-20T01:48:46.066 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0016838459999999999 2024-04-20T01:48:46.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.066 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.066 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:46.066 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.935394+0000", 2024-04-20T01:48:46.066 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0050015779999999996 2024-04-20T01:48:46.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.066 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.067 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.067 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.935437+0000", 2024-04-20T01:48:46.067 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2289000000000001e-05 2024-04-20T01:48:46.067 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.067 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.067 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.067 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.067 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.067 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:336 3.8s0 3:1746945e:::smithi16579500-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:46.067 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.973134+0000", 2024-04-20T01:48:46.067 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 62.084615716000002, 2024-04-20T01:48:46.067 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010536856000000001, 2024-04-20T01:48:46.067 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.067 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:46.067 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:46.068 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:46.068 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:46.068 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 336 2024-04-20T01:48:46.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.068 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.068 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.068 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.068 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973134+0000", 2024-04-20T01:48:46.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.068 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.068 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.068 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973134+0000", 2024-04-20T01:48:46.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.069 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973133+0000", 2024-04-20T01:48:46.069 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:46.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973135+0000", 2024-04-20T01:48:46.070 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5269999999999999e-06 2024-04-20T01:48:46.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973139+0000", 2024-04-20T01:48:46.070 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.3749999999999996e-06 2024-04-20T01:48:46.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.071 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.071 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.973142+0000", 2024-04-20T01:48:46.071 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7800000000000001e-06 2024-04-20T01:48:46.071 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.071 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.071 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.071 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.978050+0000", 2024-04-20T01:48:46.071 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0049085049999999996 2024-04-20T01:48:46.071 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.071 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.071 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:46.071 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.978106+0000", 2024-04-20T01:48:46.071 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.5648e-05 2024-04-20T01:48:46.071 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.071 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.072 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.072 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.980649+0000", 2024-04-20T01:48:46.072 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0025426630000000001 2024-04-20T01:48:46.072 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.072 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.072 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:46.072 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.980701+0000", 2024-04-20T01:48:46.072 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.1907999999999998e-05 2024-04-20T01:48:46.072 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.072 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.072 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.072 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.983670+0000", 2024-04-20T01:48:46.072 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0029697590000000002 2024-04-20T01:48:46.072 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.072 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.073 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.073 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.073 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:487 3.8s0 3:16aac79f:::smithi16579500-14:head [append 0~425984] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:46.073 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:59.760614+0000", 2024-04-20T01:48:46.073 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 46.297135699999998, 2024-04-20T01:48:46.073 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010465621, 2024-04-20T01:48:46.073 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.073 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:46.073 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:46.073 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:46.073 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:46.073 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 487 2024-04-20T01:48:46.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.074 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.074 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.760614+0000", 2024-04-20T01:48:46.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.074 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.760614+0000", 2024-04-20T01:48:46.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.074 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.760611+0000", 2024-04-20T01:48:46.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999971 2024-04-20T01:48:46.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.075 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.761151+0000", 2024-04-20T01:48:46.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00054006400000000004 2024-04-20T01:48:46.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.075 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.761173+0000", 2024-04-20T01:48:46.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1926999999999999e-05 2024-04-20T01:48:46.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.075 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.076 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.761202+0000", 2024-04-20T01:48:46.076 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9286e-05 2024-04-20T01:48:46.076 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.076 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.076 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.076 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.761276+0000", 2024-04-20T01:48:46.076 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.4319000000000005e-05 2024-04-20T01:48:46.076 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.076 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.076 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op must wait for map", 2024-04-20T01:48:46.076 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.761289+0000", 2024-04-20T01:48:46.076 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.239e-05 2024-04-20T01:48:46.076 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.076 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.076 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.077 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.763247+0000", 2024-04-20T01:48:46.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0019584989999999998 2024-04-20T01:48:46.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.077 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:46.077 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.763331+0000", 2024-04-20T01:48:46.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.3320999999999997e-05 2024-04-20T01:48:46.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.077 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.077 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.764297+0000", 2024-04-20T01:48:46.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00096590700000000001 2024-04-20T01:48:46.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.078 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.078 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768005+0000", 2024-04-20T01:48:46.078 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0037084940000000001 2024-04-20T01:48:46.078 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.078 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.078 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:46.078 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.770983+0000", 2024-04-20T01:48:46.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0029780129999999998 2024-04-20T01:48:46.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.079 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.771079+0000", 2024-04-20T01:48:46.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6250999999999997e-05 2024-04-20T01:48:46.079 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.079 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.079 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.079 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:489 3.8s0 3:16aac79f:::smithi16579500-14:head [append 0~376832] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:46.079 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:59.767481+0000", 2024-04-20T01:48:46.079 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 46.290268029000003, 2024-04-20T01:48:46.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011576091, 2024-04-20T01:48:46.080 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.080 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:46.080 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:46.080 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:46.080 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:46.080 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 489 2024-04-20T01:48:46.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.080 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.080 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.080 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.767481+0000", 2024-04-20T01:48:46.080 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.080 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.081 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.767481+0000", 2024-04-20T01:48:46.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.081 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.081 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.767480+0000", 2024-04-20T01:48:46.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:46.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.081 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.081 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768144+0000", 2024-04-20T01:48:46.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00066418499999999997 2024-04-20T01:48:46.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.081 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.082 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768157+0000", 2024-04-20T01:48:46.082 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2281e-05 2024-04-20T01:48:46.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.082 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.082 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.082 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768166+0000", 2024-04-20T01:48:46.082 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2629999999999992e-06 2024-04-20T01:48:46.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.082 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.082 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.082 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768215+0000", 2024-04-20T01:48:46.082 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.9292000000000002e-05 2024-04-20T01:48:46.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.082 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.082 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:46.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768322+0000", 2024-04-20T01:48:46.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000106461 2024-04-20T01:48:46.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.083 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.769186+0000", 2024-04-20T01:48:46.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00086420900000000001 2024-04-20T01:48:46.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.083 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.771140+0000", 2024-04-20T01:48:46.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0019545890000000001 2024-04-20T01:48:46.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:46.084 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779011+0000", 2024-04-20T01:48:46.084 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0078710150000000003 2024-04-20T01:48:46.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.084 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.084 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779057+0000", 2024-04-20T01:48:46.084 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5918000000000003e-05 2024-04-20T01:48:46.084 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.084 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.084 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.085 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.085 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:490 3.8s0 3:16aac79f:::smithi16579500-14:head [setxattr _header (57) in=7b] snapc 1a=[1a,19,18,14,b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:46.085 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:59.768179+0000", 2024-04-20T01:48:46.085 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 46.289570136000002, 2024-04-20T01:48:46.085 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011076167, 2024-04-20T01:48:46.085 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.085 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:46.085 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:46.085 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:46.085 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:46.085 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 490 2024-04-20T01:48:46.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.085 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.085 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.086 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.086 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768179+0000", 2024-04-20T01:48:46.086 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.086 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.086 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.086 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.086 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768179+0000", 2024-04-20T01:48:46.086 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.086 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.086 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.086 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.086 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768178+0000", 2024-04-20T01:48:46.086 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:46.086 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.087 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.087 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.087 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768183+0000", 2024-04-20T01:48:46.087 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.288e-06 2024-04-20T01:48:46.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.087 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.087 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.087 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768191+0000", 2024-04-20T01:48:46.087 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.5850000000000002e-06 2024-04-20T01:48:46.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.087 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.087 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.088 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768197+0000", 2024-04-20T01:48:46.088 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.9270000000000002e-06 2024-04-20T01:48:46.088 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.088 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.088 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.088 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.769809+0000", 2024-04-20T01:48:46.088 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001611913 2024-04-20T01:48:46.088 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.088 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.088 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:46.088 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.769930+0000", 2024-04-20T01:48:46.088 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000121304 2024-04-20T01:48:46.088 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.089 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.089 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.089 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.770462+0000", 2024-04-20T01:48:46.089 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00053196700000000005 2024-04-20T01:48:46.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.089 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.089 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.089 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.771310+0000", 2024-04-20T01:48:46.089 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00084813200000000003 2024-04-20T01:48:46.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.089 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.089 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:46.089 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779156+0000", 2024-04-20T01:48:46.089 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0078461580000000006 2024-04-20T01:48:46.090 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.090 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.090 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.090 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779255+0000", 2024-04-20T01:48:46.090 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.8944999999999995e-05 2024-04-20T01:48:46.090 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.090 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.090 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.090 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.090 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.090 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:491 3.8s0 3:16aac79f:::smithi16579500-14:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-04-20T01:48:46.090 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:59.768204+0000", 2024-04-20T01:48:46.090 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 46.289545046999997, 2024-04-20T01:48:46.091 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.012786562, 2024-04-20T01:48:46.091 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.091 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:46.091 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:46.091 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:46.091 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:46.091 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 491 2024-04-20T01:48:46.091 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.091 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.091 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.091 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.091 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768204+0000", 2024-04-20T01:48:46.091 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.091 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.091 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.092 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.092 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768204+0000", 2024-04-20T01:48:46.092 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.092 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.092 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.092 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768204+0000", 2024-04-20T01:48:46.092 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:46.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.092 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.092 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.092 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768207+0000", 2024-04-20T01:48:46.092 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.4180000000000001e-06 2024-04-20T01:48:46.093 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.093 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.093 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.093 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768215+0000", 2024-04-20T01:48:46.093 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.6669999999999996e-06 2024-04-20T01:48:46.093 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.093 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.093 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.093 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.768220+0000", 2024-04-20T01:48:46.093 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3149999999999999e-06 2024-04-20T01:48:46.093 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.093 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.093 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.093 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.771203+0000", 2024-04-20T01:48:46.093 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.002982434 2024-04-20T01:48:46.094 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.094 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.094 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:46.094 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.771299+0000", 2024-04-20T01:48:46.094 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6111000000000002e-05 2024-04-20T01:48:46.094 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.094 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.094 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.094 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779629+0000", 2024-04-20T01:48:46.094 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0083302529999999993 2024-04-20T01:48:46.094 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.094 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.094 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:46.094 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.779693+0000", 2024-04-20T01:48:46.095 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.4424999999999994e-05 2024-04-20T01:48:46.095 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.095 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.095 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.095 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:59.780991+0000", 2024-04-20T01:48:46.095 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0012974390000000001 2024-04-20T01:48:46.095 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.095 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.095 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.095 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.095 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.095 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.0s4 75/30 ECSubWrite(tid=55, reqid=client.4643.0:781, at_version=75'41, trim_to=0'0, roll_forward_to=66'39))", 2024-04-20T01:48:46.095 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.735057+0000", 2024-04-20T01:48:46.095 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 29.322692264000001, 2024-04-20T01:48:46.096 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.012883867, 2024-04-20T01:48:46.096 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.096 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.096 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.096 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.096 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.096 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735057+0000", 2024-04-20T01:48:46.096 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.096 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.096 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.096 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.096 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735057+0000", 2024-04-20T01:48:46.096 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.096 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.096 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.097 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.097 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735056+0000", 2024-04-20T01:48:46.097 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:46.097 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.097 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.097 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.097 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735367+0000", 2024-04-20T01:48:46.097 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00031039199999999999 2024-04-20T01:48:46.097 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.097 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.097 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.097 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735378+0000", 2024-04-20T01:48:46.097 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1389000000000001e-05 2024-04-20T01:48:46.097 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.097 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.098 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.098 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735385+0000", 2024-04-20T01:48:46.098 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.9759999999999998e-06 2024-04-20T01:48:46.098 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.098 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.102 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.102 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735429+0000", 2024-04-20T01:48:46.102 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.3699000000000003e-05 2024-04-20T01:48:46.102 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.103 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.103 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.103 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.735437+0000", 2024-04-20T01:48:46.103 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.7440000000000007e-06 2024-04-20T01:48:46.103 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.103 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.103 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.103 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747914+0000", 2024-04-20T01:48:46.103 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.012477162 2024-04-20T01:48:46.103 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.103 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.103 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.103 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747941+0000", 2024-04-20T01:48:46.104 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7107e-05 2024-04-20T01:48:46.104 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.104 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.104 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.104 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.104 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.104 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.1s3 75/30 ECSubWrite(tid=257, reqid=osd.4.0:253, at_version=0'0, trim_to=0'0, roll_forward_to=75'37))", 2024-04-20T01:48:46.104 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.737287+0000", 2024-04-20T01:48:46.104 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 29.32046188, 2024-04-20T01:48:46.104 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00931126, 2024-04-20T01:48:46.104 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.104 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.104 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.104 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.105 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.105 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737287+0000", 2024-04-20T01:48:46.105 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.105 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.105 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.105 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.105 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737287+0000", 2024-04-20T01:48:46.105 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.105 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.105 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.105 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.105 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737287+0000", 2024-04-20T01:48:46.105 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:46.105 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.105 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.106 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.106 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737307+0000", 2024-04-20T01:48:46.106 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0547000000000002e-05 2024-04-20T01:48:46.106 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.106 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.106 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.106 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737320+0000", 2024-04-20T01:48:46.106 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3112000000000001e-05 2024-04-20T01:48:46.106 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.106 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.106 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.106 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737339+0000", 2024-04-20T01:48:46.106 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8289999999999999e-05 2024-04-20T01:48:46.106 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.107 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.107 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.107 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737420+0000", 2024-04-20T01:48:46.107 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.106e-05 2024-04-20T01:48:46.107 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.107 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.107 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.107 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.737431+0000", 2024-04-20T01:48:46.107 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1289e-05 2024-04-20T01:48:46.107 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.107 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.107 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.107 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.746568+0000", 2024-04-20T01:48:46.107 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0091374909999999993 2024-04-20T01:48:46.107 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.108 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.108 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.108 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.746599+0000", 2024-04-20T01:48:46.108 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0335999999999999e-05 2024-04-20T01:48:46.108 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.108 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.108 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.108 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.108 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.108 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.cs6 75/30 ECSubWrite(tid=114, reqid=osd.14.0:113, at_version=0'0, trim_to=0'0, roll_forward_to=75'93))", 2024-04-20T01:48:46.108 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.738324+0000", 2024-04-20T01:48:46.108 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 29.319424851000001, 2024-04-20T01:48:46.108 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0095662660000000004, 2024-04-20T01:48:46.108 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.109 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.109 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.109 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.109 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.109 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738324+0000", 2024-04-20T01:48:46.109 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.109 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.109 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.109 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.109 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738324+0000", 2024-04-20T01:48:46.109 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.109 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.109 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.109 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.109 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738324+0000", 2024-04-20T01:48:46.110 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:46.110 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.110 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.110 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.110 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738337+0000", 2024-04-20T01:48:46.110 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3596e-05 2024-04-20T01:48:46.110 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.110 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.110 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.110 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738349+0000", 2024-04-20T01:48:46.110 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1857e-05 2024-04-20T01:48:46.110 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.110 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.110 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.110 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738358+0000", 2024-04-20T01:48:46.111 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4060000000000005e-06 2024-04-20T01:48:46.111 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.111 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.111 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.111 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738425+0000", 2024-04-20T01:48:46.111 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.7916000000000005e-05 2024-04-20T01:48:46.111 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.111 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.111 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.111 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738442+0000", 2024-04-20T01:48:46.111 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6872999999999999e-05 2024-04-20T01:48:46.111 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.111 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.111 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.111 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747865+0000", 2024-04-20T01:48:46.112 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0094223810000000005 2024-04-20T01:48:46.112 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.112 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.112 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.112 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747891+0000", 2024-04-20T01:48:46.112 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6061999999999999e-05 2024-04-20T01:48:46.112 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.112 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.112 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.112 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.112 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.112 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s5 75/30 ECSubWrite(tid=258, reqid=osd.4.0:258, at_version=75'21, trim_to=0'0, roll_forward_to=68'19))", 2024-04-20T01:48:46.112 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.738907+0000", 2024-04-20T01:48:46.112 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 29.318842063000002, 2024-04-20T01:48:46.112 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010750793, 2024-04-20T01:48:46.113 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.113 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.113 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.113 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.113 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.113 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738907+0000", 2024-04-20T01:48:46.113 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.113 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.113 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.113 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.113 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738907+0000", 2024-04-20T01:48:46.113 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.113 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.113 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.113 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.114 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738907+0000", 2024-04-20T01:48:46.114 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:46.114 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.114 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.114 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.114 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738926+0000", 2024-04-20T01:48:46.114 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9032e-05 2024-04-20T01:48:46.114 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.114 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.114 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.114 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738938+0000", 2024-04-20T01:48:46.114 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2802999999999999e-05 2024-04-20T01:48:46.114 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.114 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.114 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.115 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738948+0000", 2024-04-20T01:48:46.115 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.0650000000000005e-06 2024-04-20T01:48:46.115 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.115 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.115 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.115 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739017+0000", 2024-04-20T01:48:46.115 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.9621000000000003e-05 2024-04-20T01:48:46.115 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.115 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.115 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.115 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739028+0000", 2024-04-20T01:48:46.115 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0957e-05 2024-04-20T01:48:46.115 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.115 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.116 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.116 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.749625+0000", 2024-04-20T01:48:46.116 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010597031999999999 2024-04-20T01:48:46.116 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.116 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.116 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.116 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.749658+0000", 2024-04-20T01:48:46.116 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.2947999999999997e-05 2024-04-20T01:48:46.116 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.116 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.116 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.116 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.117 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.117 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.cs6 75/30 ECSubWrite(tid=115, reqid=osd.14.0:115, at_version=75'95, trim_to=0'0, roll_forward_to=75'93))", 2024-04-20T01:48:46.117 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.739080+0000", 2024-04-20T01:48:46.117 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 29.31866982, 2024-04-20T01:48:46.117 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011132998, 2024-04-20T01:48:46.117 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.117 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.117 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.117 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.117 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.117 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739080+0000", 2024-04-20T01:48:46.117 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.117 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.117 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.118 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.118 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739080+0000", 2024-04-20T01:48:46.118 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.118 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.118 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.118 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.118 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739079+0000", 2024-04-20T01:48:46.118 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:46.118 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.118 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.118 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.118 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739096+0000", 2024-04-20T01:48:46.118 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7300999999999999e-05 2024-04-20T01:48:46.119 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.119 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.119 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.119 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739123+0000", 2024-04-20T01:48:46.119 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7257999999999999e-05 2024-04-20T01:48:46.119 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.119 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.119 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.119 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739132+0000", 2024-04-20T01:48:46.119 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8859999999999993e-06 2024-04-20T01:48:46.119 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.119 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.119 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.119 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739598+0000", 2024-04-20T01:48:46.119 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00046589800000000002 2024-04-20T01:48:46.120 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.120 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.120 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.120 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739609+0000", 2024-04-20T01:48:46.120 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0737000000000001e-05 2024-04-20T01:48:46.120 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.120 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.120 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.120 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.750185+0000", 2024-04-20T01:48:46.120 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010576370999999999 2024-04-20T01:48:46.120 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.120 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.120 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.120 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.750213+0000", 2024-04-20T01:48:46.121 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7304000000000002e-05 2024-04-20T01:48:46.121 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.121 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.121 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.121 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.121 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.121 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s5 75/30 ECSubWrite(tid=259, reqid=osd.4.0:259, at_version=75'23, trim_to=0'0, roll_forward_to=68'19))", 2024-04-20T01:48:46.121 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.740102+0000", 2024-04-20T01:48:46.121 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 29.317647580999999, 2024-04-20T01:48:46.121 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011279176, 2024-04-20T01:48:46.121 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.121 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.121 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.121 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.122 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.122 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740102+0000", 2024-04-20T01:48:46.122 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.122 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.122 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.122 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.122 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740102+0000", 2024-04-20T01:48:46.122 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.122 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.122 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.122 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.122 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740101+0000", 2024-04-20T01:48:46.122 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:46.122 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.122 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.123 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.123 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740123+0000", 2024-04-20T01:48:46.123 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.1708e-05 2024-04-20T01:48:46.123 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.123 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.123 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.123 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740135+0000", 2024-04-20T01:48:46.123 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1776999999999999e-05 2024-04-20T01:48:46.123 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.123 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.123 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.123 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740142+0000", 2024-04-20T01:48:46.123 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.956e-06 2024-04-20T01:48:46.123 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.124 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.124 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.124 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742932+0000", 2024-04-20T01:48:46.124 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0027900440000000002 2024-04-20T01:48:46.124 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.124 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.124 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.124 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742942+0000", 2024-04-20T01:48:46.124 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0491e-05 2024-04-20T01:48:46.124 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.124 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.124 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.124 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.751351+0000", 2024-04-20T01:48:46.124 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0084087229999999999 2024-04-20T01:48:46.124 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.125 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.125 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.125 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.751381+0000", 2024-04-20T01:48:46.125 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0173999999999999e-05 2024-04-20T01:48:46.125 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.125 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.125 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.125 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.125 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.125 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s5 80/30 ECSubWrite(tid=309, reqid=osd.4.0:309, at_version=80'32, trim_to=0'0, roll_forward_to=79'28))", 2024-04-20T01:48:46.125 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:21.742879+0000", 2024-04-20T01:48:46.125 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 24.314870804999998, 2024-04-20T01:48:46.125 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0092419359999999992, 2024-04-20T01:48:46.125 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.125 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.125 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.126 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.126 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.126 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.742879+0000", 2024-04-20T01:48:46.126 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.126 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.126 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.126 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.126 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.742879+0000", 2024-04-20T01:48:46.126 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.126 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.126 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.126 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.126 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.742878+0000", 2024-04-20T01:48:46.126 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:46.126 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.127 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.127 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.127 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.742903+0000", 2024-04-20T01:48:46.127 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5403000000000001e-05 2024-04-20T01:48:46.127 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.127 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.127 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.127 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.742913+0000", 2024-04-20T01:48:46.127 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6390000000000004e-06 2024-04-20T01:48:46.127 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.127 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.127 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.128 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.742919+0000", 2024-04-20T01:48:46.128 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.1770000000000003e-06 2024-04-20T01:48:46.128 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.128 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.128 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.128 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744284+0000", 2024-04-20T01:48:46.128 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0013651189999999999 2024-04-20T01:48:46.128 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.128 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.128 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.128 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744295+0000", 2024-04-20T01:48:46.128 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0796e-05 2024-04-20T01:48:46.128 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.128 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.128 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.129 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.752095+0000", 2024-04-20T01:48:46.129 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0077997589999999999 2024-04-20T01:48:46.129 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.129 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.129 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.129 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.752120+0000", 2024-04-20T01:48:46.129 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.5584000000000001e-05 2024-04-20T01:48:46.129 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.129 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.129 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.129 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.129 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.129 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:998 3.8s0 3:16aac79f:::smithi16579500-14:head [read 0~1982464,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:46.129 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:31.792901+0000", 2024-04-20T01:48:46.129 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.264848138, 2024-04-20T01:48:46.130 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.94456701399999998, 2024-04-20T01:48:46.130 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.130 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:46.130 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:46.130 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:46.130 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:46.130 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 998 2024-04-20T01:48:46.130 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.130 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.130 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.130 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.130 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792901+0000", 2024-04-20T01:48:46.130 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.130 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.130 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.131 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.131 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792901+0000", 2024-04-20T01:48:46.131 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.131 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.131 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.131 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.131 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792900+0000", 2024-04-20T01:48:46.131 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:46.131 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.131 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.131 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.131 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792908+0000", 2024-04-20T01:48:46.131 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.2949999999999994e-06 2024-04-20T01:48:46.131 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.131 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.131 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.132 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792921+0000", 2024-04-20T01:48:46.132 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2738999999999999e-05 2024-04-20T01:48:46.132 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.132 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.132 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.132 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792931+0000", 2024-04-20T01:48:46.132 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0781000000000001e-05 2024-04-20T01:48:46.132 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.132 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.132 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.132 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.793020+0000", 2024-04-20T01:48:46.132 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8127000000000005e-05 2024-04-20T01:48:46.132 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.132 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.132 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for peered", 2024-04-20T01:48:46.132 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.793023+0000", 2024-04-20T01:48:46.133 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.5870000000000001e-06 2024-04-20T01:48:46.133 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.133 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.133 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.133 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.729818+0000", 2024-04-20T01:48:46.133 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.93679480800000003 2024-04-20T01:48:46.133 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.133 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.133 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:46.133 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.730070+0000", 2024-04-20T01:48:46.133 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00025204900000000001 2024-04-20T01:48:46.134 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.134 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.134 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.134 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.737468+0000", 2024-04-20T01:48:46.134 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0073981969999999996 2024-04-20T01:48:46.134 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.134 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.134 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.134 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.134 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.134 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:999 3.8s0 3:16aac79f:::smithi16579500-14:head [read 0~1982464,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:46.134 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:31.792943+0000", 2024-04-20T01:48:46.134 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.264805902999999, 2024-04-20T01:48:46.134 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.94611857099999996, 2024-04-20T01:48:46.134 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.135 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:46.135 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:46.135 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:46.135 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:46.135 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 999 2024-04-20T01:48:46.135 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.135 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.135 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.135 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.135 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792943+0000", 2024-04-20T01:48:46.135 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.135 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.135 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.135 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.135 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792943+0000", 2024-04-20T01:48:46.136 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.136 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.136 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.136 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.136 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792943+0000", 2024-04-20T01:48:46.136 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:46.136 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.136 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.136 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.136 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792947+0000", 2024-04-20T01:48:46.136 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7290000000000002e-06 2024-04-20T01:48:46.136 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.136 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.136 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.136 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792953+0000", 2024-04-20T01:48:46.137 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.8029999999999999e-06 2024-04-20T01:48:46.137 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.137 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.137 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.137 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792959+0000", 2024-04-20T01:48:46.137 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.614e-06 2024-04-20T01:48:46.137 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.138 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.138 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.138 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.793056+0000", 2024-04-20T01:48:46.138 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7196999999999998e-05 2024-04-20T01:48:46.138 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.138 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.138 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for peered", 2024-04-20T01:48:46.138 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.793058+0000", 2024-04-20T01:48:46.139 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.967e-06 2024-04-20T01:48:46.139 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.139 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.139 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.139 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.732554+0000", 2024-04-20T01:48:46.139 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.93949528900000001 2024-04-20T01:48:46.139 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.139 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.139 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:46.139 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.732656+0000", 2024-04-20T01:48:46.140 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000102125 2024-04-20T01:48:46.140 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.140 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.140 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.140 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.739062+0000", 2024-04-20T01:48:46.140 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0064063519999999997 2024-04-20T01:48:46.140 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.140 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.140 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.140 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.140 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.140 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:1000 3.8s0 3:16aac79f:::smithi16579500-14:head [read 0~1982464,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:46.140 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:31.792964+0000", 2024-04-20T01:48:46.140 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 14.264785558, 2024-04-20T01:48:46.140 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.94733045199999999, 2024-04-20T01:48:46.141 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.141 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:46.141 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:46.141 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:46.141 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:46.141 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 1000 2024-04-20T01:48:46.141 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.141 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.141 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.141 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.141 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792964+0000", 2024-04-20T01:48:46.141 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.141 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.141 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.141 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.142 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792964+0000", 2024-04-20T01:48:46.142 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.142 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.142 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.142 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.142 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792963+0000", 2024-04-20T01:48:46.142 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:46.142 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.142 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.142 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.142 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792974+0000", 2024-04-20T01:48:46.142 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1066999999999999e-05 2024-04-20T01:48:46.142 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.142 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.143 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.143 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792981+0000", 2024-04-20T01:48:46.143 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.3940000000000001e-06 2024-04-20T01:48:46.143 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.143 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.143 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.143 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.792986+0000", 2024-04-20T01:48:46.143 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6169999999999998e-06 2024-04-20T01:48:46.143 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.143 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.143 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.143 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.793087+0000", 2024-04-20T01:48:46.143 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00010083 2024-04-20T01:48:46.143 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.143 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.143 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for peered", 2024-04-20T01:48:46.144 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:31.793088+0000", 2024-04-20T01:48:46.144 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2299999999999999e-07 2024-04-20T01:48:46.144 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.144 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.144 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.144 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.733410+0000", 2024-04-20T01:48:46.144 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.94032182600000003 2024-04-20T01:48:46.144 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.144 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.144 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:46.144 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.733498+0000", 2024-04-20T01:48:46.144 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.7819999999999996e-05 2024-04-20T01:48:46.144 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.144 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.144 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.145 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.740294+0000", 2024-04-20T01:48:46.145 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0067963729999999997 2024-04-20T01:48:46.145 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.145 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.145 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.145 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.145 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.145 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:1011 3.8s0 3:1746945e:::smithi16579500-24:head [rollback 31] snapc 38=[35,31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-04-20T01:48:46.145 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:32.727422+0000", 2024-04-20T01:48:46.145 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 13.330327793, 2024-04-20T01:48:46.145 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.014864550000000001, 2024-04-20T01:48:46.145 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.145 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:46.145 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:46.145 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:46.146 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:46.146 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 1011 2024-04-20T01:48:46.146 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.146 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.146 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.146 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.146 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.727422+0000", 2024-04-20T01:48:46.146 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.146 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.147 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.147 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.147 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.727422+0000", 2024-04-20T01:48:46.147 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.147 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.147 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.147 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.147 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.727420+0000", 2024-04-20T01:48:46.147 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:46.147 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.147 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.147 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.147 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.727426+0000", 2024-04-20T01:48:46.147 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.494e-06 2024-04-20T01:48:46.148 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.148 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.148 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.148 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.727442+0000", 2024-04-20T01:48:46.148 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5877999999999999e-05 2024-04-20T01:48:46.148 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.148 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.148 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.148 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.727453+0000", 2024-04-20T01:48:46.148 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0839000000000001e-05 2024-04-20T01:48:46.148 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.148 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.148 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.148 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.727512+0000", 2024-04-20T01:48:46.148 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.9338e-05 2024-04-20T01:48:46.149 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.149 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.149 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for peered", 2024-04-20T01:48:46.149 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.727516+0000", 2024-04-20T01:48:46.149 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7850000000000002e-06 2024-04-20T01:48:46.149 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.149 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.149 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.149 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.733732+0000", 2024-04-20T01:48:46.149 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0062156850000000003 2024-04-20T01:48:46.149 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.149 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.149 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:46.149 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.733867+0000", 2024-04-20T01:48:46.149 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00013472399999999999 2024-04-20T01:48:46.150 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.150 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.150 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.150 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.742286+0000", 2024-04-20T01:48:46.150 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0084195599999999995 2024-04-20T01:48:46.150 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.150 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.150 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.150 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.150 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.150 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:46.152 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7: {} 2024-04-20T01:48:46.152 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8: {} 2024-04-20T01:48:46.152 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9: {} 2024-04-20T01:48:46.152 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10: {} 2024-04-20T01:48:46.152 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11: {} 2024-04-20T01:48:46.153 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-20T01:48:46.160 INFO:teuthology.orchestra.run.smithi044.stdout:osd.12: {} 2024-04-20T01:48:46.180 INFO:teuthology.orchestra.run.smithi044.stdout:osd.13: {} 2024-04-20T01:48:46.202 INFO:teuthology.orchestra.run.smithi044.stdout:osd.15: {} 2024-04-20T01:48:46.202 INFO:teuthology.orchestra.run.smithi044.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-20T01:48:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-20T01:48:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-20T01:48:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-20T01:48:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-20T01:48:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-20T01:48:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-20T01:48:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-20T01:48:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-20T01:48:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-20T01:48:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-20T01:48:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-20T01:48:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-20T01:48:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-20T01:48:46.203 INFO:teuthology.orchestra.run.smithi044.stderr:Error ENXIO: problem getting command descriptions from osd.14 2024-04-20T01:48:46.204 INFO:teuthology.orchestra.run.smithi044.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-20T01:48:46.205 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-20T01:48:46.206 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-20T01:48:46.282 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:46.282 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:46.282 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 0 2024-04-20T01:48:46.283 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:46.292 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-20T01:48:46.428 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:46.428 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:46.428 INFO:teuthology.orchestra.run.smithi121.stdout: "complaint_time": 30, 2024-04-20T01:48:46.428 INFO:teuthology.orchestra.run.smithi121.stdout: "num_blocked_ops": 0 2024-04-20T01:48:46.428 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:46.438 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-20T01:48:46.566 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:46.566 INFO:teuthology.orchestra.run.smithi121.stdout: "size": 20, 2024-04-20T01:48:46.566 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 600, 2024-04-20T01:48:46.566 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-04-20T01:48:46.566 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.566 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.es1 47/30 ECSubWrite(tid=40, reqid=osd.11.0:39, at_version=0'0, trim_to=0'0, roll_forward_to=47'23))", 2024-04-20T01:48:46.567 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:48.721119+0000", 2024-04-20T01:48:46.567 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 57.858604514, 2024-04-20T01:48:46.567 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0069945759999999997, 2024-04-20T01:48:46.567 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.567 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.567 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.567 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.567 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.567 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721119+0000", 2024-04-20T01:48:46.568 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.568 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.568 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.568 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.568 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721119+0000", 2024-04-20T01:48:46.568 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.568 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.568 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.568 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.568 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721119+0000", 2024-04-20T01:48:46.568 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:46.568 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.568 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.568 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.568 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721128+0000", 2024-04-20T01:48:46.569 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.0620000000000007e-06 2024-04-20T01:48:46.569 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.569 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.569 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.569 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721138+0000", 2024-04-20T01:48:46.569 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0771000000000001e-05 2024-04-20T01:48:46.569 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.569 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.569 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.569 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721146+0000", 2024-04-20T01:48:46.569 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.3370000000000002e-06 2024-04-20T01:48:46.570 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.570 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.570 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.570 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721183+0000", 2024-04-20T01:48:46.570 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7283000000000002e-05 2024-04-20T01:48:46.570 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.570 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.570 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.570 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.721194+0000", 2024-04-20T01:48:46.570 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1093e-05 2024-04-20T01:48:46.571 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.571 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.571 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.571 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.728091+0000", 2024-04-20T01:48:46.571 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0068972499999999997 2024-04-20T01:48:46.571 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.571 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.571 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.571 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:48.728114+0000", 2024-04-20T01:48:46.571 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2410000000000001e-05 2024-04-20T01:48:46.571 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.571 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.571 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.571 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.571 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.572 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.6s6 68/30 ECSubWrite(tid=20, reqid=client.4643.0:643, at_version=68'13, trim_to=0'0, roll_forward_to=68'10))", 2024-04-20T01:48:46.572 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:09.878445+0000", 2024-04-20T01:48:46.572 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 36.701278756999997, 2024-04-20T01:48:46.572 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00751251, 2024-04-20T01:48:46.572 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.572 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.572 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.572 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.572 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.572 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.878445+0000", 2024-04-20T01:48:46.572 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.572 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.572 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.572 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.573 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.878445+0000", 2024-04-20T01:48:46.573 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.573 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.573 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.573 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.573 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.878443+0000", 2024-04-20T01:48:46.573 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:46.573 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.573 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.573 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.573 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.878670+0000", 2024-04-20T01:48:46.573 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00022695199999999999 2024-04-20T01:48:46.573 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.574 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.574 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.574 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.878680+0000", 2024-04-20T01:48:46.574 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.217e-06 2024-04-20T01:48:46.574 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.574 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.574 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.574 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.878688+0000", 2024-04-20T01:48:46.574 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.6000000000000007e-06 2024-04-20T01:48:46.574 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.574 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.574 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.574 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.878733+0000", 2024-04-20T01:48:46.574 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.4490000000000003e-05 2024-04-20T01:48:46.575 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.575 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.575 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.575 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.878743+0000", 2024-04-20T01:48:46.575 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0294e-05 2024-04-20T01:48:46.575 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.575 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.575 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.575 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.885926+0000", 2024-04-20T01:48:46.575 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0071831669999999999 2024-04-20T01:48:46.575 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.575 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.575 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.575 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.885957+0000", 2024-04-20T01:48:46.576 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.1411999999999997e-05 2024-04-20T01:48:46.577 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.577 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.577 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.577 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.577 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.577 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.6s6 68/30 ECSubWrite(tid=22, reqid=client.4643.0:645, at_version=68'15, trim_to=0'0, roll_forward_to=68'10))", 2024-04-20T01:48:46.577 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:09.880701+0000", 2024-04-20T01:48:46.577 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 36.699022329000002, 2024-04-20T01:48:46.577 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0082028380000000005, 2024-04-20T01:48:46.577 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.577 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.578 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.578 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.578 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.578 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.880701+0000", 2024-04-20T01:48:46.578 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.578 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.578 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.578 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.578 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.880701+0000", 2024-04-20T01:48:46.578 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.578 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.579 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.579 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.579 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.880701+0000", 2024-04-20T01:48:46.579 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:46.579 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.579 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.579 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.579 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.880714+0000", 2024-04-20T01:48:46.579 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3468999999999999e-05 2024-04-20T01:48:46.579 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.579 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.579 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.579 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.880726+0000", 2024-04-20T01:48:46.579 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1727e-05 2024-04-20T01:48:46.580 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.580 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.580 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.580 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.880732+0000", 2024-04-20T01:48:46.580 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.3829999999999997e-06 2024-04-20T01:48:46.580 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.580 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.580 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.580 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.880785+0000", 2024-04-20T01:48:46.580 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.2932000000000001e-05 2024-04-20T01:48:46.580 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.580 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.580 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.580 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.880807+0000", 2024-04-20T01:48:46.581 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.143e-05 2024-04-20T01:48:46.581 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.581 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.581 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.581 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.888877+0000", 2024-04-20T01:48:46.581 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0080705440000000007 2024-04-20T01:48:46.581 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.581 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.581 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.581 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:09.888904+0000", 2024-04-20T01:48:46.581 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7033999999999999e-05 2024-04-20T01:48:46.581 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.581 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.581 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.581 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.582 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.582 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 68/30 ECSubWrite(tid=216, reqid=client.4643.0:684, at_version=68'115, trim_to=0'0, roll_forward_to=68'113))", 2024-04-20T01:48:46.582 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:10.004517+0000", 2024-04-20T01:48:46.582 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 36.575206661999999, 2024-04-20T01:48:46.582 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0078424040000000007, 2024-04-20T01:48:46.582 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.582 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.582 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.582 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.582 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.582 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.004517+0000", 2024-04-20T01:48:46.582 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.582 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.582 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.583 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.583 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.004517+0000", 2024-04-20T01:48:46.583 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.583 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.583 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.583 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.583 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.004516+0000", 2024-04-20T01:48:46.583 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:46.583 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.583 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.583 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.583 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.004540+0000", 2024-04-20T01:48:46.583 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3144999999999999e-05 2024-04-20T01:48:46.583 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.583 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.584 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.584 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.004551+0000", 2024-04-20T01:48:46.584 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1516e-05 2024-04-20T01:48:46.584 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.584 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.584 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.584 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.004557+0000", 2024-04-20T01:48:46.584 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.8080000000000001e-06 2024-04-20T01:48:46.584 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.584 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.584 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.584 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.004601+0000", 2024-04-20T01:48:46.585 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.3911e-05 2024-04-20T01:48:46.585 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.585 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.585 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.585 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.004611+0000", 2024-04-20T01:48:46.585 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0404000000000001e-05 2024-04-20T01:48:46.585 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.585 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.585 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.585 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.012340+0000", 2024-04-20T01:48:46.585 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0077283819999999998 2024-04-20T01:48:46.585 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.585 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.586 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.586 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:10.012359+0000", 2024-04-20T01:48:46.586 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9862999999999999e-05 2024-04-20T01:48:46.586 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.586 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.586 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.586 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.586 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.586 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.es1 70/30 ECSubWrite(tid=82, reqid=client.4643.0:728, at_version=70'61, trim_to=0'0, roll_forward_to=68'58))", 2024-04-20T01:48:46.586 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:11.821013+0000", 2024-04-20T01:48:46.586 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 34.758710358999998, 2024-04-20T01:48:46.586 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0073864279999999996, 2024-04-20T01:48:46.586 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.587 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.587 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.587 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.587 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.587 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.821013+0000", 2024-04-20T01:48:46.587 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.587 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.587 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.587 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.587 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.821013+0000", 2024-04-20T01:48:46.587 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.587 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.587 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.587 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.588 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.821013+0000", 2024-04-20T01:48:46.588 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:46.588 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.588 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.588 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.588 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.821466+0000", 2024-04-20T01:48:46.588 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00045382300000000001 2024-04-20T01:48:46.588 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.588 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.588 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.588 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.821482+0000", 2024-04-20T01:48:46.588 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5341000000000001e-05 2024-04-20T01:48:46.588 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.588 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.588 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.589 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.821489+0000", 2024-04-20T01:48:46.589 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.4819999999999997e-06 2024-04-20T01:48:46.589 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.589 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.589 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.589 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.821535+0000", 2024-04-20T01:48:46.589 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5583e-05 2024-04-20T01:48:46.589 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.589 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.589 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.589 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.821542+0000", 2024-04-20T01:48:46.589 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.3830000000000003e-06 2024-04-20T01:48:46.589 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.589 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.589 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.589 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.828362+0000", 2024-04-20T01:48:46.590 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0068202860000000001 2024-04-20T01:48:46.590 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.590 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.590 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.590 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.828400+0000", 2024-04-20T01:48:46.590 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7379999999999998e-05 2024-04-20T01:48:46.590 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.590 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.590 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.590 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.590 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.590 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.es1 70/30 ECSubWrite(tid=83, reqid=client.4643.0:729, at_version=70'62, trim_to=0'0, roll_forward_to=68'58))", 2024-04-20T01:48:46.590 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:11.823811+0000", 2024-04-20T01:48:46.590 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 34.755912324000001, 2024-04-20T01:48:46.591 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.007216582, 2024-04-20T01:48:46.591 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.591 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.591 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.591 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.591 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.591 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.823811+0000", 2024-04-20T01:48:46.591 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.591 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.591 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.591 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.591 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.823811+0000", 2024-04-20T01:48:46.591 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.591 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.591 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.592 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.592 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.823811+0000", 2024-04-20T01:48:46.592 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:46.592 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.592 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.592 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.592 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.824011+0000", 2024-04-20T01:48:46.592 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000200216 2024-04-20T01:48:46.592 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.592 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.592 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.592 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.824026+0000", 2024-04-20T01:48:46.592 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4861999999999999e-05 2024-04-20T01:48:46.592 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.592 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.593 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.593 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.824034+0000", 2024-04-20T01:48:46.593 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.6549999999999993e-06 2024-04-20T01:48:46.593 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.593 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.593 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.593 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.824091+0000", 2024-04-20T01:48:46.593 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6532e-05 2024-04-20T01:48:46.593 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.593 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.593 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.593 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.824098+0000", 2024-04-20T01:48:46.593 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0389999999999999e-06 2024-04-20T01:48:46.593 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.593 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.594 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.594 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.831007+0000", 2024-04-20T01:48:46.594 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0069089640000000001 2024-04-20T01:48:46.594 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.594 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.594 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.594 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.831028+0000", 2024-04-20T01:48:46.594 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0982e-05 2024-04-20T01:48:46.594 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.594 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.594 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.594 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.594 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.594 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.es1 70/30 ECSubWrite(tid=84, reqid=client.4643.0:730, at_version=70'63, trim_to=0'0, roll_forward_to=68'58))", 2024-04-20T01:48:46.595 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:11.826634+0000", 2024-04-20T01:48:46.595 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 34.753089873, 2024-04-20T01:48:46.595 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0073249450000000002, 2024-04-20T01:48:46.595 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.595 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.595 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.595 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.595 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.595 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.826634+0000", 2024-04-20T01:48:46.595 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.595 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.595 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.595 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.595 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.826634+0000", 2024-04-20T01:48:46.595 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.595 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.596 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.596 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.596 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.826633+0000", 2024-04-20T01:48:46.596 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:46.596 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.596 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.596 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.596 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.826728+0000", 2024-04-20T01:48:46.596 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.5006999999999993e-05 2024-04-20T01:48:46.596 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.596 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.596 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.596 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.826742+0000", 2024-04-20T01:48:46.596 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.361e-05 2024-04-20T01:48:46.597 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.597 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.597 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.597 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.826751+0000", 2024-04-20T01:48:46.597 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7529999999999996e-06 2024-04-20T01:48:46.597 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.597 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.597 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.597 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.826793+0000", 2024-04-20T01:48:46.597 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.1112999999999998e-05 2024-04-20T01:48:46.597 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.597 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.597 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.597 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.826800+0000", 2024-04-20T01:48:46.597 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.4039999999999998e-06 2024-04-20T01:48:46.598 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.598 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.598 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.598 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.833928+0000", 2024-04-20T01:48:46.598 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0071284549999999997 2024-04-20T01:48:46.598 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.598 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.598 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.598 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:11.833959+0000", 2024-04-20T01:48:46.598 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0426e-05 2024-04-20T01:48:46.598 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.598 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.598 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.598 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.598 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.599 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.1s2 75/30 ECSubWrite(tid=253, reqid=osd.4.0:253, at_version=75'37, trim_to=0'0, roll_forward_to=73'35))", 2024-04-20T01:48:46.599 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.725657+0000", 2024-04-20T01:48:46.599 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 29.854066233000001, 2024-04-20T01:48:46.599 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0071635270000000003, 2024-04-20T01:48:46.599 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.599 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.599 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.599 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.599 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.599 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.725657+0000", 2024-04-20T01:48:46.599 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.599 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.599 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.599 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.599 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.725657+0000", 2024-04-20T01:48:46.600 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.600 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.600 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.600 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.600 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.725657+0000", 2024-04-20T01:48:46.600 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967296 2024-04-20T01:48:46.600 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.600 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.600 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.600 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.725677+0000", 2024-04-20T01:48:46.600 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0139000000000001e-05 2024-04-20T01:48:46.600 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.600 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.600 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.600 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.725690+0000", 2024-04-20T01:48:46.601 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2459e-05 2024-04-20T01:48:46.601 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.601 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.601 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.601 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.725699+0000", 2024-04-20T01:48:46.601 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8529999999999998e-06 2024-04-20T01:48:46.601 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.601 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.601 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.601 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.725775+0000", 2024-04-20T01:48:46.601 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.6748000000000004e-05 2024-04-20T01:48:46.601 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.601 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.601 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.601 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.725786+0000", 2024-04-20T01:48:46.602 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0428000000000001e-05 2024-04-20T01:48:46.602 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.602 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.602 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.602 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.732794+0000", 2024-04-20T01:48:46.602 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0070079160000000003 2024-04-20T01:48:46.602 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.602 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.602 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.602 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.732821+0000", 2024-04-20T01:48:46.602 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7195e-05 2024-04-20T01:48:46.602 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.602 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.602 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.603 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.603 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.603 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 75/30 ECSubWrite(tid=254, reqid=osd.4.0:254, at_version=75'133, trim_to=0'0, roll_forward_to=73'131))", 2024-04-20T01:48:46.603 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.727466+0000", 2024-04-20T01:48:46.603 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 29.852257615999999, 2024-04-20T01:48:46.603 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.012011501000000001, 2024-04-20T01:48:46.603 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.603 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.603 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.603 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.603 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.603 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.727466+0000", 2024-04-20T01:48:46.603 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.603 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.603 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.604 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.604 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.727466+0000", 2024-04-20T01:48:46.604 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.604 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.604 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.604 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.604 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.727465+0000", 2024-04-20T01:48:46.604 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:46.604 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.604 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.604 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.604 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.727485+0000", 2024-04-20T01:48:46.604 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0041999999999999e-05 2024-04-20T01:48:46.604 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.604 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.605 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.605 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.727495+0000", 2024-04-20T01:48:46.605 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0127e-05 2024-04-20T01:48:46.605 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.605 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.605 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.605 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.727502+0000", 2024-04-20T01:48:46.605 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.6019999999999996e-06 2024-04-20T01:48:46.605 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.605 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.605 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.605 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.727540+0000", 2024-04-20T01:48:46.605 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.7867999999999997e-05 2024-04-20T01:48:46.605 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.606 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.606 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.606 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.727550+0000", 2024-04-20T01:48:46.606 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6129999999999993e-06 2024-04-20T01:48:46.606 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.606 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.606 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.606 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739449+0000", 2024-04-20T01:48:46.606 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011899442 2024-04-20T01:48:46.606 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.606 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.606 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.606 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739478+0000", 2024-04-20T01:48:46.606 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8643e-05 2024-04-20T01:48:46.607 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.607 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.607 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.607 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.607 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.607 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 75/30 ECSubWrite(tid=255, reqid=osd.4.0:255, at_version=75'135, trim_to=0'0, roll_forward_to=73'131))", 2024-04-20T01:48:46.607 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.729295+0000", 2024-04-20T01:48:46.607 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 29.850429024, 2024-04-20T01:48:46.607 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.012091924, 2024-04-20T01:48:46.607 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.607 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.607 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.607 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.607 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.607 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.729295+0000", 2024-04-20T01:48:46.608 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.608 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.608 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.608 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.608 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.729295+0000", 2024-04-20T01:48:46.608 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.608 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.608 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.608 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.608 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.729294+0000", 2024-04-20T01:48:46.608 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:46.608 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.608 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.608 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.608 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.729312+0000", 2024-04-20T01:48:46.609 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8488e-05 2024-04-20T01:48:46.609 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.609 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.609 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.609 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.729325+0000", 2024-04-20T01:48:46.609 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2143999999999999e-05 2024-04-20T01:48:46.609 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.609 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.609 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.609 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.729334+0000", 2024-04-20T01:48:46.609 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3149999999999998e-06 2024-04-20T01:48:46.609 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.609 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.609 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.610 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730067+0000", 2024-04-20T01:48:46.610 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00073259599999999996 2024-04-20T01:48:46.610 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.610 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.610 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.610 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730078+0000", 2024-04-20T01:48:46.610 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1635e-05 2024-04-20T01:48:46.610 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.610 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.610 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.610 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.741357+0000", 2024-04-20T01:48:46.610 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011278923999999999 2024-04-20T01:48:46.610 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.610 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.611 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.611 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.741387+0000", 2024-04-20T01:48:46.611 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9530999999999999e-05 2024-04-20T01:48:46.611 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.611 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.611 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.611 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.611 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.611 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 75/30 ECSubWrite(tid=256, reqid=client.4643.0:783, at_version=75'137, trim_to=0'0, roll_forward_to=73'131))", 2024-04-20T01:48:46.611 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.730687+0000", 2024-04-20T01:48:46.611 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 29.849036382000001, 2024-04-20T01:48:46.611 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.019480198000000001, 2024-04-20T01:48:46.611 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.611 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.611 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.612 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.612 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.612 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730687+0000", 2024-04-20T01:48:46.612 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.612 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.612 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.612 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.612 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730687+0000", 2024-04-20T01:48:46.612 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.612 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.612 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.612 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.612 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730687+0000", 2024-04-20T01:48:46.613 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:46.613 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.613 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.613 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.613 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730847+0000", 2024-04-20T01:48:46.613 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00016085900000000001 2024-04-20T01:48:46.613 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.613 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.613 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.613 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730863+0000", 2024-04-20T01:48:46.613 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5324000000000001e-05 2024-04-20T01:48:46.613 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.613 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.613 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.614 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.730873+0000", 2024-04-20T01:48:46.614 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0059e-05 2024-04-20T01:48:46.614 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.614 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.614 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.614 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.733352+0000", 2024-04-20T01:48:46.614 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0024793559999999998 2024-04-20T01:48:46.614 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.614 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.614 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.614 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.733363+0000", 2024-04-20T01:48:46.614 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0456e-05 2024-04-20T01:48:46.614 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.615 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.615 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.615 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.750144+0000", 2024-04-20T01:48:46.615 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.016780970999999999 2024-04-20T01:48:46.615 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.615 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.615 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.615 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.750168+0000", 2024-04-20T01:48:46.615 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3966999999999999e-05 2024-04-20T01:48:46.615 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.615 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.615 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.615 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.615 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.616 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.es1 75/30 ECSubWrite(tid=102, reqid=osd.11.0:101, at_version=0'0, trim_to=0'0, roll_forward_to=75'78))", 2024-04-20T01:48:46.616 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.738230+0000", 2024-04-20T01:48:46.616 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 29.841493922000002, 2024-04-20T01:48:46.616 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0089712109999999998, 2024-04-20T01:48:46.616 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.616 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.616 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.616 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.616 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.616 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738230+0000", 2024-04-20T01:48:46.616 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.616 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.616 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.617 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.617 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738230+0000", 2024-04-20T01:48:46.617 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.617 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.617 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.617 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.617 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738229+0000", 2024-04-20T01:48:46.617 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:46.617 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.617 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.617 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.617 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738241+0000", 2024-04-20T01:48:46.617 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2243e-05 2024-04-20T01:48:46.617 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.617 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.618 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.618 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738255+0000", 2024-04-20T01:48:46.618 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4003e-05 2024-04-20T01:48:46.618 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.618 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.618 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.618 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738263+0000", 2024-04-20T01:48:46.618 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.2330000000000006e-06 2024-04-20T01:48:46.618 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.618 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.618 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.618 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738309+0000", 2024-04-20T01:48:46.618 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.6035999999999998e-05 2024-04-20T01:48:46.618 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.618 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.619 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.619 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738321+0000", 2024-04-20T01:48:46.619 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1874e-05 2024-04-20T01:48:46.619 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.619 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.619 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.619 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747172+0000", 2024-04-20T01:48:46.619 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0088509569999999996 2024-04-20T01:48:46.619 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.619 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.619 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.619 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747201+0000", 2024-04-20T01:48:46.619 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.8683e-05 2024-04-20T01:48:46.620 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.620 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.620 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.620 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.620 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.620 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s2 75/30 ECSubWrite(tid=258, reqid=osd.4.0:258, at_version=75'21, trim_to=0'0, roll_forward_to=68'19))", 2024-04-20T01:48:46.620 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.738933+0000", 2024-04-20T01:48:46.620 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 29.840790520999999, 2024-04-20T01:48:46.620 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0088958019999999995, 2024-04-20T01:48:46.620 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.620 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.620 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.620 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.621 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.621 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738933+0000", 2024-04-20T01:48:46.621 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.621 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.621 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.621 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.621 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738933+0000", 2024-04-20T01:48:46.621 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.621 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.621 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.621 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.621 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738932+0000", 2024-04-20T01:48:46.621 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:46.621 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738957+0000", 2024-04-20T01:48:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4559e-05 2024-04-20T01:48:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738970+0000", 2024-04-20T01:48:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.349e-05 2024-04-20T01:48:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.738980+0000", 2024-04-20T01:48:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.3149999999999998e-06 2024-04-20T01:48:46.622 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.623 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.623 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.623 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739052+0000", 2024-04-20T01:48:46.623 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.2657999999999995e-05 2024-04-20T01:48:46.623 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.623 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.623 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.623 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.739062+0000", 2024-04-20T01:48:46.623 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7569999999999999e-06 2024-04-20T01:48:46.623 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.623 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.623 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.623 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747798+0000", 2024-04-20T01:48:46.623 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0087363609999999998 2024-04-20T01:48:46.624 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.624 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.624 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.624 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747829+0000", 2024-04-20T01:48:46.624 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0534000000000003e-05 2024-04-20T01:48:46.624 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.624 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.624 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.624 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.624 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.624 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s2 75/30 ECSubWrite(tid=259, reqid=osd.4.0:259, at_version=75'23, trim_to=0'0, roll_forward_to=68'19))", 2024-04-20T01:48:46.624 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.740125+0000", 2024-04-20T01:48:46.624 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 29.839599047, 2024-04-20T01:48:46.624 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0094825380000000004, 2024-04-20T01:48:46.625 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.625 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.625 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.625 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.625 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.625 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740125+0000", 2024-04-20T01:48:46.625 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.625 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.625 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.625 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.625 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740125+0000", 2024-04-20T01:48:46.625 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.625 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.625 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.625 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.626 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740124+0000", 2024-04-20T01:48:46.626 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:46.626 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.626 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.626 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.626 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740152+0000", 2024-04-20T01:48:46.626 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7498e-05 2024-04-20T01:48:46.626 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.626 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.626 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.626 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740167+0000", 2024-04-20T01:48:46.626 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5787e-05 2024-04-20T01:48:46.626 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.626 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.627 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.627 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740177+0000", 2024-04-20T01:48:46.627 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.4730000000000006e-06 2024-04-20T01:48:46.627 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.627 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.627 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.627 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740234+0000", 2024-04-20T01:48:46.627 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6827999999999999e-05 2024-04-20T01:48:46.627 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.627 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.627 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.627 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.740241+0000", 2024-04-20T01:48:46.627 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.3810000000000001e-06 2024-04-20T01:48:46.628 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.628 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.628 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.628 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.749580+0000", 2024-04-20T01:48:46.628 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0093387439999999995 2024-04-20T01:48:46.628 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.628 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.628 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.628 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.749607+0000", 2024-04-20T01:48:46.628 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7492999999999999e-05 2024-04-20T01:48:46.628 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.628 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.628 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.628 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.628 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.629 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 75/30 ECSubWrite(tid=260, reqid=osd.4.0:260, at_version=75'139, trim_to=0'0, roll_forward_to=75'135))", 2024-04-20T01:48:46.629 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.742474+0000", 2024-04-20T01:48:46.629 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 29.837249525000001, 2024-04-20T01:48:46.629 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0080654139999999999, 2024-04-20T01:48:46.629 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.629 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.629 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.629 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.629 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.629 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742474+0000", 2024-04-20T01:48:46.629 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.629 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.629 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.630 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.630 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742474+0000", 2024-04-20T01:48:46.630 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.630 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.630 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.630 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.630 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742473+0000", 2024-04-20T01:48:46.630 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:46.630 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.630 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.630 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.630 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742507+0000", 2024-04-20T01:48:46.630 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.3868000000000002e-05 2024-04-20T01:48:46.631 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.631 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.631 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.631 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742523+0000", 2024-04-20T01:48:46.631 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6126000000000001e-05 2024-04-20T01:48:46.631 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.631 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.631 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.631 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742534+0000", 2024-04-20T01:48:46.631 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.049e-05 2024-04-20T01:48:46.631 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.631 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.631 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.631 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742659+0000", 2024-04-20T01:48:46.632 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00012551000000000001 2024-04-20T01:48:46.632 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.632 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.632 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.632 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.742671+0000", 2024-04-20T01:48:46.632 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1731e-05 2024-04-20T01:48:46.632 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.632 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.632 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.632 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.750515+0000", 2024-04-20T01:48:46.632 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0078435850000000001 2024-04-20T01:48:46.632 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.632 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.632 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.633 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.750540+0000", 2024-04-20T01:48:46.633 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.4887000000000001e-05 2024-04-20T01:48:46.633 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.633 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.633 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.633 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.633 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.633 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 75/30 ECSubWrite(tid=261, reqid=osd.4.0:261, at_version=75'141, trim_to=0'0, roll_forward_to=75'135))", 2024-04-20T01:48:46.633 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.744213+0000", 2024-04-20T01:48:46.633 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 29.835510471999999, 2024-04-20T01:48:46.633 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0088062550000000007, 2024-04-20T01:48:46.633 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.633 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.633 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.634 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.634 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.634 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744213+0000", 2024-04-20T01:48:46.634 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.634 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.634 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.634 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.634 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744213+0000", 2024-04-20T01:48:46.634 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.634 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.634 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.634 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.634 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744212+0000", 2024-04-20T01:48:46.634 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:46.635 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.635 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.635 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.635 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744236+0000", 2024-04-20T01:48:46.635 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3189000000000001e-05 2024-04-20T01:48:46.635 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.635 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.635 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.635 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744260+0000", 2024-04-20T01:48:46.635 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.3845e-05 2024-04-20T01:48:46.635 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.635 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.635 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.635 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.744269+0000", 2024-04-20T01:48:46.636 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2690000000000005e-06 2024-04-20T01:48:46.636 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.636 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.636 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.636 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747687+0000", 2024-04-20T01:48:46.636 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0034180909999999998 2024-04-20T01:48:46.636 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.636 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.636 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.636 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.747694+0000", 2024-04-20T01:48:46.636 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.8580000000000002e-06 2024-04-20T01:48:46.636 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.636 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.636 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.637 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.752999+0000", 2024-04-20T01:48:46.637 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0053053809999999996 2024-04-20T01:48:46.637 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.637 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.637 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.637 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.753020+0000", 2024-04-20T01:48:46.637 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0383000000000001e-05 2024-04-20T01:48:46.637 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.637 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.637 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.637 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.637 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.637 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.1s2 75/30 ECSubWrite(tid=264, reqid=client.4643.0:793, at_version=75'40, trim_to=0'0, roll_forward_to=75'37))", 2024-04-20T01:48:46.638 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.756570+0000", 2024-04-20T01:48:46.638 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 29.823153318999999, 2024-04-20T01:48:46.638 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0075168049999999997, 2024-04-20T01:48:46.638 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.638 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.638 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.638 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.638 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.638 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.756570+0000", 2024-04-20T01:48:46.638 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.638 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.638 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.639 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.639 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.756570+0000", 2024-04-20T01:48:46.639 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.639 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.639 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.639 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.639 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.756570+0000", 2024-04-20T01:48:46.639 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:46.639 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.639 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.639 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.639 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.756590+0000", 2024-04-20T01:48:46.639 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.9802000000000001e-05 2024-04-20T01:48:46.639 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.639 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.640 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.640 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.756609+0000", 2024-04-20T01:48:46.640 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.8992e-05 2024-04-20T01:48:46.640 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.640 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.640 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.640 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.756618+0000", 2024-04-20T01:48:46.640 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7769999999999997e-06 2024-04-20T01:48:46.640 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.640 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.640 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.640 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.756657+0000", 2024-04-20T01:48:46.640 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.8383999999999997e-05 2024-04-20T01:48:46.640 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.640 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.641 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.641 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.756663+0000", 2024-04-20T01:48:46.641 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.0800000000000002e-06 2024-04-20T01:48:46.641 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.641 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.641 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.641 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.764054+0000", 2024-04-20T01:48:46.641 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0073911519999999998 2024-04-20T01:48:46.641 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.641 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.641 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.641 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.764087+0000", 2024-04-20T01:48:46.641 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.3312000000000002e-05 2024-04-20T01:48:46.642 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.642 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.642 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.642 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.642 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.642 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 75/30 ECSubWrite(tid=268, reqid=osd.4.0:268, at_version=75'145, trim_to=0'0, roll_forward_to=75'141))", 2024-04-20T01:48:46.642 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.759435+0000", 2024-04-20T01:48:46.642 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 29.820288229999999, 2024-04-20T01:48:46.642 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0072970989999999996, 2024-04-20T01:48:46.642 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.642 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.642 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.642 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.643 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.643 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.759435+0000", 2024-04-20T01:48:46.643 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.643 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.643 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.643 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.643 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.759435+0000", 2024-04-20T01:48:46.643 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.643 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.643 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.643 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.643 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.759435+0000", 2024-04-20T01:48:46.643 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:46.643 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.643 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.644 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.644 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.759456+0000", 2024-04-20T01:48:46.644 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.0958000000000002e-05 2024-04-20T01:48:46.644 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.644 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.644 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.644 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.759470+0000", 2024-04-20T01:48:46.644 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.4589e-05 2024-04-20T01:48:46.644 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.644 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.644 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.644 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.759479+0000", 2024-04-20T01:48:46.644 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.0999999999999993e-06 2024-04-20T01:48:46.644 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.645 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.645 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.645 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.759526+0000", 2024-04-20T01:48:46.645 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.7120999999999998e-05 2024-04-20T01:48:46.645 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.645 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.645 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.645 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.759536+0000", 2024-04-20T01:48:46.645 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6010000000000001e-06 2024-04-20T01:48:46.645 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.645 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.645 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.645 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.766700+0000", 2024-04-20T01:48:46.645 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0071636 2024-04-20T01:48:46.646 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.646 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.646 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.646 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.766733+0000", 2024-04-20T01:48:46.646 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.2928000000000001e-05 2024-04-20T01:48:46.646 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.646 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.646 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.646 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.646 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.646 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.bs6 80/30 ECSubWrite(tid=310, reqid=osd.4.0:303, at_version=0'0, trim_to=0'0, roll_forward_to=80'162))", 2024-04-20T01:48:46.646 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:21.744639+0000", 2024-04-20T01:48:46.646 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 24.835084325, 2024-04-20T01:48:46.646 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0072014840000000002, 2024-04-20T01:48:46.646 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.647 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.647 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.647 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.647 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.647 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744639+0000", 2024-04-20T01:48:46.647 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.647 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.647 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.647 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.647 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744639+0000", 2024-04-20T01:48:46.647 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.647 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.647 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.647 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.648 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744639+0000", 2024-04-20T01:48:46.648 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:46.648 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.648 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.648 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.648 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744649+0000", 2024-04-20T01:48:46.648 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7189999999999996e-06 2024-04-20T01:48:46.648 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.648 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.648 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.648 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744661+0000", 2024-04-20T01:48:46.648 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2013e-05 2024-04-20T01:48:46.648 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.648 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.648 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.649 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744668+0000", 2024-04-20T01:48:46.649 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.9209999999999992e-06 2024-04-20T01:48:46.649 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.649 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.649 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.649 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744725+0000", 2024-04-20T01:48:46.649 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6833000000000003e-05 2024-04-20T01:48:46.649 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.649 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.649 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.649 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.744738+0000", 2024-04-20T01:48:46.649 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2343e-05 2024-04-20T01:48:46.649 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.650 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.650 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.650 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.751817+0000", 2024-04-20T01:48:46.650 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0070796779999999998 2024-04-20T01:48:46.650 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.650 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.650 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.650 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:21.751841+0000", 2024-04-20T01:48:46.650 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.357e-05 2024-04-20T01:48:46.650 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.650 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.650 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.650 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.651 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.651 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "MOSDECSubOpWrite(3.7s2 91/90 ECSubWrite(tid=345, reqid=client.4643.0:1004, at_version=91'48, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:48:46.651 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:32.734870+0000", 2024-04-20T01:48:46.651 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 13.844853234, 2024-04-20T01:48:46.651 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0079398590000000005, 2024-04-20T01:48:46.651 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:46.651 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:46.651 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:46.651 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.651 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:46.651 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.734870+0000", 2024-04-20T01:48:46.651 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.651 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.651 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.651 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:46.652 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.734870+0000", 2024-04-20T01:48:46.652 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:46.652 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.652 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.652 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:46.652 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.734870+0000", 2024-04-20T01:48:46.652 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:46.652 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.652 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.652 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:46.652 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.735593+0000", 2024-04-20T01:48:46.652 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00072282499999999996 2024-04-20T01:48:46.652 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.652 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.652 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:46.653 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.735606+0000", 2024-04-20T01:48:46.653 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3198e-05 2024-04-20T01:48:46.653 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.653 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.653 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:46.653 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.735614+0000", 2024-04-20T01:48:46.653 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.8439999999999998e-06 2024-04-20T01:48:46.653 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.653 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.653 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:46.653 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.735667+0000", 2024-04-20T01:48:46.653 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3588000000000003e-05 2024-04-20T01:48:46.653 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.653 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.654 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:46.654 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.735679+0000", 2024-04-20T01:48:46.654 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1616e-05 2024-04-20T01:48:46.654 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.654 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.654 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:46.654 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.742779+0000", 2024-04-20T01:48:46.654 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.007100543 2024-04-20T01:48:46.654 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:46.654 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:46.654 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:46.655 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:32.742810+0000", 2024-04-20T01:48:46.655 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0985999999999998e-05 2024-04-20T01:48:46.655 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.655 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.655 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.655 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:46.655 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:46.655 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:46.656 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-20T01:48:46.783 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:46.784 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:46.784 INFO:teuthology.orchestra.run.smithi121.stdout: "num_ops": 0 2024-04-20T01:48:46.784 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:46.793 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-20T01:48:46.831 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-20T01:48:46.922 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:46.922 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [], 2024-04-20T01:48:46.922 INFO:teuthology.orchestra.run.smithi121.stdout: "complaint_time": 30, 2024-04-20T01:48:46.922 INFO:teuthology.orchestra.run.smithi121.stdout: "num_blocked_ops": 0 2024-04-20T01:48:46.922 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:46.931 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-20T01:48:47.061 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T01:48:47.061 INFO:teuthology.orchestra.run.smithi121.stdout: "size": 20, 2024-04-20T01:48:47.061 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 600, 2024-04-20T01:48:47.061 INFO:teuthology.orchestra.run.smithi121.stdout: "ops": [ 2024-04-20T01:48:47.061 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.061 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4227.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-20T01:48:47.062 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:17.004323+0000", 2024-04-20T01:48:47.062 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 90.070401649999994, 2024-04-20T01:48:47.062 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.29931750600000001, 2024-04-20T01:48:47.062 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:47.062 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:47.062 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:47.062 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4227", 2024-04-20T01:48:47.062 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/2652873865", 2024-04-20T01:48:47.062 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 1 2024-04-20T01:48:47.062 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.062 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:47.062 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.062 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:47.063 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004323+0000", 2024-04-20T01:48:47.063 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.063 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.063 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.063 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:47.063 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004323+0000", 2024-04-20T01:48:47.063 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.063 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.063 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.063 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:47.063 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004321+0000", 2024-04-20T01:48:47.063 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:47.063 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.063 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:47.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004337+0000", 2024-04-20T01:48:47.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5866999999999999e-05 2024-04-20T01:48:47.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:47.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004349+0000", 2024-04-20T01:48:47.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1802e-05 2024-04-20T01:48:47.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.064 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.064 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004360+0000", 2024-04-20T01:48:47.064 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1348999999999999e-05 2024-04-20T01:48:47.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.064 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004411+0000", 2024-04-20T01:48:47.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.0385e-05 2024-04-20T01:48:47.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for peered", 2024-04-20T01:48:47.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.004419+0000", 2024-04-20T01:48:47.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4430000000000003e-06 2024-04-20T01:48:47.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.065 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.065 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.065 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.302330+0000", 2024-04-20T01:48:47.065 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.29791090199999998 2024-04-20T01:48:47.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.066 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.066 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:47.066 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.302488+0000", 2024-04-20T01:48:47.066 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00015830799999999999 2024-04-20T01:48:47.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.066 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.066 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for subops from 15", 2024-04-20T01:48:47.066 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.302650+0000", 2024-04-20T01:48:47.066 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00016196900000000001 2024-04-20T01:48:47.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.066 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.066 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "op_commit", 2024-04-20T01:48:47.066 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.303204+0000", 2024-04-20T01:48:47.066 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00055385000000000005 2024-04-20T01:48:47.067 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.067 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.067 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_commit_rec", 2024-04-20T01:48:47.067 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.303574+0000", 2024-04-20T01:48:47.067 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00037030800000000002 2024-04-20T01:48:47.067 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.067 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.067 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:47.067 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.303614+0000", 2024-04-20T01:48:47.067 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.9161e-05 2024-04-20T01:48:47.067 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.067 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.067 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:47.067 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:17.303640+0000", 2024-04-20T01:48:47.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.6752999999999999e-05 2024-04-20T01:48:47.068 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.068 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.068 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.068 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.068 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:18 3.3s0 3:cb777d78:::smithi16579500-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:47.068 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.756072+0000", 2024-04-20T01:48:47.068 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 72.318652243000002, 2024-04-20T01:48:47.068 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013705281, 2024-04-20T01:48:47.068 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:47.068 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:47.068 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:47.069 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:47.069 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.069 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 18 2024-04-20T01:48:47.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.069 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:47.069 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:47.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756072+0000", 2024-04-20T01:48:47.069 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.069 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.069 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:47.069 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756072+0000", 2024-04-20T01:48:47.069 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:47.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756072+0000", 2024-04-20T01:48:47.070 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:47.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756074+0000", 2024-04-20T01:48:47.070 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.148e-06 2024-04-20T01:48:47.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.070 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.070 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:47.070 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756079+0000", 2024-04-20T01:48:47.071 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.4790000000000004e-06 2024-04-20T01:48:47.071 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.071 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.071 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.071 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.756083+0000", 2024-04-20T01:48:47.071 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.4910000000000001e-06 2024-04-20T01:48:47.071 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.071 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.071 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.071 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.758130+0000", 2024-04-20T01:48:47.071 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00204725 2024-04-20T01:48:47.072 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.072 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.072 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:47.072 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.758200+0000", 2024-04-20T01:48:47.072 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0246000000000005e-05 2024-04-20T01:48:47.072 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.072 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.072 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.072 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766738+0000", 2024-04-20T01:48:47.072 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0085377939999999996 2024-04-20T01:48:47.072 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.072 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.073 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:47.073 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.766828+0000", 2024-04-20T01:48:47.073 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.9677000000000002e-05 2024-04-20T01:48:47.073 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.073 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:47.073 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.769777+0000", 2024-04-20T01:48:47.073 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0029495369999999999 2024-04-20T01:48:47.073 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.073 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.073 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.073 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.073 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.073 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:48 3.2s0 3:4efae9ea:::smithi16579500-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:47.073 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.768007+0000", 2024-04-20T01:48:47.074 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 72.306717098999997, 2024-04-20T01:48:47.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010953058, 2024-04-20T01:48:47.074 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:47.074 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:47.074 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:47.074 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:47.074 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.074 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 48 2024-04-20T01:48:47.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.074 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:47.074 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.074 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:47.074 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768007+0000", 2024-04-20T01:48:47.074 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.074 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.075 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:47.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768007+0000", 2024-04-20T01:48:47.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.075 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:47.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768007+0000", 2024-04-20T01:48:47.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:47.075 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.075 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.075 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:47.075 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768011+0000", 2024-04-20T01:48:47.075 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2899999999999996e-06 2024-04-20T01:48:47.076 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.076 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.076 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:47.076 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768018+0000", 2024-04-20T01:48:47.076 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0060000000000003e-06 2024-04-20T01:48:47.076 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.076 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.076 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.076 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.768025+0000", 2024-04-20T01:48:47.076 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.8390000000000001e-06 2024-04-20T01:48:47.076 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.076 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.076 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.076 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.772765+0000", 2024-04-20T01:48:47.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0047401409999999998 2024-04-20T01:48:47.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.077 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:47.077 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.772861+0000", 2024-04-20T01:48:47.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6513000000000002e-05 2024-04-20T01:48:47.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.077 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.077 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.776375+0000", 2024-04-20T01:48:47.077 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0035133679999999998 2024-04-20T01:48:47.077 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.077 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.077 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:47.078 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.776459+0000", 2024-04-20T01:48:47.078 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.4368000000000003e-05 2024-04-20T01:48:47.078 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.078 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.078 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:47.078 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.778960+0000", 2024-04-20T01:48:47.078 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0025012910000000001 2024-04-20T01:48:47.078 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.078 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.078 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.078 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.078 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.078 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:67 3.3s0 3:c86e2dd3:::smithi16579500-23:head [append 0~376832,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:47.078 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.783364+0000", 2024-04-20T01:48:47.078 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 72.291360308999998, 2024-04-20T01:48:47.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013527099000000001, 2024-04-20T01:48:47.079 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:47.079 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:47.079 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:47.079 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:47.079 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.079 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 67 2024-04-20T01:48:47.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.079 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:47.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.079 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:47.079 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783364+0000", 2024-04-20T01:48:47.079 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.079 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.079 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.080 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:47.080 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783364+0000", 2024-04-20T01:48:47.080 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.080 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:47.080 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783355+0000", 2024-04-20T01:48:47.080 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999909 2024-04-20T01:48:47.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.080 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.080 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:47.080 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783704+0000", 2024-04-20T01:48:47.080 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00034900399999999998 2024-04-20T01:48:47.080 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.081 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:47.081 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783720+0000", 2024-04-20T01:48:47.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5464e-05 2024-04-20T01:48:47.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.081 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.081 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783727+0000", 2024-04-20T01:48:47.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.6199999999999999e-06 2024-04-20T01:48:47.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.081 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.081 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.081 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783773+0000", 2024-04-20T01:48:47.081 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5237000000000001e-05 2024-04-20T01:48:47.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.082 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.082 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:47.082 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783881+0000", 2024-04-20T01:48:47.082 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00010799200000000001 2024-04-20T01:48:47.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.082 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.082 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.082 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.784468+0000", 2024-04-20T01:48:47.082 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00058749400000000004 2024-04-20T01:48:47.082 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.082 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.082 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.785849+0000", 2024-04-20T01:48:47.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0013810420000000001 2024-04-20T01:48:47.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.083 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:47.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.796778+0000", 2024-04-20T01:48:47.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010928955000000001 2024-04-20T01:48:47.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.083 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.083 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:47.083 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.796891+0000", 2024-04-20T01:48:47.083 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00011315399999999999 2024-04-20T01:48:47.083 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.083 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.083 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.084 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.084 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:68 3.3s0 3:c86e2dd3:::smithi16579500-23:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:47.084 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.783739+0000", 2024-04-20T01:48:47.084 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 72.290985035000006, 2024-04-20T01:48:47.084 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013359042, 2024-04-20T01:48:47.084 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:47.084 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:47.084 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:47.084 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:47.084 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.084 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 68 2024-04-20T01:48:47.084 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.084 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:47.085 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.085 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:47.085 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783739+0000", 2024-04-20T01:48:47.085 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.085 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.085 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:47.085 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783739+0000", 2024-04-20T01:48:47.085 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.085 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.085 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:47.085 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783739+0000", 2024-04-20T01:48:47.085 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.085 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.086 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.086 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:47.086 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783744+0000", 2024-04-20T01:48:47.086 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.7690000000000004e-06 2024-04-20T01:48:47.086 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.086 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.086 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:47.086 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783750+0000", 2024-04-20T01:48:47.086 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.0719999999999996e-06 2024-04-20T01:48:47.086 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.086 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.086 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.086 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783754+0000", 2024-04-20T01:48:47.087 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.6450000000000004e-06 2024-04-20T01:48:47.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.087 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.087 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.087 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.784916+0000", 2024-04-20T01:48:47.087 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001161477 2024-04-20T01:48:47.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.087 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.087 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:47.087 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.785022+0000", 2024-04-20T01:48:47.087 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000106153 2024-04-20T01:48:47.087 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.087 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.087 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.087 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.785477+0000", 2024-04-20T01:48:47.088 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00045462699999999998 2024-04-20T01:48:47.088 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.088 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.088 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.088 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.787709+0000", 2024-04-20T01:48:47.088 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0022322309999999999 2024-04-20T01:48:47.088 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.088 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.088 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:47.088 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797021+0000", 2024-04-20T01:48:47.088 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0093120730000000006 2024-04-20T01:48:47.088 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.088 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.088 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:47.089 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797098+0000", 2024-04-20T01:48:47.089 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.7526000000000003e-05 2024-04-20T01:48:47.089 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.089 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.089 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.089 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.089 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.089 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:69 3.3s0 3:c86e2dd3:::smithi16579500-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:47.089 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.783758+0000", 2024-04-20T01:48:47.089 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 72.290966487999995, 2024-04-20T01:48:47.089 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.017903170999999999, 2024-04-20T01:48:47.089 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:47.089 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:47.089 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:47.090 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:47.090 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.090 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 69 2024-04-20T01:48:47.090 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.090 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:47.090 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.090 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:47.090 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783758+0000", 2024-04-20T01:48:47.090 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.090 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.090 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.090 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:47.090 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783758+0000", 2024-04-20T01:48:47.090 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.091 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.091 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.091 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:47.091 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783758+0000", 2024-04-20T01:48:47.091 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.091 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.091 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.091 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:47.091 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783759+0000", 2024-04-20T01:48:47.091 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.734e-06 2024-04-20T01:48:47.091 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.091 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.091 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:47.091 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783763+0000", 2024-04-20T01:48:47.092 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.5820000000000002e-06 2024-04-20T01:48:47.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.092 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.092 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.092 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.783766+0000", 2024-04-20T01:48:47.092 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.0469999999999998e-06 2024-04-20T01:48:47.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.092 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.092 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.092 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.785726+0000", 2024-04-20T01:48:47.092 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0019601079999999999 2024-04-20T01:48:47.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.092 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.092 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:47.092 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.785810+0000", 2024-04-20T01:48:47.093 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.3522000000000004e-05 2024-04-20T01:48:47.093 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.093 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.093 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.093 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797485+0000", 2024-04-20T01:48:47.093 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011675207 2024-04-20T01:48:47.093 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.093 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.093 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:47.093 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797573+0000", 2024-04-20T01:48:47.093 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8146000000000006e-05 2024-04-20T01:48:47.093 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.093 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.093 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:47.094 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.801661+0000", 2024-04-20T01:48:47.094 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0040880969999999997 2024-04-20T01:48:47.094 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.094 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.094 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.094 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.094 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.094 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:75 3.2s0 3:48443997:::smithi16579500-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:47.094 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.789051+0000", 2024-04-20T01:48:47.094 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 72.285672953000002, 2024-04-20T01:48:47.094 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013741625, 2024-04-20T01:48:47.094 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:47.094 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:47.094 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:47.095 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:47.095 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.095 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 75 2024-04-20T01:48:47.095 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.095 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:47.095 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.095 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:47.095 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789051+0000", 2024-04-20T01:48:47.095 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.095 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.095 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.095 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:47.095 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789051+0000", 2024-04-20T01:48:47.095 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.095 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.096 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.096 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:47.096 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789051+0000", 2024-04-20T01:48:47.096 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.096 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.096 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.096 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:47.096 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789054+0000", 2024-04-20T01:48:47.096 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7499999999999999e-06 2024-04-20T01:48:47.096 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.096 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.096 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:47.096 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789060+0000", 2024-04-20T01:48:47.096 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.567e-06 2024-04-20T01:48:47.096 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.097 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.097 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.097 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.789066+0000", 2024-04-20T01:48:47.097 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.1499999999999998e-06 2024-04-20T01:48:47.097 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.097 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.097 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.097 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.790905+0000", 2024-04-20T01:48:47.097 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0018397000000000001 2024-04-20T01:48:47.097 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.097 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.097 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:47.097 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.790994+0000", 2024-04-20T01:48:47.097 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8679999999999998e-05 2024-04-20T01:48:47.098 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.098 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.098 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.098 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797798+0000", 2024-04-20T01:48:47.098 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0068040899999999996 2024-04-20T01:48:47.098 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.098 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.098 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:47.098 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797879+0000", 2024-04-20T01:48:47.098 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.1318999999999998e-05 2024-04-20T01:48:47.098 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.098 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.098 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:47.098 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.802793+0000", 2024-04-20T01:48:47.099 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.004913789 2024-04-20T01:48:47.099 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.099 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.099 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.099 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.099 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.099 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:84 3.3s0 3:cb8bd5ef:::smithi16579500-28:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:47.099 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:34.797448+0000", 2024-04-20T01:48:47.099 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 72.277276490999995, 2024-04-20T01:48:47.099 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.01109836, 2024-04-20T01:48:47.099 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:47.099 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "reached pg", 2024-04-20T01:48:47.099 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:47.099 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:47.099 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.100 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 84 2024-04-20T01:48:47.100 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.100 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:47.100 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.100 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:47.100 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797448+0000", 2024-04-20T01:48:47.100 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.100 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.100 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.100 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:47.100 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797448+0000", 2024-04-20T01:48:47.100 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.100 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.100 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.101 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:47.101 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797448+0000", 2024-04-20T01:48:47.101 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.101 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.101 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.101 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:47.101 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797450+0000", 2024-04-20T01:48:47.101 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.7999999999999999e-06 2024-04-20T01:48:47.101 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.101 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.101 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:47.101 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797458+0000", 2024-04-20T01:48:47.101 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.2590000000000003e-06 2024-04-20T01:48:47.101 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.102 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.102 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.102 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.797464+0000", 2024-04-20T01:48:47.102 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.2979999999999997e-06 2024-04-20T01:48:47.102 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.102 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.102 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.102 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.801705+0000", 2024-04-20T01:48:47.102 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0042409819999999999 2024-04-20T01:48:47.102 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.102 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.102 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:47.102 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.801761+0000", 2024-04-20T01:48:47.102 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.5696999999999999e-05 2024-04-20T01:48:47.103 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.103 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.103 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.103 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.808533+0000", 2024-04-20T01:48:47.103 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0067724639999999997 2024-04-20T01:48:47.103 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.103 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.103 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:47.103 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:34.808546+0000", 2024-04-20T01:48:47.103 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3276e-05 2024-04-20T01:48:47.103 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.103 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.103 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.103 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.103 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.104 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:232 3.3s0 3:cb777d78:::smithi16579500-6:head [copy-from ver 6] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:47.104 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.787663+0000", 2024-04-20T01:48:47.104 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 68.287061886999993, 2024-04-20T01:48:47.104 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.019650832, 2024-04-20T01:48:47.104 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:47.104 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:47.104 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:47.104 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:47.104 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.104 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 232 2024-04-20T01:48:47.104 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.104 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:47.104 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.105 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:47.105 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787663+0000", 2024-04-20T01:48:47.105 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.105 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.105 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.105 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:47.105 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787663+0000", 2024-04-20T01:48:47.105 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.105 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.105 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.105 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:47.105 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787661+0000", 2024-04-20T01:48:47.105 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:47.105 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.105 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.106 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:47.106 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787673+0000", 2024-04-20T01:48:47.106 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.181e-05 2024-04-20T01:48:47.106 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.106 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.106 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:47.106 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787689+0000", 2024-04-20T01:48:47.106 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6731999999999998e-05 2024-04-20T01:48:47.106 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.106 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.106 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.106 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787706+0000", 2024-04-20T01:48:47.106 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6739e-05 2024-04-20T01:48:47.106 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.107 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.107 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.107 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787795+0000", 2024-04-20T01:48:47.107 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.9226000000000003e-05 2024-04-20T01:48:47.107 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.107 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.107 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:47.107 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787904+0000", 2024-04-20T01:48:47.107 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000108452 2024-04-20T01:48:47.107 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.107 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.107 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.107 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.797712+0000", 2024-04-20T01:48:47.107 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0098079599999999992 2024-04-20T01:48:47.108 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.108 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.108 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.108 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.799543+0000", 2024-04-20T01:48:47.108 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001831596 2024-04-20T01:48:47.108 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.108 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.108 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:47.108 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807224+0000", 2024-04-20T01:48:47.108 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0076805249999999997 2024-04-20T01:48:47.108 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.108 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.108 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:47.108 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.807313+0000", 2024-04-20T01:48:47.109 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.9431999999999993e-05 2024-04-20T01:48:47.109 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.109 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.109 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.109 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.109 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.109 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:233 3.3s0 3:cb777d78:::smithi16579500-6:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:47.109 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:38.787721+0000", 2024-04-20T01:48:47.109 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 68.287003275000004, 2024-04-20T01:48:47.109 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.01143623, 2024-04-20T01:48:47.109 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:47.109 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:47.109 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:47.109 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:47.109 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.110 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 233 2024-04-20T01:48:47.110 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.110 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:47.110 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.110 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:47.110 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787721+0000", 2024-04-20T01:48:47.110 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.110 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.110 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.110 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:47.110 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787721+0000", 2024-04-20T01:48:47.110 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.110 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.111 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.111 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:47.111 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787721+0000", 2024-04-20T01:48:47.111 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.111 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.111 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.111 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:47.111 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787743+0000", 2024-04-20T01:48:47.111 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.2540000000000001e-05 2024-04-20T01:48:47.111 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.111 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.111 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:47.111 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787753+0000", 2024-04-20T01:48:47.111 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0101e-05 2024-04-20T01:48:47.111 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.112 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.112 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.112 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.787760+0000", 2024-04-20T01:48:47.112 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.3729999999999998e-06 2024-04-20T01:48:47.112 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.112 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.112 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.112 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.788051+0000", 2024-04-20T01:48:47.112 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00029180999999999999 2024-04-20T01:48:47.112 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.112 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.112 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:47.112 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.788092+0000", 2024-04-20T01:48:47.112 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.0309000000000003e-05 2024-04-20T01:48:47.112 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.113 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.113 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.113 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.798980+0000", 2024-04-20T01:48:47.113 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010888070999999999 2024-04-20T01:48:47.113 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.113 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.113 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:47.113 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.799065+0000", 2024-04-20T01:48:47.113 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.5241999999999995e-05 2024-04-20T01:48:47.113 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.113 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.113 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:47.113 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:38.799157+0000", 2024-04-20T01:48:47.113 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.2388999999999999e-05 2024-04-20T01:48:47.113 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.114 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.114 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.114 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.114 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.114 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:296 3.2s0 3:44d9544b:::smithi16579500-38:head [read 0~1474560,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:47.114 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.807470+0000", 2024-04-20T01:48:47.114 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 63.267254301999998, 2024-04-20T01:48:47.114 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011565101, 2024-04-20T01:48:47.114 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:47.114 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:47.114 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:47.114 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:47.114 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.115 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 296 2024-04-20T01:48:47.115 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.115 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:47.115 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.115 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:47.115 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807470+0000", 2024-04-20T01:48:47.115 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.115 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.115 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.115 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:47.115 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807470+0000", 2024-04-20T01:48:47.115 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.115 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.115 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.116 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:47.116 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807470+0000", 2024-04-20T01:48:47.116 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.116 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.116 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.116 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:47.116 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807473+0000", 2024-04-20T01:48:47.116 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 2.9809999999999999e-06 2024-04-20T01:48:47.116 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.116 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.116 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:47.116 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807479+0000", 2024-04-20T01:48:47.116 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.1970000000000001e-06 2024-04-20T01:48:47.116 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.116 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.117 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.117 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807483+0000", 2024-04-20T01:48:47.117 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.053e-06 2024-04-20T01:48:47.117 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.117 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.117 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.117 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807633+0000", 2024-04-20T01:48:47.117 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00015004999999999999 2024-04-20T01:48:47.117 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.117 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.117 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:47.117 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.807643+0000", 2024-04-20T01:48:47.117 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.7049999999999994e-06 2024-04-20T01:48:47.117 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.118 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.118 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.118 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.811287+0000", 2024-04-20T01:48:47.118 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0036436699999999999 2024-04-20T01:48:47.118 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.118 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.118 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:47.118 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.811372+0000", 2024-04-20T01:48:47.118 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.5251000000000001e-05 2024-04-20T01:48:47.118 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.118 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.118 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:47.118 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819035+0000", 2024-04-20T01:48:47.118 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.007663464 2024-04-20T01:48:47.119 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.119 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.119 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.119 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.119 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.119 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:299 3.2s0 3:4302a124:::smithi16579500-33:head [rollback a] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:47.119 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.808485+0000", 2024-04-20T01:48:47.119 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 63.266239530999997, 2024-04-20T01:48:47.119 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011156045, 2024-04-20T01:48:47.119 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:47.119 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:47.119 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:47.119 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:47.119 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.119 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 299 2024-04-20T01:48:47.120 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.120 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:47.120 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.120 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:47.120 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808485+0000", 2024-04-20T01:48:47.120 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.120 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.120 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.120 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:47.120 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808485+0000", 2024-04-20T01:48:47.120 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.120 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.120 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.120 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:47.121 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808484+0000", 2024-04-20T01:48:47.121 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.121 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.121 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.121 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:47.121 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808488+0000", 2024-04-20T01:48:47.121 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.3529999999999999e-06 2024-04-20T01:48:47.121 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.121 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.121 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:47.121 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808495+0000", 2024-04-20T01:48:47.121 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.4390000000000003e-06 2024-04-20T01:48:47.121 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.121 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.121 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.122 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808501+0000", 2024-04-20T01:48:47.122 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.4480000000000002e-06 2024-04-20T01:48:47.122 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.122 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.122 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.122 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808634+0000", 2024-04-20T01:48:47.122 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000133461 2024-04-20T01:48:47.122 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.122 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.122 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:47.122 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808650+0000", 2024-04-20T01:48:47.122 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.6271999999999999e-05 2024-04-20T01:48:47.122 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.122 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.123 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.123 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.815215+0000", 2024-04-20T01:48:47.123 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0065644509999999998 2024-04-20T01:48:47.125 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.126 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.126 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:47.126 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.815328+0000", 2024-04-20T01:48:47.126 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000112754 2024-04-20T01:48:47.126 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.126 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.126 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.126 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.815818+0000", 2024-04-20T01:48:47.126 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00049083000000000004 2024-04-20T01:48:47.126 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.126 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.126 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.126 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819209+0000", 2024-04-20T01:48:47.126 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0033910139999999999 2024-04-20T01:48:47.127 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.127 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.127 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:47.127 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819596+0000", 2024-04-20T01:48:47.127 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00038603299999999998 2024-04-20T01:48:47.127 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.127 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.127 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:47.127 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819641+0000", 2024-04-20T01:48:47.127 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.5442999999999998e-05 2024-04-20T01:48:47.127 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.127 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.127 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.127 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.128 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.128 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:300 3.2s0 3:4302a124:::smithi16579500-33:head [append 0~0] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:47.128 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.808516+0000", 2024-04-20T01:48:47.128 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 63.266208394000003, 2024-04-20T01:48:47.128 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.016176293000000001, 2024-04-20T01:48:47.128 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:47.128 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:47.128 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:47.128 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:47.128 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.128 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 300 2024-04-20T01:48:47.128 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.128 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:47.128 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.128 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:47.129 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808516+0000", 2024-04-20T01:48:47.129 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.129 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.129 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.129 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:47.129 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808516+0000", 2024-04-20T01:48:47.129 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.129 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.129 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.129 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:47.129 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808515+0000", 2024-04-20T01:48:47.129 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.129 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.129 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.129 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:47.130 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808521+0000", 2024-04-20T01:48:47.130 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.8919999999999997e-06 2024-04-20T01:48:47.130 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.130 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.130 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:47.130 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808528+0000", 2024-04-20T01:48:47.130 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.0190000000000001e-06 2024-04-20T01:48:47.130 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.130 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.130 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.130 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.808534+0000", 2024-04-20T01:48:47.130 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.891e-06 2024-04-20T01:48:47.130 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.130 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.131 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.131 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.809751+0000", 2024-04-20T01:48:47.131 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0012163580000000001 2024-04-20T01:48:47.131 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.131 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.131 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:47.131 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.809758+0000", 2024-04-20T01:48:47.131 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.6580000000000002e-06 2024-04-20T01:48:47.131 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.131 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.131 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.131 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.819973+0000", 2024-04-20T01:48:47.131 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010214983 2024-04-20T01:48:47.131 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.132 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.132 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:47.132 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.820076+0000", 2024-04-20T01:48:47.132 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00010318 2024-04-20T01:48:47.132 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.132 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.132 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.132 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.820519+0000", 2024-04-20T01:48:47.132 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00044209799999999998 2024-04-20T01:48:47.132 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.132 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.132 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.132 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.821591+0000", 2024-04-20T01:48:47.132 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.001072677 2024-04-20T01:48:47.132 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.133 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.133 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:47.133 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.824619+0000", 2024-04-20T01:48:47.133 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0030279170000000002 2024-04-20T01:48:47.133 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.133 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.133 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:47.133 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.824692+0000", 2024-04-20T01:48:47.133 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.3220999999999996e-05 2024-04-20T01:48:47.133 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.133 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.133 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.133 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.133 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.134 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:302 3.2s0 3:4cb85a84:::smithi16579500-29:head [copy-from ver 10] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:47.134 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:43.823551+0000", 2024-04-20T01:48:47.134 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 63.251173338000001, 2024-04-20T01:48:47.134 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.014229169999999999, 2024-04-20T01:48:47.134 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:47.134 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:47.134 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:47.134 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:47.134 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.134 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 302 2024-04-20T01:48:47.134 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.134 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:47.134 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.134 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:47.135 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823551+0000", 2024-04-20T01:48:47.135 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.135 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.135 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.135 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:47.135 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823551+0000", 2024-04-20T01:48:47.135 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.135 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.135 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.135 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:47.135 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823550+0000", 2024-04-20T01:48:47.135 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:47.135 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.135 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.135 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:47.136 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823556+0000", 2024-04-20T01:48:47.136 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.455e-06 2024-04-20T01:48:47.136 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.136 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.136 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:47.136 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823568+0000", 2024-04-20T01:48:47.136 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.1921e-05 2024-04-20T01:48:47.136 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.136 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.136 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.136 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823575+0000", 2024-04-20T01:48:47.136 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.1169999999999998e-06 2024-04-20T01:48:47.136 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.136 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.136 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.137 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823617+0000", 2024-04-20T01:48:47.137 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2012999999999999e-05 2024-04-20T01:48:47.137 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.137 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.137 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:47.137 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.823741+0000", 2024-04-20T01:48:47.137 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000123454 2024-04-20T01:48:47.137 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.137 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.137 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.137 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.829454+0000", 2024-04-20T01:48:47.137 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0057129570000000003 2024-04-20T01:48:47.137 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.137 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.137 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.138 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.831037+0000", 2024-04-20T01:48:47.138 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0015832559999999999 2024-04-20T01:48:47.138 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.138 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.138 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:47.138 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.837719+0000", 2024-04-20T01:48:47.138 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.006681953 2024-04-20T01:48:47.138 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.138 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.138 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:47.138 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:43.837780+0000", 2024-04-20T01:48:47.138 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.1561000000000002e-05 2024-04-20T01:48:47.138 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.138 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.139 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.139 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.139 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.139 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:403 3.2s0 3:44d9544b:::smithi16579500-38:head [append 0~638976] snapc d=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-04-20T01:48:47.139 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.784120+0000", 2024-04-20T01:48:47.139 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 57.290604815999998, 2024-04-20T01:48:47.139 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.014614749999999999, 2024-04-20T01:48:47.139 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:47.139 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:47.139 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:47.139 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:47.139 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.139 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 403 2024-04-20T01:48:47.139 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.140 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:47.140 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.140 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:47.140 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.784120+0000", 2024-04-20T01:48:47.140 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.140 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.140 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.140 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:47.140 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.784120+0000", 2024-04-20T01:48:47.140 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.140 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.140 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.140 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:47.140 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.784118+0000", 2024-04-20T01:48:47.140 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:47.141 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.141 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.141 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:47.141 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788188+0000", 2024-04-20T01:48:47.141 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0040692860000000001 2024-04-20T01:48:47.141 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.141 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.141 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:47.141 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788201+0000", 2024-04-20T01:48:47.141 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.3142999999999999e-05 2024-04-20T01:48:47.141 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.141 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.141 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.141 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788207+0000", 2024-04-20T01:48:47.141 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.4300000000000003e-06 2024-04-20T01:48:47.141 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.142 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.142 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.142 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788261+0000", 2024-04-20T01:48:47.142 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3257000000000002e-05 2024-04-20T01:48:47.142 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.142 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.142 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:47.142 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788365+0000", 2024-04-20T01:48:47.142 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000103928 2024-04-20T01:48:47.142 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.142 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.142 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.142 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.789323+0000", 2024-04-20T01:48:47.142 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00095858900000000003 2024-04-20T01:48:47.143 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.143 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.143 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.143 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.793128+0000", 2024-04-20T01:48:47.143 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.003805307 2024-04-20T01:48:47.143 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.143 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.143 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:47.143 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.798692+0000", 2024-04-20T01:48:47.143 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0055635160000000001 2024-04-20T01:48:47.143 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.143 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.143 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:47.143 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.798734+0000", 2024-04-20T01:48:47.143 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.2453999999999997e-05 2024-04-20T01:48:47.143 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.144 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.144 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.144 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.144 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.144 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:404 3.2s0 3:44d9544b:::smithi16579500-38:head [append 0~622592] snapc d=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-04-20T01:48:47.144 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:47:49.788218+0000", 2024-04-20T01:48:47.144 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 57.286506516000003, 2024-04-20T01:48:47.144 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.012422041, 2024-04-20T01:48:47.144 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:47.144 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:47.144 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:47.144 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:47.144 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.144 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 404 2024-04-20T01:48:47.144 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.145 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:47.145 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.145 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:47.145 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788218+0000", 2024-04-20T01:48:47.145 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.145 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.145 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.145 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:47.145 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788218+0000", 2024-04-20T01:48:47.145 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.145 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.145 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.145 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:47.145 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.788217+0000", 2024-04-20T01:48:47.145 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.999999 2024-04-20T01:48:47.146 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.146 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.146 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:47.146 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.790736+0000", 2024-04-20T01:48:47.146 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0025187270000000001 2024-04-20T01:48:47.146 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.146 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.146 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:47.146 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.790752+0000", 2024-04-20T01:48:47.146 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.5684000000000001e-05 2024-04-20T01:48:47.146 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.146 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.146 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.146 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.790762+0000", 2024-04-20T01:48:47.147 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.0693000000000001e-05 2024-04-20T01:48:47.147 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.147 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.147 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.147 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.790813+0000", 2024-04-20T01:48:47.147 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.0235000000000003e-05 2024-04-20T01:48:47.147 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.147 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.147 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:47.147 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.790933+0000", 2024-04-20T01:48:47.147 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000120417 2024-04-20T01:48:47.147 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.147 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.147 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.148 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.791589+0000", 2024-04-20T01:48:47.148 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000655972 2024-04-20T01:48:47.148 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.148 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.148 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.148 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.793913+0000", 2024-04-20T01:48:47.148 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.002323716 2024-04-20T01:48:47.148 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.148 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.148 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:47.148 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.800601+0000", 2024-04-20T01:48:47.148 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0066881880000000003 2024-04-20T01:48:47.148 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.148 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.148 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:47.149 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:47:49.800640+0000", 2024-04-20T01:48:47.149 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.9150999999999998e-05 2024-04-20T01:48:47.149 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.149 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.149 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.149 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.149 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.149 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:523 3.2s0 3:44d9544b:::smithi16579500-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-04-20T01:48:47.149 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:01.840230+0000", 2024-04-20T01:48:47.149 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 45.234494310000002, 2024-04-20T01:48:47.149 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.010941562, 2024-04-20T01:48:47.149 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:47.149 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:47.149 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:47.149 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:47.149 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.150 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 523 2024-04-20T01:48:47.150 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.150 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:47.150 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.150 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:47.150 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.840230+0000", 2024-04-20T01:48:47.150 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.150 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.150 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.150 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:47.150 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.840230+0000", 2024-04-20T01:48:47.150 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.150 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.150 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.151 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:47.151 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.840230+0000", 2024-04-20T01:48:47.151 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.151 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.151 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.151 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:47.151 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.840237+0000", 2024-04-20T01:48:47.151 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.2309999999999999e-06 2024-04-20T01:48:47.151 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.151 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.151 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:47.151 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.840243+0000", 2024-04-20T01:48:47.151 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.0079999999999999e-06 2024-04-20T01:48:47.151 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.151 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.152 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.152 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.840251+0000", 2024-04-20T01:48:47.152 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.3019999999999995e-06 2024-04-20T01:48:47.152 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.152 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.152 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.152 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.843356+0000", 2024-04-20T01:48:47.152 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0031046419999999999 2024-04-20T01:48:47.152 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.152 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.152 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:47.152 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.843447+0000", 2024-04-20T01:48:47.152 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.1217000000000004e-05 2024-04-20T01:48:47.152 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.152 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.153 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.153 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.850113+0000", 2024-04-20T01:48:47.153 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0066663060000000003 2024-04-20T01:48:47.153 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.153 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.153 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:47.153 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.850210+0000", 2024-04-20T01:48:47.153 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.6875999999999999e-05 2024-04-20T01:48:47.153 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.153 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.153 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:47.153 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:01.851172+0000", 2024-04-20T01:48:47.153 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00096138500000000004 2024-04-20T01:48:47.153 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.153 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.153 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.153 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.154 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.154 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:574 3.2s0 3:44d9544b:::smithi16579500-38:head [append 0~655360] snapc 1f=[1f,1b,1a,b] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-04-20T01:48:47.154 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:04.793176+0000", 2024-04-20T01:48:47.154 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 42.281548227999998, 2024-04-20T01:48:47.154 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.012353861000000001, 2024-04-20T01:48:47.154 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:47.154 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:47.154 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:47.154 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:47.154 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.154 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 574 2024-04-20T01:48:47.154 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.154 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:47.155 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.155 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:47.155 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793176+0000", 2024-04-20T01:48:47.155 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.155 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.155 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.155 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:47.155 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793176+0000", 2024-04-20T01:48:47.155 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.155 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.155 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.155 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:47.155 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793174+0000", 2024-04-20T01:48:47.155 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:47.155 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.156 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.156 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:47.156 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793851+0000", 2024-04-20T01:48:47.156 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00067681499999999999 2024-04-20T01:48:47.156 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.156 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.156 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:47.156 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793869+0000", 2024-04-20T01:48:47.156 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.7569000000000001e-05 2024-04-20T01:48:47.156 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.156 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.156 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.156 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793878+0000", 2024-04-20T01:48:47.157 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.8659999999999995e-06 2024-04-20T01:48:47.157 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.157 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.157 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.157 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.793935+0000", 2024-04-20T01:48:47.157 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.6984999999999997e-05 2024-04-20T01:48:47.157 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.157 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.157 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:47.157 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.794017+0000", 2024-04-20T01:48:47.157 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.2719000000000005e-05 2024-04-20T01:48:47.157 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.157 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.157 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.158 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.794790+0000", 2024-04-20T01:48:47.158 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.000772495 2024-04-20T01:48:47.158 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.158 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.158 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.158 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.796798+0000", 2024-04-20T01:48:47.158 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0020083980000000002 2024-04-20T01:48:47.158 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.158 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.158 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:47.158 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.805483+0000", 2024-04-20T01:48:47.158 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0086847739999999993 2024-04-20T01:48:47.158 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.159 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.159 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:47.159 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:04.805530+0000", 2024-04-20T01:48:47.159 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.7042999999999997e-05 2024-04-20T01:48:47.159 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.159 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.159 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.159 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.159 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.159 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:790 3.3s0 3:cb777d78:::smithi16579500-6:head [copy-from ver 101] snapc 2a=[2a,28] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-04-20T01:48:47.159 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.754166+0000", 2024-04-20T01:48:47.159 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 30.320558641000002, 2024-04-20T01:48:47.159 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.020397073000000002, 2024-04-20T01:48:47.159 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:47.159 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:47.160 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:47.160 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:47.160 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.160 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 790 2024-04-20T01:48:47.160 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.160 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:47.160 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.160 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:47.160 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754166+0000", 2024-04-20T01:48:47.160 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.160 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.160 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.160 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:47.160 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754166+0000", 2024-04-20T01:48:47.160 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.161 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.161 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.161 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:47.161 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754165+0000", 2024-04-20T01:48:47.161 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:47.161 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.161 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.161 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:47.161 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754171+0000", 2024-04-20T01:48:47.161 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.3489999999999997e-06 2024-04-20T01:48:47.161 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.161 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.161 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:47.161 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754183+0000", 2024-04-20T01:48:47.161 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 1.2428e-05 2024-04-20T01:48:47.161 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.162 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.162 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.162 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754190+0000", 2024-04-20T01:48:47.162 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 6.3350000000000003e-06 2024-04-20T01:48:47.162 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.162 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.162 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.162 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754243+0000", 2024-04-20T01:48:47.162 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.3245000000000003e-05 2024-04-20T01:48:47.162 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.162 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.162 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:47.162 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754343+0000", 2024-04-20T01:48:47.162 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 9.9770000000000002e-05 2024-04-20T01:48:47.162 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.163 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.163 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.163 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.766334+0000", 2024-04-20T01:48:47.163 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.011991017 2024-04-20T01:48:47.163 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.163 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.163 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.163 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.772169+0000", 2024-04-20T01:48:47.163 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0058356240000000002 2024-04-20T01:48:47.163 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.163 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.163 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "commit_sent", 2024-04-20T01:48:47.163 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.774513+0000", 2024-04-20T01:48:47.163 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.0023441130000000001 2024-04-20T01:48:47.164 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.164 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.164 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:47.164 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.774563+0000", 2024-04-20T01:48:47.164 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4.9421e-05 2024-04-20T01:48:47.164 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.164 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.164 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.164 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.164 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.164 INFO:teuthology.orchestra.run.smithi121.stdout: "description": "osd_op(client.4643.0:791 3.3s0 3:cb777d78:::smithi16579500-6:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e75)", 2024-04-20T01:48:47.164 INFO:teuthology.orchestra.run.smithi121.stdout: "initiated_at": "2024-04-20T01:48:16.754206+0000", 2024-04-20T01:48:47.164 INFO:teuthology.orchestra.run.smithi121.stdout: "age": 30.320518190000001, 2024-04-20T01:48:47.164 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.014483546999999999, 2024-04-20T01:48:47.165 INFO:teuthology.orchestra.run.smithi121.stdout: "type_data": { 2024-04-20T01:48:47.165 INFO:teuthology.orchestra.run.smithi121.stdout: "flag_point": "started", 2024-04-20T01:48:47.165 INFO:teuthology.orchestra.run.smithi121.stdout: "client_info": { 2024-04-20T01:48:47.165 INFO:teuthology.orchestra.run.smithi121.stdout: "client": "client.4643", 2024-04-20T01:48:47.165 INFO:teuthology.orchestra.run.smithi121.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.165 INFO:teuthology.orchestra.run.smithi121.stdout: "tid": 791 2024-04-20T01:48:47.165 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.165 INFO:teuthology.orchestra.run.smithi121.stdout: "events": [ 2024-04-20T01:48:47.165 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.165 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "initiated", 2024-04-20T01:48:47.165 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754206+0000", 2024-04-20T01:48:47.165 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.165 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.165 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.165 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "throttled", 2024-04-20T01:48:47.165 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754206+0000", 2024-04-20T01:48:47.166 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0 2024-04-20T01:48:47.166 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.166 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.166 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "header_read", 2024-04-20T01:48:47.166 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754206+0000", 2024-04-20T01:48:47.166 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.166 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.166 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.166 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "all_read", 2024-04-20T01:48:47.166 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754209+0000", 2024-04-20T01:48:47.166 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.5190000000000001e-06 2024-04-20T01:48:47.166 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.166 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.166 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "dispatched", 2024-04-20T01:48:47.167 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754216+0000", 2024-04-20T01:48:47.167 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.1060000000000002e-06 2024-04-20T01:48:47.167 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.167 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.167 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.167 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754222+0000", 2024-04-20T01:48:47.167 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 5.9479999999999996e-06 2024-04-20T01:48:47.167 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.167 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.167 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.167 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754512+0000", 2024-04-20T01:48:47.167 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.00028918800000000001 2024-04-20T01:48:47.167 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.167 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.168 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:47.168 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.754551+0000", 2024-04-20T01:48:47.168 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 3.9524999999999999e-05 2024-04-20T01:48:47.168 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.168 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.168 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "reached_pg", 2024-04-20T01:48:47.168 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.768532+0000", 2024-04-20T01:48:47.168 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 0.013981247 2024-04-20T01:48:47.168 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.168 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.168 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "started", 2024-04-20T01:48:47.168 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.768614+0000", 2024-04-20T01:48:47.168 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 8.2078999999999998e-05 2024-04-20T01:48:47.168 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T01:48:47.169 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T01:48:47.169 INFO:teuthology.orchestra.run.smithi121.stdout: "event": "done", 2024-04-20T01:48:47.169 INFO:teuthology.orchestra.run.smithi121.stdout: "time": "2024-04-20T01:48:16.768690+0000", 2024-04-20T01:48:47.169 INFO:teuthology.orchestra.run.smithi121.stdout: "duration": 7.5413000000000005e-05 2024-04-20T01:48:47.169 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.169 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.169 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.169 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T01:48:47.169 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T01:48:47.169 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T01:48:47.171 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-20T01:48:47.320 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:47.320 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:47.320 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:48:47.320 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:47.334 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-20T01:48:47.465 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:47.465 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:47.465 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 30, 2024-04-20T01:48:47.465 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-20T01:48:47.465 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:47.475 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-20T01:48:47.604 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-20T01:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-20T01:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-20T01:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:230 3.fs0 3:fc9748e2:::smithi16579500-32:head [read 0~524288,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.787639+0000", 2024-04-20T01:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 68.830256229, 2024-04-20T01:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052033440000000004, 2024-04-20T01:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 230 2024-04-20T01:48:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787639+0000", 2024-04-20T01:48:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787639+0000", 2024-04-20T01:48:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787638+0000", 2024-04-20T01:48:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787642+0000", 2024-04-20T01:48:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4539999999999999e-06 2024-04-20T01:48:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787646+0000", 2024-04-20T01:48:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1269999999999996e-06 2024-04-20T01:48:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787650+0000", 2024-04-20T01:48:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.467e-06 2024-04-20T01:48:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787796+0000", 2024-04-20T01:48:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014651000000000001 2024-04-20T01:48:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787801+0000", 2024-04-20T01:48:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5059999999999997e-06 2024-04-20T01:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.790292+0000", 2024-04-20T01:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024917329999999999 2024-04-20T01:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.790354+0000", 2024-04-20T01:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1407000000000003e-05 2024-04-20T01:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.792842+0000", 2024-04-20T01:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024885139999999998 2024-04-20T01:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:231 3.fs0 3:fc9748e2:::smithi16579500-32:head [read 0~524288,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.787695+0000", 2024-04-20T01:48:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 68.830199996999994, 2024-04-20T01:48:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0078083780000000004, 2024-04-20T01:48:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 231 2024-04-20T01:48:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787695+0000", 2024-04-20T01:48:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787695+0000", 2024-04-20T01:48:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787694+0000", 2024-04-20T01:48:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787699+0000", 2024-04-20T01:48:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8439999999999997e-06 2024-04-20T01:48:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787708+0000", 2024-04-20T01:48:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2679999999999994e-06 2024-04-20T01:48:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787715+0000", 2024-04-20T01:48:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8290000000000008e-06 2024-04-20T01:48:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787839+0000", 2024-04-20T01:48:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012379700000000001 2024-04-20T01:48:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787845+0000", 2024-04-20T01:48:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1249999999999998e-06 2024-04-20T01:48:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.793431+0000", 2024-04-20T01:48:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055859880000000001 2024-04-20T01:48:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.793524+0000", 2024-04-20T01:48:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2943999999999996e-05 2024-04-20T01:48:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.795503+0000", 2024-04-20T01:48:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019791520000000001 2024-04-20T01:48:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.ds1 38/30 ECSubWrite(tid=19, reqid=osd.6.0:18, at_version=0'0, trim_to=0'0, roll_forward_to=38'10))", 2024-04-20T01:48:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:39.794291+0000", 2024-04-20T01:48:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 67.823603868000006, 2024-04-20T01:48:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068904140000000001, 2024-04-20T01:48:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794291+0000", 2024-04-20T01:48:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794291+0000", 2024-04-20T01:48:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794290+0000", 2024-04-20T01:48:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794317+0000", 2024-04-20T01:48:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7328999999999999e-05 2024-04-20T01:48:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794331+0000", 2024-04-20T01:48:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4076999999999999e-05 2024-04-20T01:48:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794339+0000", 2024-04-20T01:48:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6629999999999993e-06 2024-04-20T01:48:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794383+0000", 2024-04-20T01:48:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4106000000000001e-05 2024-04-20T01:48:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.794390+0000", 2024-04-20T01:48:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5710000000000003e-06 2024-04-20T01:48:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.801146+0000", 2024-04-20T01:48:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0067565300000000002 2024-04-20T01:48:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:39.801182+0000", 2024-04-20T01:48:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5472999999999997e-05 2024-04-20T01:48:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 42/30 ECSubWrite(tid=17, reqid=client.4643.0:307, at_version=42'10, trim_to=0'0, roll_forward_to=33'8))", 2024-04-20T01:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:43.830042+0000", 2024-04-20T01:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 63.787852817000001, 2024-04-20T01:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0099127780000000006, 2024-04-20T01:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830042+0000", 2024-04-20T01:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830042+0000", 2024-04-20T01:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830042+0000", 2024-04-20T01:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830306+0000", 2024-04-20T01:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026456199999999998 2024-04-20T01:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830326+0000", 2024-04-20T01:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9919e-05 2024-04-20T01:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830337+0000", 2024-04-20T01:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0693000000000001e-05 2024-04-20T01:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830414+0000", 2024-04-20T01:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6916999999999996e-05 2024-04-20T01:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.830426+0000", 2024-04-20T01:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2231e-05 2024-04-20T01:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.839923+0000", 2024-04-20T01:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0094965540000000008 2024-04-20T01:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:43.839955+0000", 2024-04-20T01:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2299999999999999e-05 2024-04-20T01:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.as6 48/30 ECSubWrite(tid=33, reqid=osd.6.0:33, at_version=48'14, trim_to=0'0, roll_forward_to=47'12))", 2024-04-20T01:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.716709+0000", 2024-04-20T01:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 57.901186369999998, 2024-04-20T01:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060784910000000001, 2024-04-20T01:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716709+0000", 2024-04-20T01:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716709+0000", 2024-04-20T01:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716709+0000", 2024-04-20T01:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716729+0000", 2024-04-20T01:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0735999999999999e-05 2024-04-20T01:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716740+0000", 2024-04-20T01:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0852e-05 2024-04-20T01:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716749+0000", 2024-04-20T01:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1700000000000003e-06 2024-04-20T01:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716830+0000", 2024-04-20T01:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0618999999999995e-05 2024-04-20T01:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.716841+0000", 2024-04-20T01:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1421e-05 2024-04-20T01:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.722760+0000", 2024-04-20T01:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059184210000000001 2024-04-20T01:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.722787+0000", 2024-04-20T01:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7432000000000001e-05 2024-04-20T01:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.8s5 48/30 ECSubWrite(tid=45, reqid=osd.5.0:44, at_version=0'0, trim_to=0'0, roll_forward_to=48'33))", 2024-04-20T01:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.726177+0000", 2024-04-20T01:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 57.891718191000002, 2024-04-20T01:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062928719999999997, 2024-04-20T01:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726177+0000", 2024-04-20T01:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726177+0000", 2024-04-20T01:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726176+0000", 2024-04-20T01:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726187+0000", 2024-04-20T01:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1216000000000001e-05 2024-04-20T01:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726202+0000", 2024-04-20T01:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4324999999999999e-05 2024-04-20T01:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726216+0000", 2024-04-20T01:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4294000000000001e-05 2024-04-20T01:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726289+0000", 2024-04-20T01:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3349000000000005e-05 2024-04-20T01:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.726300+0000", 2024-04-20T01:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0190000000000001e-05 2024-04-20T01:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.732436+0000", 2024-04-20T01:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006136028 2024-04-20T01:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.732470+0000", 2024-04-20T01:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4167e-05 2024-04-20T01:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:697 3.fs0 3:f3742432:::smithi16579500-42:head [append 0~573440] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.031311+0000", 2024-04-20T01:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 37.586584104000003, 2024-04-20T01:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007174181, 2024-04-20T01:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 697 2024-04-20T01:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.031311+0000", 2024-04-20T01:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.031311+0000", 2024-04-20T01:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.031294+0000", 2024-04-20T01:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999828 2024-04-20T01:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.032139+0000", 2024-04-20T01:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00084458099999999996 2024-04-20T01:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.032156+0000", 2024-04-20T01:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7090000000000001e-05 2024-04-20T01:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.032169+0000", 2024-04-20T01:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3336000000000001e-05 2024-04-20T01:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.032222+0000", 2024-04-20T01:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3338000000000003e-05 2024-04-20T01:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.032307+0000", 2024-04-20T01:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4655999999999997e-05 2024-04-20T01:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.034259+0000", 2024-04-20T01:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019517340000000001 2024-04-20T01:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.035704+0000", 2024-04-20T01:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014451609999999999 2024-04-20T01:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.038432+0000", 2024-04-20T01:48:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002728285 2024-04-20T01:48:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.038485+0000", 2024-04-20T01:48:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3019999999999997e-05 2024-04-20T01:48:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:698 3.fs0 3:f3742432:::smithi16579500-42:head [append 0~622592] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.035540+0000", 2024-04-20T01:48:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 37.582355233999998, 2024-04-20T01:48:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060756079999999997, 2024-04-20T01:48:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 698 2024-04-20T01:48:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.035540+0000", 2024-04-20T01:48:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.035540+0000", 2024-04-20T01:48:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.035538+0000", 2024-04-20T01:48:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.036167+0000", 2024-04-20T01:48:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000628924 2024-04-20T01:48:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.036179+0000", 2024-04-20T01:48:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1865000000000001e-05 2024-04-20T01:48:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.036185+0000", 2024-04-20T01:48:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2570000000000004e-06 2024-04-20T01:48:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.036220+0000", 2024-04-20T01:48:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4646e-05 2024-04-20T01:48:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.036301+0000", 2024-04-20T01:48:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1258e-05 2024-04-20T01:48:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.036852+0000", 2024-04-20T01:48:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00055088000000000001 2024-04-20T01:48:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.037828+0000", 2024-04-20T01:48:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00097533299999999999 2024-04-20T01:48:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041571+0000", 2024-04-20T01:48:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037428990000000001 2024-04-20T01:48:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041615+0000", 2024-04-20T01:48:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4934e-05 2024-04-20T01:48:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:700 3.fs0 3:f3742432:::smithi16579500-42:head [append 0~311296] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:47.642 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.041315+0000", 2024-04-20T01:48:47.642 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 37.576579979000002, 2024-04-20T01:48:47.642 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007465185, 2024-04-20T01:48:47.642 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:47.642 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:47.642 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:47.642 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:47.642 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.642 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 700 2024-04-20T01:48:47.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.642 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:47.642 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.642 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:47.642 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041315+0000", 2024-04-20T01:48:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041315+0000", 2024-04-20T01:48:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041314+0000", 2024-04-20T01:48:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041748+0000", 2024-04-20T01:48:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00043357900000000002 2024-04-20T01:48:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041762+0000", 2024-04-20T01:48:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3529999999999999e-05 2024-04-20T01:48:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041770+0000", 2024-04-20T01:48:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.242e-06 2024-04-20T01:48:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041797+0000", 2024-04-20T01:48:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7515999999999999e-05 2024-04-20T01:48:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041859+0000", 2024-04-20T01:48:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1190000000000002e-05 2024-04-20T01:48:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.042286+0000", 2024-04-20T01:48:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000427401 2024-04-20T01:48:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.043538+0000", 2024-04-20T01:48:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012518799999999999 2024-04-20T01:48:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.646 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:47.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.048737+0000", 2024-04-20T01:48:47.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051987370000000001 2024-04-20T01:48:47.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.646 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:47.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.048780+0000", 2024-04-20T01:48:47.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3735000000000001e-05 2024-04-20T01:48:47.646 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.646 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.646 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.647 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:701 3.fs0 3:f3742432:::smithi16579500-42:head [setxattr _header (57) in=7b] snapc 20=[1a] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:47.647 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.041779+0000", 2024-04-20T01:48:47.647 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 37.576116444, 2024-04-20T01:48:47.647 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007335467, 2024-04-20T01:48:47.647 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:47.647 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:47.647 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:47.647 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:47.647 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.647 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 701 2024-04-20T01:48:47.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.647 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:47.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.647 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:47.647 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041779+0000", 2024-04-20T01:48:47.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:47.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041779+0000", 2024-04-20T01:48:47.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:47.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041778+0000", 2024-04-20T01:48:47.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:47.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:47.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041784+0000", 2024-04-20T01:48:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8649999999999998e-06 2024-04-20T01:48:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041791+0000", 2024-04-20T01:48:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8619999999999997e-06 2024-04-20T01:48:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041796+0000", 2024-04-20T01:48:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.429e-06 2024-04-20T01:48:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.042635+0000", 2024-04-20T01:48:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00083878299999999995 2024-04-20T01:48:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.042700+0000", 2024-04-20T01:48:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5116999999999993e-05 2024-04-20T01:48:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.043007+0000", 2024-04-20T01:48:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00030746499999999998 2024-04-20T01:48:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.043875+0000", 2024-04-20T01:48:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00086782099999999996 2024-04-20T01:48:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.049067+0000", 2024-04-20T01:48:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051916810000000001 2024-04-20T01:48:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.049114+0000", 2024-04-20T01:48:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7163999999999998e-05 2024-04-20T01:48:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:702 3.fs0 3:f3742432:::smithi16579500-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.041802+0000", 2024-04-20T01:48:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 37.576092744999997, 2024-04-20T01:48:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0093865919999999992, 2024-04-20T01:48:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 702 2024-04-20T01:48:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041802+0000", 2024-04-20T01:48:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041802+0000", 2024-04-20T01:48:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041802+0000", 2024-04-20T01:48:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041805+0000", 2024-04-20T01:48:47.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.689e-06 2024-04-20T01:48:47.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:47.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041812+0000", 2024-04-20T01:48:47.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3119999999999996e-06 2024-04-20T01:48:47.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.041817+0000", 2024-04-20T01:48:47.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2889999999999997e-06 2024-04-20T01:48:47.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.043248+0000", 2024-04-20T01:48:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001430361 2024-04-20T01:48:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.043309+0000", 2024-04-20T01:48:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1328999999999995e-05 2024-04-20T01:48:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.049322+0000", 2024-04-20T01:48:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060133299999999999 2024-04-20T01:48:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.049422+0000", 2024-04-20T01:48:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9431000000000004e-05 2024-04-20T01:48:47.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:47.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.051189+0000", 2024-04-20T01:48:47.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001767315 2024-04-20T01:48:47.656 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.656 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.656 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.656 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s1 75/30 ECSubWrite(tid=363, reqid=client.4643.0:790, at_version=75'30, trim_to=0'0, roll_forward_to=75'28))", 2024-04-20T01:48:47.656 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:16.766177+0000", 2024-04-20T01:48:47.656 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 30.851717847, 2024-04-20T01:48:47.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0078257810000000004, 2024-04-20T01:48:47.656 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:47.657 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:47.657 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:47.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:47.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.766177+0000", 2024-04-20T01:48:47.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:47.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.766177+0000", 2024-04-20T01:48:47.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.766176+0000", 2024-04-20T01:48:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.767764+0000", 2024-04-20T01:48:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015875609999999999 2024-04-20T01:48:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.767781+0000", 2024-04-20T01:48:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6869999999999999e-05 2024-04-20T01:48:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.767792+0000", 2024-04-20T01:48:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0991e-05 2024-04-20T01:48:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.767844+0000", 2024-04-20T01:48:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2840999999999998e-05 2024-04-20T01:48:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.767856+0000", 2024-04-20T01:48:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1908e-05 2024-04-20T01:48:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.773978+0000", 2024-04-20T01:48:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061211390000000003 2024-04-20T01:48:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.774003+0000", 2024-04-20T01:48:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5449999999999999e-05 2024-04-20T01:48:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:995 3.fs0 3:f3742432:::smithi16579500-42:31 [read 0~5160960,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:31.792839+0000", 2024-04-20T01:48:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.825056194, 2024-04-20T01:48:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011729463000000001, 2024-04-20T01:48:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 995 2024-04-20T01:48:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.792839+0000", 2024-04-20T01:48:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.792839+0000", 2024-04-20T01:48:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.792838+0000", 2024-04-20T01:48:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.792844+0000", 2024-04-20T01:48:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3069999999999999e-06 2024-04-20T01:48:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.792855+0000", 2024-04-20T01:48:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0813e-05 2024-04-20T01:48:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.792865+0000", 2024-04-20T01:48:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0278999999999999e-05 2024-04-20T01:48:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.792907+0000", 2024-04-20T01:48:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1433999999999997e-05 2024-04-20T01:48:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.792980+0000", 2024-04-20T01:48:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3761000000000001e-05 2024-04-20T01:48:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.804568+0000", 2024-04-20T01:48:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011588002999999999 2024-04-20T01:48:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:996 3.fs0 3:f3742432:::smithi16579500-42:31 [read 0~5160960,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:31.792874+0000", 2024-04-20T01:48:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.825020908000001, 2024-04-20T01:48:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013892335, 2024-04-20T01:48:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 996 2024-04-20T01:48:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.792874+0000", 2024-04-20T01:48:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.792874+0000", 2024-04-20T01:48:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.792874+0000", 2024-04-20T01:48:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.792877+0000", 2024-04-20T01:48:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5729999999999999e-06 2024-04-20T01:48:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.792884+0000", 2024-04-20T01:48:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.319e-06 2024-04-20T01:48:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.792889+0000", 2024-04-20T01:48:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6300000000000003e-06 2024-04-20T01:48:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.793197+0000", 2024-04-20T01:48:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000307638 2024-04-20T01:48:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.793253+0000", 2024-04-20T01:48:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6557000000000001e-05 2024-04-20T01:48:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.806766+0000", 2024-04-20T01:48:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013513094999999999 2024-04-20T01:48:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.669 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.669 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.669 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:997 3.fs0 3:f3742432:::smithi16579500-42:31 [read 0~5160960,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:47.669 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:31.792894+0000", 2024-04-20T01:48:47.669 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.82500093, 2024-04-20T01:48:47.669 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015567675, 2024-04-20T01:48:47.669 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:47.669 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:47.669 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:47.669 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:47.669 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.669 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 997 2024-04-20T01:48:47.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.792894+0000", 2024-04-20T01:48:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.792894+0000", 2024-04-20T01:48:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.792894+0000", 2024-04-20T01:48:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.792897+0000", 2024-04-20T01:48:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9730000000000002e-06 2024-04-20T01:48:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.792903+0000", 2024-04-20T01:48:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0279999999999997e-06 2024-04-20T01:48:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.792907+0000", 2024-04-20T01:48:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7620000000000003e-06 2024-04-20T01:48:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.793434+0000", 2024-04-20T01:48:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00052670199999999999 2024-04-20T01:48:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.793486+0000", 2024-04-20T01:48:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1669999999999998e-05 2024-04-20T01:48:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.808462+0000", 2024-04-20T01:48:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014975952000000001 2024-04-20T01:48:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpReadReply(3.fs0 89/30 ECSubReadReply(tid=85, attrs_read=0))", 2024-04-20T01:48:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:31.799614+0000", 2024-04-20T01:48:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.818280897999999, 2024-04-20T01:48:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050573110000000001, 2024-04-20T01:48:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.799614+0000", 2024-04-20T01:48:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.799614+0000", 2024-04-20T01:48:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.799614+0000", 2024-04-20T01:48:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.802051+0000", 2024-04-20T01:48:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024372759999999999 2024-04-20T01:48:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.802062+0000", 2024-04-20T01:48:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0497e-05 2024-04-20T01:48:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.802067+0000", 2024-04-20T01:48:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8919999999999997e-06 2024-04-20T01:48:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.802094+0000", 2024-04-20T01:48:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6922000000000001e-05 2024-04-20T01:48:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:31.804671+0000", 2024-04-20T01:48:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025770799999999998 2024-04-20T01:48:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:1013 3.fs0 3:f3742432:::smithi16579500-42:head [append 0~294912] snapc 38=[35,31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-04-20T01:48:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:33.286153+0000", 2024-04-20T01:48:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.331742164, 2024-04-20T01:48:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077531270000000003, 2024-04-20T01:48:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 1013 2024-04-20T01:48:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.286153+0000", 2024-04-20T01:48:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.286153+0000", 2024-04-20T01:48:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.286143+0000", 2024-04-20T01:48:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.99999 2024-04-20T01:48:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.286541+0000", 2024-04-20T01:48:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00039778900000000001 2024-04-20T01:48:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.286554+0000", 2024-04-20T01:48:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2777e-05 2024-04-20T01:48:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.286578+0000", 2024-04-20T01:48:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4467000000000001e-05 2024-04-20T01:48:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.286635+0000", 2024-04-20T01:48:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7299000000000001e-05 2024-04-20T01:48:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.286752+0000", 2024-04-20T01:48:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000116665 2024-04-20T01:48:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.287628+0000", 2024-04-20T01:48:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00087609700000000005 2024-04-20T01:48:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.289821+0000", 2024-04-20T01:48:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002192594 2024-04-20T01:48:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.293839+0000", 2024-04-20T01:48:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040186939999999997 2024-04-20T01:48:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.293906+0000", 2024-04-20T01:48:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6618000000000005e-05 2024-04-20T01:48:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:1014 3.fs0 3:f3742432:::smithi16579500-42:head [setxattr _header (57) in=7b] snapc 38=[35,31] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-04-20T01:48:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:33.286587+0000", 2024-04-20T01:48:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.331308290000001, 2024-04-20T01:48:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0074478160000000003, 2024-04-20T01:48:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 1014 2024-04-20T01:48:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.286587+0000", 2024-04-20T01:48:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.286587+0000", 2024-04-20T01:48:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.286586+0000", 2024-04-20T01:48:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.286589+0000", 2024-04-20T01:48:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6630000000000002e-06 2024-04-20T01:48:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.286593+0000", 2024-04-20T01:48:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.783e-06 2024-04-20T01:48:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.286596+0000", 2024-04-20T01:48:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8930000000000001e-06 2024-04-20T01:48:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.289055+0000", 2024-04-20T01:48:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024595070000000001 2024-04-20T01:48:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.289130+0000", 2024-04-20T01:48:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4548000000000005e-05 2024-04-20T01:48:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.289460+0000", 2024-04-20T01:48:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00032978499999999997 2024-04-20T01:48:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.292577+0000", 2024-04-20T01:48:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031173260000000001 2024-04-20T01:48:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.293977+0000", 2024-04-20T01:48:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014004950000000001 2024-04-20T01:48:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.294035+0000", 2024-04-20T01:48:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7244999999999998e-05 2024-04-20T01:48:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.687 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.687 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:1015 3.fs0 3:f3742432:::smithi16579500-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e91)", 2024-04-20T01:48:47.687 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:33.286619+0000", 2024-04-20T01:48:47.687 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.331276225, 2024-04-20T01:48:47.687 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083003069999999998, 2024-04-20T01:48:47.687 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:47.687 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:47.687 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:47.687 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:47.687 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:47.687 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 1015 2024-04-20T01:48:47.687 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.687 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:47.687 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.687 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:47.688 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.286619+0000", 2024-04-20T01:48:47.688 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.688 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.688 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:47.688 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.286619+0000", 2024-04-20T01:48:47.688 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.688 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.688 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:47.688 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.286619+0000", 2024-04-20T01:48:47.688 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.688 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.688 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:47.689 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.286622+0000", 2024-04-20T01:48:47.689 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.523e-06 2024-04-20T01:48:47.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.689 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.689 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:47.689 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.286626+0000", 2024-04-20T01:48:47.689 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2769999999999999e-06 2024-04-20T01:48:47.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.689 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.689 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.689 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.286631+0000", 2024-04-20T01:48:47.689 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0900000000000004e-06 2024-04-20T01:48:47.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.689 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.690 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.690 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.289736+0000", 2024-04-20T01:48:47.690 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031045410000000002 2024-04-20T01:48:47.690 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.690 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.690 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:47.690 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.289810+0000", 2024-04-20T01:48:47.690 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4010000000000005e-05 2024-04-20T01:48:47.690 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.690 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.690 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.690 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.294258+0000", 2024-04-20T01:48:47.690 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044483789999999997 2024-04-20T01:48:47.690 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.690 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.691 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:47.691 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.294319+0000", 2024-04-20T01:48:47.691 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0776999999999998e-05 2024-04-20T01:48:47.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.691 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.691 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:47.691 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.294919+0000", 2024-04-20T01:48:47.691 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00059997900000000005 2024-04-20T01:48:47.691 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.691 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.691 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.691 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.691 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s1 91/90 ECSubWrite(tid=406, reqid=client.4643.0:1019, at_version=91'51, trim_to=0'0, roll_forward_to=0'0))", 2024-04-20T01:48:47.691 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:33.288897+0000", 2024-04-20T01:48:47.692 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.328998507, 2024-04-20T01:48:47.692 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062765570000000003, 2024-04-20T01:48:47.692 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:47.692 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:47.692 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:47.692 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.692 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:47.692 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.288897+0000", 2024-04-20T01:48:47.692 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.692 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.692 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.692 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:47.692 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.288897+0000", 2024-04-20T01:48:47.692 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:47.692 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.693 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.693 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:47.693 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.288896+0000", 2024-04-20T01:48:47.693 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:47.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.693 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.693 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:47.693 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.288963+0000", 2024-04-20T01:48:47.693 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7497999999999996e-05 2024-04-20T01:48:47.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.693 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.693 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:47.693 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.288975+0000", 2024-04-20T01:48:47.693 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1992e-05 2024-04-20T01:48:47.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.693 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:47.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.288982+0000", 2024-04-20T01:48:47.694 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8549999999999996e-06 2024-04-20T01:48:47.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:47.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.289036+0000", 2024-04-20T01:48:47.694 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3662000000000002e-05 2024-04-20T01:48:47.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:47.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.289046+0000", 2024-04-20T01:48:47.694 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0647e-05 2024-04-20T01:48:47.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.695 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.695 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:47.695 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.295155+0000", 2024-04-20T01:48:47.695 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061080240000000001 2024-04-20T01:48:47.695 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:47.695 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:47.695 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:47.695 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:33.295173+0000", 2024-04-20T01:48:47.695 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8589000000000001e-05 2024-04-20T01:48:47.695 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.695 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.695 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.695 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:47.695 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:47.695 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:47.697 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-20T01:48:47.795 INFO:tasks.thrashosds.thrasher:reviving osd 2024-04-20T01:48:47.795 INFO:tasks.thrashosds.thrasher:Reviving osd 14 2024-04-20T01:48:47.795 INFO:tasks.ceph.osd.14:Restarting daemon 2024-04-20T01:48:47.796 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 14 2024-04-20T01:48:47.798 INFO:tasks.ceph.osd.14:Started 2024-04-20T01:48:47.798 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-20T01:48:47.833 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:47.833 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:47.833 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:48:47.833 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:47.834 INFO:teuthology.orchestra.run.smithi044.stderr:noscrub is unset 2024-04-20T01:48:47.845 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-20T01:48:47.849 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-20T01:48:47.925 INFO:teuthology.orchestra.run.smithi123.stderr:no valid command found; 10 closest matches: 2024-04-20T01:48:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:0 2024-04-20T01:48:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:1 2024-04-20T01:48:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:2 2024-04-20T01:48:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:abort 2024-04-20T01:48:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:assert 2024-04-20T01:48:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:config diff 2024-04-20T01:48:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:config diff get 2024-04-20T01:48:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:config get 2024-04-20T01:48:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:config help [] 2024-04-20T01:48:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:config set ... 2024-04-20T01:48:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:admin_socket: invalid command 2024-04-20T01:48:47.928 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-20T01:48:47.929 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-14, ['dump_ops_in_flight'] 2024-04-20T01:48:48.015 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:48.015 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:48.015 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 30, 2024-04-20T01:48:48.015 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-20T01:48:48.015 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:48.026 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-20T01:48:48.155 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:48.155 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-20T01:48:48.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-20T01:48:48.155 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-20T01:48:48.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:243 3.ds0 3:b226b74f:::smithi16579500-7:head [delete] snapc 5=[5,2] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:38.787656+0000", 2024-04-20T01:48:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 69.381009570000003, 2024-04-20T01:48:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013205184, 2024-04-20T01:48:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 243 2024-04-20T01:48:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787656+0000", 2024-04-20T01:48:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787656+0000", 2024-04-20T01:48:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787654+0000", 2024-04-20T01:48:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787662+0000", 2024-04-20T01:48:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6599999999999995e-06 2024-04-20T01:48:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787680+0000", 2024-04-20T01:48:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7672999999999998e-05 2024-04-20T01:48:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787688+0000", 2024-04-20T01:48:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.405e-06 2024-04-20T01:48:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787739+0000", 2024-04-20T01:48:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0652999999999998e-05 2024-04-20T01:48:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.787851+0000", 2024-04-20T01:48:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112172 2024-04-20T01:48:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.788643+0000", 2024-04-20T01:48:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00079174300000000003 2024-04-20T01:48:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.789564+0000", 2024-04-20T01:48:48.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00092135199999999996 2024-04-20T01:48:48.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.800781+0000", 2024-04-20T01:48:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011217187999999999 2024-04-20T01:48:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:38.800861+0000", 2024-04-20T01:48:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9796999999999995e-05 2024-04-20T01:48:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.1s4 48/30 ECSubWrite(tid=119, reqid=osd.4.0:119, at_version=48'14, trim_to=0'0, roll_forward_to=47'12))", 2024-04-20T01:48:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.717339+0000", 2024-04-20T01:48:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 58.451326883, 2024-04-20T01:48:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0079470240000000004, 2024-04-20T01:48:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717339+0000", 2024-04-20T01:48:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717339+0000", 2024-04-20T01:48:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717338+0000", 2024-04-20T01:48:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717357+0000", 2024-04-20T01:48:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8575000000000001e-05 2024-04-20T01:48:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717368+0000", 2024-04-20T01:48:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.145e-05 2024-04-20T01:48:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717377+0000", 2024-04-20T01:48:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4929999999999998e-06 2024-04-20T01:48:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717424+0000", 2024-04-20T01:48:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7240000000000002e-05 2024-04-20T01:48:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717435+0000", 2024-04-20T01:48:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0533000000000001e-05 2024-04-20T01:48:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.725244+0000", 2024-04-20T01:48:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0078092129999999997 2024-04-20T01:48:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.725286+0000", 2024-04-20T01:48:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1792999999999998e-05 2024-04-20T01:48:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 60/30 ECSubWrite(tid=280, reqid=client.4643.0:519, at_version=60'66, trim_to=0'0, roll_forward_to=60'63))", 2024-04-20T01:48:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:01.837460+0000", 2024-04-20T01:48:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.331205678000003, 2024-04-20T01:48:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071325599999999996, 2024-04-20T01:48:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837460+0000", 2024-04-20T01:48:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837460+0000", 2024-04-20T01:48:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837459+0000", 2024-04-20T01:48:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837876+0000", 2024-04-20T01:48:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00041722999999999999 2024-04-20T01:48:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837893+0000", 2024-04-20T01:48:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6803000000000002e-05 2024-04-20T01:48:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837903+0000", 2024-04-20T01:48:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6830000000000003e-06 2024-04-20T01:48:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837955+0000", 2024-04-20T01:48:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2160000000000002e-05 2024-04-20T01:48:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.837967+0000", 2024-04-20T01:48:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2057e-05 2024-04-20T01:48:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.844571+0000", 2024-04-20T01:48:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066042319999999998 2024-04-20T01:48:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:01.844592+0000", 2024-04-20T01:48:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1268999999999999e-05 2024-04-20T01:48:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 63/30 ECSubWrite(tid=295, reqid=client.4643.0:574, at_version=63'79, trim_to=0'0, roll_forward_to=62'77))", 2024-04-20T01:48:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:04.795046+0000", 2024-04-20T01:48:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 43.373619554999998, 2024-04-20T01:48:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076412559999999999, 2024-04-20T01:48:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795046+0000", 2024-04-20T01:48:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795046+0000", 2024-04-20T01:48:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795044+0000", 2024-04-20T01:48:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795601+0000", 2024-04-20T01:48:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00055720399999999995 2024-04-20T01:48:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795617+0000", 2024-04-20T01:48:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5341000000000001e-05 2024-04-20T01:48:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795627+0000", 2024-04-20T01:48:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0429e-05 2024-04-20T01:48:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795692+0000", 2024-04-20T01:48:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4712e-05 2024-04-20T01:48:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795704+0000", 2024-04-20T01:48:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.258e-05 2024-04-20T01:48:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.802650+0000", 2024-04-20T01:48:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069452869999999996 2024-04-20T01:48:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.802687+0000", 2024-04-20T01:48:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7483e-05 2024-04-20T01:48:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 63/30 ECSubWrite(tid=296, reqid=client.4643.0:575, at_version=63'80, trim_to=0'0, roll_forward_to=62'77))", 2024-04-20T01:48:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:04.798415+0000", 2024-04-20T01:48:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 43.370250755999997, 2024-04-20T01:48:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071008770000000002, 2024-04-20T01:48:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798415+0000", 2024-04-20T01:48:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798415+0000", 2024-04-20T01:48:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798414+0000", 2024-04-20T01:48:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798883+0000", 2024-04-20T01:48:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00046914899999999998 2024-04-20T01:48:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798898+0000", 2024-04-20T01:48:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4534e-05 2024-04-20T01:48:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798905+0000", 2024-04-20T01:48:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9149999999999996e-06 2024-04-20T01:48:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798951+0000", 2024-04-20T01:48:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5776999999999999e-05 2024-04-20T01:48:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798961+0000", 2024-04-20T01:48:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9289999999999993e-06 2024-04-20T01:48:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.805490+0000", 2024-04-20T01:48:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065291029999999996 2024-04-20T01:48:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.805516+0000", 2024-04-20T01:48:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5347999999999999e-05 2024-04-20T01:48:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.cs2 68/30 ECSubWrite(tid=68, reqid=client.4643.0:614, at_version=68'59, trim_to=0'0, roll_forward_to=68'56))", 2024-04-20T01:48:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:09.812830+0000", 2024-04-20T01:48:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.355835599000002, 2024-04-20T01:48:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066006179999999999, 2024-04-20T01:48:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.812830+0000", 2024-04-20T01:48:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.812830+0000", 2024-04-20T01:48:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.812829+0000", 2024-04-20T01:48:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.812899+0000", 2024-04-20T01:48:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9803999999999998e-05 2024-04-20T01:48:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.812912+0000", 2024-04-20T01:48:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3552e-05 2024-04-20T01:48:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.812921+0000", 2024-04-20T01:48:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7469999999999994e-06 2024-04-20T01:48:48.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.813009+0000", 2024-04-20T01:48:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8298000000000001e-05 2024-04-20T01:48:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.813023+0000", 2024-04-20T01:48:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3701e-05 2024-04-20T01:48:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.819402+0000", 2024-04-20T01:48:48.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063790799999999996 2024-04-20T01:48:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.819431+0000", 2024-04-20T01:48:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8513999999999998e-05 2024-04-20T01:48:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.cs2 68/30 ECSubWrite(tid=69, reqid=client.4643.0:615, at_version=68'60, trim_to=0'0, roll_forward_to=68'56))", 2024-04-20T01:48:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:09.814877+0000", 2024-04-20T01:48:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.353788356999999, 2024-04-20T01:48:48.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076095620000000003, 2024-04-20T01:48:48.184 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.184 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.184 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.814877+0000", 2024-04-20T01:48:48.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.814877+0000", 2024-04-20T01:48:48.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.814876+0000", 2024-04-20T01:48:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.814983+0000", 2024-04-20T01:48:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010743699999999999 2024-04-20T01:48:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.814996+0000", 2024-04-20T01:48:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2884000000000001e-05 2024-04-20T01:48:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.186 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815005+0000", 2024-04-20T01:48:48.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5369999999999997e-06 2024-04-20T01:48:48.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.186 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815050+0000", 2024-04-20T01:48:48.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5364999999999997e-05 2024-04-20T01:48:48.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.186 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815059+0000", 2024-04-20T01:48:48.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9530000000000005e-06 2024-04-20T01:48:48.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.187 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.187 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.187 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.822460+0000", 2024-04-20T01:48:48.187 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0074009109999999996 2024-04-20T01:48:48.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.187 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.187 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.187 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.822487+0000", 2024-04-20T01:48:48.187 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6939000000000001e-05 2024-04-20T01:48:48.187 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.188 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.188 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.188 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.188 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.188 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.cs2 68/30 ECSubWrite(tid=70, reqid=client.4643.0:616, at_version=68'61, trim_to=0'0, roll_forward_to=68'56))", 2024-04-20T01:48:48.188 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:09.815775+0000", 2024-04-20T01:48:48.188 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.352890785, 2024-04-20T01:48:48.188 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0067272979999999996, 2024-04-20T01:48:48.188 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.188 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.188 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.188 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.188 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.188 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815775+0000", 2024-04-20T01:48:48.189 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.189 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815775+0000", 2024-04-20T01:48:48.189 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.189 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815774+0000", 2024-04-20T01:48:48.189 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:48.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815794+0000", 2024-04-20T01:48:48.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9891999999999999e-05 2024-04-20T01:48:48.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815808+0000", 2024-04-20T01:48:48.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4584e-05 2024-04-20T01:48:48.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.815819+0000", 2024-04-20T01:48:48.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0216e-05 2024-04-20T01:48:48.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.191 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.816802+0000", 2024-04-20T01:48:48.191 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0009835480000000001 2024-04-20T01:48:48.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.191 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.816813+0000", 2024-04-20T01:48:48.191 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1156e-05 2024-04-20T01:48:48.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.191 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.822491+0000", 2024-04-20T01:48:48.192 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056774030000000001 2024-04-20T01:48:48.192 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.192 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.192 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.192 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:09.822502+0000", 2024-04-20T01:48:48.192 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1297999999999999e-05 2024-04-20T01:48:48.192 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.192 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.192 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.192 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.192 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.192 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.cs2 70/30 ECSubWrite(tid=103, reqid=client.4643.0:738, at_version=70'83, trim_to=0'0, roll_forward_to=70'80))", 2024-04-20T01:48:48.192 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:11.861902+0000", 2024-04-20T01:48:48.192 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 36.306763697999997, 2024-04-20T01:48:48.193 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0073928659999999997, 2024-04-20T01:48:48.193 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.193 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.193 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.193 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.193 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.193 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861902+0000", 2024-04-20T01:48:48.193 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.193 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.193 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.193 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861902+0000", 2024-04-20T01:48:48.193 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.193 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.194 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.194 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861901+0000", 2024-04-20T01:48:48.194 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:48.194 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.194 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.194 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.194 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861922+0000", 2024-04-20T01:48:48.194 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0917e-05 2024-04-20T01:48:48.194 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.194 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.194 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.194 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861935+0000", 2024-04-20T01:48:48.194 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3237e-05 2024-04-20T01:48:48.194 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.195 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.195 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.195 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.861945+0000", 2024-04-20T01:48:48.195 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0326000000000001e-05 2024-04-20T01:48:48.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.195 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.195 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.195 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.862010+0000", 2024-04-20T01:48:48.195 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4083000000000003e-05 2024-04-20T01:48:48.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.195 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.195 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.195 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.862019+0000", 2024-04-20T01:48:48.195 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0659999999999993e-06 2024-04-20T01:48:48.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.196 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.196 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.196 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.869223+0000", 2024-04-20T01:48:48.196 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007204612 2024-04-20T01:48:48.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.196 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.196 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.196 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:11.869295+0000", 2024-04-20T01:48:48.196 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1421999999999995e-05 2024-04-20T01:48:48.196 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.196 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.196 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.196 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.197 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 75/30 ECSubWrite(tid=357, reqid=osd.13.0:357, at_version=75'127, trim_to=0'0, roll_forward_to=75'123))", 2024-04-20T01:48:48.197 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:16.739745+0000", 2024-04-20T01:48:48.197 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 31.428920628, 2024-04-20T01:48:48.197 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006651985, 2024-04-20T01:48:48.197 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.197 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.197 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.197 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.197 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.197 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739745+0000", 2024-04-20T01:48:48.197 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.197 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.197 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.197 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.198 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739745+0000", 2024-04-20T01:48:48.198 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.198 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.198 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.198 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.198 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739744+0000", 2024-04-20T01:48:48.198 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:48.198 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.198 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.198 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.198 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739763+0000", 2024-04-20T01:48:48.198 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9182e-05 2024-04-20T01:48:48.198 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.199 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.199 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.199 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739776+0000", 2024-04-20T01:48:48.199 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2548000000000001e-05 2024-04-20T01:48:48.199 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.199 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.199 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.199 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739782+0000", 2024-04-20T01:48:48.199 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0909999999999998e-06 2024-04-20T01:48:48.199 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.199 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.199 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.199 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739826+0000", 2024-04-20T01:48:48.199 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3927999999999997e-05 2024-04-20T01:48:48.200 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.200 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.200 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.200 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739836+0000", 2024-04-20T01:48:48.200 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0368e-05 2024-04-20T01:48:48.200 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.200 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.200 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.200 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.746372+0000", 2024-04-20T01:48:48.200 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065361309999999997 2024-04-20T01:48:48.200 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.200 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.200 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.200 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.746397+0000", 2024-04-20T01:48:48.201 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4477999999999998e-05 2024-04-20T01:48:48.201 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.201 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.201 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.201 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.201 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.201 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 75/30 ECSubWrite(tid=358, reqid=osd.13.0:357, at_version=0'0, trim_to=0'0, roll_forward_to=75'127))", 2024-04-20T01:48:48.201 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:16.750738+0000", 2024-04-20T01:48:48.201 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 31.417927789, 2024-04-20T01:48:48.201 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066441419999999996, 2024-04-20T01:48:48.201 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.201 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.202 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.202 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.202 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.202 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.750738+0000", 2024-04-20T01:48:48.202 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.202 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.202 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.202 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.202 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.750738+0000", 2024-04-20T01:48:48.202 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.202 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.202 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.202 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.202 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.750737+0000", 2024-04-20T01:48:48.203 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:48.203 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.203 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.203 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.203 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.750748+0000", 2024-04-20T01:48:48.203 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0586e-05 2024-04-20T01:48:48.203 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.203 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.203 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.203 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.750760+0000", 2024-04-20T01:48:48.203 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2051e-05 2024-04-20T01:48:48.203 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.203 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.203 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.204 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.750768+0000", 2024-04-20T01:48:48.204 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0180000000000001e-06 2024-04-20T01:48:48.204 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.204 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.204 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.204 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.750817+0000", 2024-04-20T01:48:48.204 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9332000000000001e-05 2024-04-20T01:48:48.204 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.204 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.204 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.204 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.750827+0000", 2024-04-20T01:48:48.204 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0071000000000001e-05 2024-04-20T01:48:48.204 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.204 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.204 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.205 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.757354+0000", 2024-04-20T01:48:48.205 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065269899999999999 2024-04-20T01:48:48.205 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.205 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.205 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.205 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.757382+0000", 2024-04-20T01:48:48.205 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7761000000000001e-05 2024-04-20T01:48:48.205 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.205 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.205 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.205 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.205 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.205 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.1s4 76/30 ECSubWrite(tid=271, reqid=osd.4.0:271, at_version=76'43, trim_to=0'0, roll_forward_to=75'41))", 2024-04-20T01:48:48.205 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:17.716916+0000", 2024-04-20T01:48:48.206 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 30.451749221, 2024-04-20T01:48:48.206 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066297489999999999, 2024-04-20T01:48:48.206 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.206 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.206 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.206 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.206 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.206 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:17.716916+0000", 2024-04-20T01:48:48.206 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.206 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.206 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.206 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.206 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:17.716916+0000", 2024-04-20T01:48:48.206 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.207 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.207 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.207 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.207 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:17.716916+0000", 2024-04-20T01:48:48.207 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:48:48.207 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.207 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.207 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.207 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:17.716936+0000", 2024-04-20T01:48:48.207 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9553000000000001e-05 2024-04-20T01:48:48.207 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.207 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.207 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.207 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:17.716947+0000", 2024-04-20T01:48:48.207 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.13e-05 2024-04-20T01:48:48.208 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.208 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.208 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.208 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:17.716961+0000", 2024-04-20T01:48:48.208 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4402e-05 2024-04-20T01:48:48.208 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.208 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.208 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.208 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:17.717016+0000", 2024-04-20T01:48:48.208 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4577999999999998e-05 2024-04-20T01:48:48.208 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.208 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.208 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.208 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:17.717027+0000", 2024-04-20T01:48:48.209 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0776e-05 2024-04-20T01:48:48.209 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.209 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.209 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.209 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:17.723515+0000", 2024-04-20T01:48:48.209 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064878319999999998 2024-04-20T01:48:48.209 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.209 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.209 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.209 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:17.723546+0000", 2024-04-20T01:48:48.209 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1439000000000002e-05 2024-04-20T01:48:48.209 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.209 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.209 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.210 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.210 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.210 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.1s4 80/30 ECSubWrite(tid=302, reqid=osd.4.0:302, at_version=80'47, trim_to=0'0, roll_forward_to=77'45))", 2024-04-20T01:48:48.210 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:21.728944+0000", 2024-04-20T01:48:48.210 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.439721217999999, 2024-04-20T01:48:48.210 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0093705790000000004, 2024-04-20T01:48:48.210 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.210 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.210 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.210 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.210 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.210 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.728944+0000", 2024-04-20T01:48:48.210 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.211 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.211 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.211 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.211 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.728944+0000", 2024-04-20T01:48:48.211 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.211 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.211 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.211 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.211 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.728944+0000", 2024-04-20T01:48:48.211 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:48.211 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.211 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.211 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.211 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.728974+0000", 2024-04-20T01:48:48.212 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9991000000000001e-05 2024-04-20T01:48:48.212 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.212 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.212 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.212 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.728997+0000", 2024-04-20T01:48:48.212 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2796999999999999e-05 2024-04-20T01:48:48.212 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.212 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.212 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.212 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729010+0000", 2024-04-20T01:48:48.212 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2819999999999999e-05 2024-04-20T01:48:48.212 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.212 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.212 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.212 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729099+0000", 2024-04-20T01:48:48.213 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9382999999999994e-05 2024-04-20T01:48:48.213 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.213 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.213 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.213 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729151+0000", 2024-04-20T01:48:48.213 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1569e-05 2024-04-20T01:48:48.213 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.213 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.213 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.213 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.738292+0000", 2024-04-20T01:48:48.213 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0091419180000000006 2024-04-20T01:48:48.213 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.213 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.213 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.214 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.738315+0000", 2024-04-20T01:48:48.214 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2424000000000001e-05 2024-04-20T01:48:48.214 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.214 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.214 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.214 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.214 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.214 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.cs2 80/30 ECSubWrite(tid=132, reqid=osd.14.0:132, at_version=80'111, trim_to=0'0, roll_forward_to=77'109))", 2024-04-20T01:48:48.214 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:21.729632+0000", 2024-04-20T01:48:48.214 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.439033105, 2024-04-20T01:48:48.214 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0086544840000000005, 2024-04-20T01:48:48.214 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.214 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.214 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.215 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.215 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.215 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729632+0000", 2024-04-20T01:48:48.215 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.215 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.215 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.215 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.215 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729632+0000", 2024-04-20T01:48:48.215 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.215 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.215 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.215 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.215 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729632+0000", 2024-04-20T01:48:48.215 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:48.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.216 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.216 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.216 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729665+0000", 2024-04-20T01:48:48.216 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2699e-05 2024-04-20T01:48:48.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.216 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.216 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.216 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729689+0000", 2024-04-20T01:48:48.216 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4661e-05 2024-04-20T01:48:48.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.216 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.216 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.216 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729701+0000", 2024-04-20T01:48:48.217 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1432e-05 2024-04-20T01:48:48.217 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.217 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.217 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.217 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729801+0000", 2024-04-20T01:48:48.217 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100319 2024-04-20T01:48:48.217 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.217 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.217 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.217 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729812+0000", 2024-04-20T01:48:48.218 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1139e-05 2024-04-20T01:48:48.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.218 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.218 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.218 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.738257+0000", 2024-04-20T01:48:48.218 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0084448430000000005 2024-04-20T01:48:48.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.218 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.218 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.218 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.738287+0000", 2024-04-20T01:48:48.218 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9682000000000001e-05 2024-04-20T01:48:48.218 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.218 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.218 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.219 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.219 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.219 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:906 3.as0 3:5c6da789:::smithi16579500-40:head [append 0~0] snapc 2f=[2f,2e,28] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-04-20T01:48:48.219 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:21.729776+0000", 2024-04-20T01:48:48.219 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.438889423999999, 2024-04-20T01:48:48.219 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.023520553, 2024-04-20T01:48:48.219 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.219 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:48.219 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:48.219 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:48.219 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:48.219 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 906 2024-04-20T01:48:48.219 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.219 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.220 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.220 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.220 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729776+0000", 2024-04-20T01:48:48.220 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.220 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.220 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.220 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.220 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729776+0000", 2024-04-20T01:48:48.220 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.220 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.220 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.220 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.220 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729774+0000", 2024-04-20T01:48:48.220 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:48.221 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.221 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.221 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.221 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729787+0000", 2024-04-20T01:48:48.221 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3174e-05 2024-04-20T01:48:48.221 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.221 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.221 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.221 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729805+0000", 2024-04-20T01:48:48.221 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7558999999999999e-05 2024-04-20T01:48:48.221 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.221 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.221 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.221 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729816+0000", 2024-04-20T01:48:48.222 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.111e-05 2024-04-20T01:48:48.222 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.222 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.222 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.222 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729925+0000", 2024-04-20T01:48:48.222 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000109251 2024-04-20T01:48:48.222 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.222 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.222 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op must wait for map", 2024-04-20T01:48:48.222 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729934+0000", 2024-04-20T01:48:48.222 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0880000000000001e-06 2024-04-20T01:48:48.222 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.222 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.223 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.223 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.733659+0000", 2024-04-20T01:48:48.223 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037253389999999998 2024-04-20T01:48:48.223 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.223 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.223 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:48.223 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.733787+0000", 2024-04-20T01:48:48.223 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012743499999999999 2024-04-20T01:48:48.223 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.223 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.223 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.223 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.741029+0000", 2024-04-20T01:48:48.224 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0072423269999999998 2024-04-20T01:48:48.224 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.224 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.224 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.224 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.749061+0000", 2024-04-20T01:48:48.224 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080322569999999992 2024-04-20T01:48:48.224 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.224 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.224 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:48.224 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.753216+0000", 2024-04-20T01:48:48.224 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00415468 2024-04-20T01:48:48.224 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.224 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.224 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.224 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.753297+0000", 2024-04-20T01:48:48.225 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0539999999999998e-05 2024-04-20T01:48:48.225 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.225 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.225 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.225 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.225 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.225 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:907 3.as0 3:5c6da789:::smithi16579500-40:head [rollback 28] snapc 2f=[2f,2e,28] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-04-20T01:48:48.225 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:21.729835+0000", 2024-04-20T01:48:48.225 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.438830600999999, 2024-04-20T01:48:48.225 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.02475664, 2024-04-20T01:48:48.225 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.225 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:48.225 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:48.226 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:48.226 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:48.226 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 907 2024-04-20T01:48:48.226 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.226 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.226 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.226 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.226 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729835+0000", 2024-04-20T01:48:48.226 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.226 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.226 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.226 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.227 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729835+0000", 2024-04-20T01:48:48.227 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.227 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.227 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.227 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729834+0000", 2024-04-20T01:48:48.227 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:48.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.227 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.227 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.228 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729838+0000", 2024-04-20T01:48:48.228 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1729999999999997e-06 2024-04-20T01:48:48.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.228 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.228 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.228 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729846+0000", 2024-04-20T01:48:48.228 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9379999999999992e-06 2024-04-20T01:48:48.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.228 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.228 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.228 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729853+0000", 2024-04-20T01:48:48.228 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6590000000000001e-06 2024-04-20T01:48:48.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.229 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.229 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729995+0000", 2024-04-20T01:48:48.229 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000142397 2024-04-20T01:48:48.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.229 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.229 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:48.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.730002+0000", 2024-04-20T01:48:48.229 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7950000000000002e-06 2024-04-20T01:48:48.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.229 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.229 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.741764+0000", 2024-04-20T01:48:48.230 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011761466 2024-04-20T01:48:48.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.230 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.230 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:48.230 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.741851+0000", 2024-04-20T01:48:48.230 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7447999999999994e-05 2024-04-20T01:48:48.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.230 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.230 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.230 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.742330+0000", 2024-04-20T01:48:48.230 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00047901900000000002 2024-04-20T01:48:48.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.230 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.231 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.231 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.752214+0000", 2024-04-20T01:48:48.231 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0098841180000000008 2024-04-20T01:48:48.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.231 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.231 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:48.231 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.754530+0000", 2024-04-20T01:48:48.231 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0023153750000000002 2024-04-20T01:48:48.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.231 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.231 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.231 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.754592+0000", 2024-04-20T01:48:48.231 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1982000000000004e-05 2024-04-20T01:48:48.231 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.232 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.232 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.232 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.232 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:908 3.as0 3:5c6da789:::smithi16579500-40:head [append 0~0] snapc 2f=[2f,2e,28] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-04-20T01:48:48.232 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:21.729859+0000", 2024-04-20T01:48:48.232 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.438806486000001, 2024-04-20T01:48:48.232 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.024873134000000002, 2024-04-20T01:48:48.232 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.232 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:48.232 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:48.232 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:48.232 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:48.232 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 908 2024-04-20T01:48:48.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.233 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.233 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.233 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.233 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729859+0000", 2024-04-20T01:48:48.233 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.233 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.233 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.233 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729859+0000", 2024-04-20T01:48:48.233 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.233 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.233 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.234 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729858+0000", 2024-04-20T01:48:48.234 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:48.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.234 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.234 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.234 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729862+0000", 2024-04-20T01:48:48.234 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3440000000000001e-06 2024-04-20T01:48:48.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.234 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.234 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.234 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729868+0000", 2024-04-20T01:48:48.234 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1009999999999997e-06 2024-04-20T01:48:48.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.235 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.235 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.235 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.729874+0000", 2024-04-20T01:48:48.235 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8250000000000001e-06 2024-04-20T01:48:48.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.235 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.235 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.235 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.730024+0000", 2024-04-20T01:48:48.235 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000150233 2024-04-20T01:48:48.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.235 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.235 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:48.235 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.730030+0000", 2024-04-20T01:48:48.236 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0070000000000003e-06 2024-04-20T01:48:48.236 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.236 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.236 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.236 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.744082+0000", 2024-04-20T01:48:48.236 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014052090999999999 2024-04-20T01:48:48.236 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.236 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.236 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:48.236 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.744147+0000", 2024-04-20T01:48:48.236 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5204999999999997e-05 2024-04-20T01:48:48.236 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.236 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.236 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.236 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.744490+0000", 2024-04-20T01:48:48.237 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000343135 2024-04-20T01:48:48.237 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.237 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.237 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.237 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.752246+0000", 2024-04-20T01:48:48.237 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077554449999999997 2024-04-20T01:48:48.237 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.237 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.237 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:48.237 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.754690+0000", 2024-04-20T01:48:48.237 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024443870000000001 2024-04-20T01:48:48.237 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.237 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.237 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.237 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.754732+0000", 2024-04-20T01:48:48.238 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1870999999999999e-05 2024-04-20T01:48:48.238 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.238 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.238 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.238 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.238 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.238 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s2 80/30 ECSubWrite(tid=388, reqid=osd.13.0:386, at_version=0'0, trim_to=0'0, roll_forward_to=80'136))", 2024-04-20T01:48:48.238 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:21.735749+0000", 2024-04-20T01:48:48.238 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.432916548000001, 2024-04-20T01:48:48.238 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016494279000000001, 2024-04-20T01:48:48.238 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.238 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.238 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.238 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.239 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.239 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.735749+0000", 2024-04-20T01:48:48.239 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.239 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.239 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.239 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.239 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.735749+0000", 2024-04-20T01:48:48.239 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.239 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.239 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.239 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.239 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.735748+0000", 2024-04-20T01:48:48.239 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:48.239 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.240 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.240 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.240 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.735761+0000", 2024-04-20T01:48:48.240 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2860000000000001e-05 2024-04-20T01:48:48.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.240 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.240 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.240 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.735771+0000", 2024-04-20T01:48:48.240 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0312000000000001e-05 2024-04-20T01:48:48.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.240 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.240 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.240 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.735780+0000", 2024-04-20T01:48:48.240 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1600000000000004e-06 2024-04-20T01:48:48.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.241 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.241 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.241 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.744021+0000", 2024-04-20T01:48:48.241 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.008240486 2024-04-20T01:48:48.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.241 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.241 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.241 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.744026+0000", 2024-04-20T01:48:48.241 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3190000000000002e-06 2024-04-20T01:48:48.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.241 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.241 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.241 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.752220+0000", 2024-04-20T01:48:48.241 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0081939379999999996 2024-04-20T01:48:48.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.242 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.242 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.242 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.752243+0000", 2024-04-20T01:48:48.242 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3215000000000001e-05 2024-04-20T01:48:48.242 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.242 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.242 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.242 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.242 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.0s2 80/30 ECSubWrite(tid=69, reqid=osd.3.0:69, at_version=80'51, trim_to=0'0, roll_forward_to=77'49))", 2024-04-20T01:48:48.242 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:21.740775+0000", 2024-04-20T01:48:48.242 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.427890011999999, 2024-04-20T01:48:48.242 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083246099999999996, 2024-04-20T01:48:48.243 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.243 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.243 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.243 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.243 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.243 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740775+0000", 2024-04-20T01:48:48.243 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.243 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.243 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.243 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740775+0000", 2024-04-20T01:48:48.243 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.243 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.244 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.244 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740775+0000", 2024-04-20T01:48:48.244 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967296 2024-04-20T01:48:48.244 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.244 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.244 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.244 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740795+0000", 2024-04-20T01:48:48.244 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9623000000000002e-05 2024-04-20T01:48:48.244 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.244 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.244 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.244 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740804+0000", 2024-04-20T01:48:48.244 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3789999999999995e-06 2024-04-20T01:48:48.244 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.244 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.244 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.245 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740811+0000", 2024-04-20T01:48:48.245 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0249999999999997e-06 2024-04-20T01:48:48.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.245 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.245 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.245 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740869+0000", 2024-04-20T01:48:48.245 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8099000000000001e-05 2024-04-20T01:48:48.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.245 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.245 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.245 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.740877+0000", 2024-04-20T01:48:48.245 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4730000000000003e-06 2024-04-20T01:48:48.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.245 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.246 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.246 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.749075+0000", 2024-04-20T01:48:48.246 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0081977590000000006 2024-04-20T01:48:48.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.246 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.246 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.246 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.749100+0000", 2024-04-20T01:48:48.246 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5443999999999999e-05 2024-04-20T01:48:48.246 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.246 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.246 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.246 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.246 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:953 3.ds0 3:b226b74f:::smithi16579500-7:head [delete] snapc 36=[36,35,33,31,28] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e86)", 2024-04-20T01:48:48.247 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:27.764111+0000", 2024-04-20T01:48:48.247 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.404554211000001, 2024-04-20T01:48:48.247 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0066564759999999997, 2024-04-20T01:48:48.247 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.247 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:48.247 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:48.247 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:48.247 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:48.247 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 953 2024-04-20T01:48:48.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.247 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.247 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.247 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.248 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.764111+0000", 2024-04-20T01:48:48.248 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.248 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.248 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.248 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.764111+0000", 2024-04-20T01:48:48.248 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.248 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.248 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.248 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.764110+0000", 2024-04-20T01:48:48.249 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:48.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.249 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.249 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.249 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.764117+0000", 2024-04-20T01:48:48.249 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7309999999999993e-06 2024-04-20T01:48:48.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.249 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.249 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.249 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.764132+0000", 2024-04-20T01:48:48.249 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4289e-05 2024-04-20T01:48:48.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.250 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.250 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.250 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.764142+0000", 2024-04-20T01:48:48.250 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0710000000000001e-05 2024-04-20T01:48:48.250 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.250 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.250 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.250 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.764201+0000", 2024-04-20T01:48:48.250 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8347999999999998e-05 2024-04-20T01:48:48.250 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.250 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.250 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op must wait for map", 2024-04-20T01:48:48.251 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.764208+0000", 2024-04-20T01:48:48.251 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4059999999999999e-06 2024-04-20T01:48:48.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.251 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.251 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.251 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.766405+0000", 2024-04-20T01:48:48.251 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021969889999999999 2024-04-20T01:48:48.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.251 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.251 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:48.251 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.766512+0000", 2024-04-20T01:48:48.251 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000107145 2024-04-20T01:48:48.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.252 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.252 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.252 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.770768+0000", 2024-04-20T01:48:48.252 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042555759999999996 2024-04-20T01:48:48.252 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.252 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.252 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.252 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.252 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.252 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:48.254 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-20T01:48:48.382 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:48.382 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:48.383 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:48:48.383 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:48.393 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-20T01:48:48.403 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:48:48.417+0000 7fbd5bdee7c0 -1 Falling back to public interface 2024-04-20T01:48:48.520 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:48.520 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:48.520 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 30, 2024-04-20T01:48:48.520 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-04-20T01:48:48.520 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:48.530 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-20T01:48:48.701 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:48.701 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-04-20T01:48:48.701 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-04-20T01:48:48.701 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-04-20T01:48:48.701 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.701 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:10 3.5s0 3:a2e9ac6b:::smithi16579500-4:head [append 0~65536,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-20T01:48:48.701 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.752800+0000", 2024-04-20T01:48:48.701 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 73.961713118000006, 2024-04-20T01:48:48.701 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011440333, 2024-04-20T01:48:48.701 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.702 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:48.702 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:48.702 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:48.702 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:48.702 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 10 2024-04-20T01:48:48.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.702 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.702 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.702 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752800+0000", 2024-04-20T01:48:48.702 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.702 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752800+0000", 2024-04-20T01:48:48.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.703 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.703 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752799+0000", 2024-04-20T01:48:48.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:48.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.703 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.703 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752945+0000", 2024-04-20T01:48:48.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000145761 2024-04-20T01:48:48.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.703 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.704 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752955+0000", 2024-04-20T01:48:48.704 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0229e-05 2024-04-20T01:48:48.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.704 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752970+0000", 2024-04-20T01:48:48.704 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5242000000000001e-05 2024-04-20T01:48:48.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.704 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753046+0000", 2024-04-20T01:48:48.704 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6218999999999996e-05 2024-04-20T01:48:48.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:48.705 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753172+0000", 2024-04-20T01:48:48.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012566899999999999 2024-04-20T01:48:48.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.705 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.705 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753648+0000", 2024-04-20T01:48:48.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00047565500000000001 2024-04-20T01:48:48.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.705 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.705 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.756671+0000", 2024-04-20T01:48:48.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030231849999999998 2024-04-20T01:48:48.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.705 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:48.706 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764191+0000", 2024-04-20T01:48:48.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007520143 2024-04-20T01:48:48.706 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.706 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.706 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.706 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764240+0000", 2024-04-20T01:48:48.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9359e-05 2024-04-20T01:48:48.706 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.706 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.707 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.707 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:11 3.5s0 3:a2e9ac6b:::smithi16579500-4:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-20T01:48:48.707 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.752987+0000", 2024-04-20T01:48:48.707 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 73.961526481999996, 2024-04-20T01:48:48.707 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011858695000000001, 2024-04-20T01:48:48.707 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.707 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:48.707 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:48.707 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:48.707 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:48.707 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 11 2024-04-20T01:48:48.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.707 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.708 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752987+0000", 2024-04-20T01:48:48.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.708 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752987+0000", 2024-04-20T01:48:48.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.708 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752986+0000", 2024-04-20T01:48:48.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:48.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.709 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.709 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.709 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.752995+0000", 2024-04-20T01:48:48.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7099999999999996e-06 2024-04-20T01:48:48.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.709 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.709 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.709 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753000+0000", 2024-04-20T01:48:48.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4900000000000002e-06 2024-04-20T01:48:48.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.709 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.709 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.709 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753003+0000", 2024-04-20T01:48:48.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4620000000000001e-06 2024-04-20T01:48:48.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753974+0000", 2024-04-20T01:48:48.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000971388 2024-04-20T01:48:48.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:48.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754054+0000", 2024-04-20T01:48:48.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9159000000000006e-05 2024-04-20T01:48:48.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754358+0000", 2024-04-20T01:48:48.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00030434800000000002 2024-04-20T01:48:48.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.759666+0000", 2024-04-20T01:48:48.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053076169999999997 2024-04-20T01:48:48.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:48.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764756+0000", 2024-04-20T01:48:48.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050904139999999997 2024-04-20T01:48:48.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.764845+0000", 2024-04-20T01:48:48.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9488999999999996e-05 2024-04-20T01:48:48.712 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.712 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.712 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.712 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:12 3.5s0 3:a2e9ac6b:::smithi16579500-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-20T01:48:48.712 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.753012+0000", 2024-04-20T01:48:48.712 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 73.961501464999998, 2024-04-20T01:48:48.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014201745, 2024-04-20T01:48:48.712 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.712 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:48.712 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:48.712 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:48.712 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:48.713 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 12 2024-04-20T01:48:48.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.713 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.713 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.713 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753012+0000", 2024-04-20T01:48:48.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.713 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.713 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753012+0000", 2024-04-20T01:48:48.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.713 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.714 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753011+0000", 2024-04-20T01:48:48.714 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:48.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.714 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.714 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753016+0000", 2024-04-20T01:48:48.714 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.989e-06 2024-04-20T01:48:48.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.714 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753026+0000", 2024-04-20T01:48:48.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2690000000000005e-06 2024-04-20T01:48:48.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.753033+0000", 2024-04-20T01:48:48.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3010000000000001e-06 2024-04-20T01:48:48.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754553+0000", 2024-04-20T01:48:48.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001519947 2024-04-20T01:48:48.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:48.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.754638+0000", 2024-04-20T01:48:48.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5421999999999996e-05 2024-04-20T01:48:48.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.765222+0000", 2024-04-20T01:48:48.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010583979 2024-04-20T01:48:48.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:48.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.765288+0000", 2024-04-20T01:48:48.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5296999999999995e-05 2024-04-20T01:48:48.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.717 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.717 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.717 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.767213+0000", 2024-04-20T01:48:48.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019259520000000001 2024-04-20T01:48:48.717 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.717 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.717 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.717 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.717 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:64 3.5s0 3:ae9f92b8:::smithi16579500-22:head [append 0~360448,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:48.717 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.781054+0000", 2024-04-20T01:48:48.717 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 73.933458997000002, 2024-04-20T01:48:48.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01202491, 2024-04-20T01:48:48.718 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.718 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:48.718 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:48.718 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:48.718 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:48.718 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 64 2024-04-20T01:48:48.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.718 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.718 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.718 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781054+0000", 2024-04-20T01:48:48.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781054+0000", 2024-04-20T01:48:48.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781053+0000", 2024-04-20T01:48:48.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:48.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781384+0000", 2024-04-20T01:48:48.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00033060899999999999 2024-04-20T01:48:48.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781394+0000", 2024-04-20T01:48:48.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0016e-05 2024-04-20T01:48:48.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781399+0000", 2024-04-20T01:48:48.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7930000000000003e-06 2024-04-20T01:48:48.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781446+0000", 2024-04-20T01:48:48.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6768999999999999e-05 2024-04-20T01:48:48.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:48.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781598+0000", 2024-04-20T01:48:48.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015158500000000001 2024-04-20T01:48:48.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.782377+0000", 2024-04-20T01:48:48.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000778975 2024-04-20T01:48:48.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784105+0000", 2024-04-20T01:48:48.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017285340000000001 2024-04-20T01:48:48.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.722 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:48.722 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793040+0000", 2024-04-20T01:48:48.722 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0089346440000000003 2024-04-20T01:48:48.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.722 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.722 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793079+0000", 2024-04-20T01:48:48.722 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9239000000000002e-05 2024-04-20T01:48:48.722 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.722 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.722 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.723 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:65 3.5s0 3:ae9f92b8:::smithi16579500-22:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:48.723 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.781409+0000", 2024-04-20T01:48:48.723 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 73.933104396999994, 2024-04-20T01:48:48.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011886404, 2024-04-20T01:48:48.723 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.723 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:48.723 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:48.723 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:48.723 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:48.723 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 65 2024-04-20T01:48:48.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.723 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.723 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781409+0000", 2024-04-20T01:48:48.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781409+0000", 2024-04-20T01:48:48.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781408+0000", 2024-04-20T01:48:48.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:48.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781411+0000", 2024-04-20T01:48:48.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6309999999999999e-06 2024-04-20T01:48:48.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781416+0000", 2024-04-20T01:48:48.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6550000000000003e-06 2024-04-20T01:48:48.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781423+0000", 2024-04-20T01:48:48.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4540000000000001e-06 2024-04-20T01:48:48.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.782838+0000", 2024-04-20T01:48:48.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001414741 2024-04-20T01:48:48.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:48.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.782921+0000", 2024-04-20T01:48:48.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3018999999999999e-05 2024-04-20T01:48:48.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.783284+0000", 2024-04-20T01:48:48.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00036349 2024-04-20T01:48:48.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.727 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.727 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.784149+0000", 2024-04-20T01:48:48.727 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00086487700000000001 2024-04-20T01:48:48.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.727 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-04-20T01:48:48.727 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793195+0000", 2024-04-20T01:48:48.727 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0090454590000000005 2024-04-20T01:48:48.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.727 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.727 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793295+0000", 2024-04-20T01:48:48.727 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010049 2024-04-20T01:48:48.727 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.728 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.728 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.728 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:66 3.5s0 3:ae9f92b8:::smithi16579500-22:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:48.728 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:34.781436+0000", 2024-04-20T01:48:48.728 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 73.933076966000002, 2024-04-20T01:48:48.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014237204999999999, 2024-04-20T01:48:48.728 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.728 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:48.728 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:48.728 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:48.728 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:48.728 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 66 2024-04-20T01:48:48.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.729 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781436+0000", 2024-04-20T01:48:48.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781436+0000", 2024-04-20T01:48:48.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781436+0000", 2024-04-20T01:48:48.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:48.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781441+0000", 2024-04-20T01:48:48.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3199999999999999e-06 2024-04-20T01:48:48.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781446+0000", 2024-04-20T01:48:48.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.544e-06 2024-04-20T01:48:48.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.781451+0000", 2024-04-20T01:48:48.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5570000000000003e-06 2024-04-20T01:48:48.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.783430+0000", 2024-04-20T01:48:48.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019785699999999998 2024-04-20T01:48:48.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:48.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.783482+0000", 2024-04-20T01:48:48.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1780000000000002e-05 2024-04-20T01:48:48.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793826+0000", 2024-04-20T01:48:48.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010344132000000001 2024-04-20T01:48:48.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:48.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.793907+0000", 2024-04-20T01:48:48.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1173000000000004e-05 2024-04-20T01:48:48.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:34.795673+0000", 2024-04-20T01:48:48.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001766442 2024-04-20T01:48:48.733 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.733 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.733 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.733 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s6 48/30 ECSubWrite(tid=251, reqid=osd.13.0:251, at_version=48'22, trim_to=0'0, roll_forward_to=47'20))", 2024-04-20T01:48:48.733 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.710374+0000", 2024-04-20T01:48:48.733 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 59.004139062999997, 2024-04-20T01:48:48.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069805379999999997, 2024-04-20T01:48:48.733 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.733 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.733 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710374+0000", 2024-04-20T01:48:48.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710374+0000", 2024-04-20T01:48:48.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710374+0000", 2024-04-20T01:48:48.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:48.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710395+0000", 2024-04-20T01:48:48.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1396e-05 2024-04-20T01:48:48.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710417+0000", 2024-04-20T01:48:48.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1857000000000001e-05 2024-04-20T01:48:48.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710423+0000", 2024-04-20T01:48:48.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1469999999999998e-06 2024-04-20T01:48:48.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710498+0000", 2024-04-20T01:48:48.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5072000000000002e-05 2024-04-20T01:48:48.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.710509+0000", 2024-04-20T01:48:48.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0685e-05 2024-04-20T01:48:48.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717332+0000", 2024-04-20T01:48:48.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006823048 2024-04-20T01:48:48.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717355+0000", 2024-04-20T01:48:48.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2731e-05 2024-04-20T01:48:48.737 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.737 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.737 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.737 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 48/30 ECSubWrite(tid=253, reqid=osd.13.0:253, at_version=48'44, trim_to=0'0, roll_forward_to=47'40))", 2024-04-20T01:48:48.737 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:47:49.711134+0000", 2024-04-20T01:48:48.738 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 59.003379086000002, 2024-04-20T01:48:48.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006623304, 2024-04-20T01:48:48.738 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.738 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.738 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711134+0000", 2024-04-20T01:48:48.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711134+0000", 2024-04-20T01:48:48.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711134+0000", 2024-04-20T01:48:48.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:48.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711155+0000", 2024-04-20T01:48:48.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1673000000000001e-05 2024-04-20T01:48:48.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711166+0000", 2024-04-20T01:48:48.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0558e-05 2024-04-20T01:48:48.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711174+0000", 2024-04-20T01:48:48.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3359999999999995e-06 2024-04-20T01:48:48.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711257+0000", 2024-04-20T01:48:48.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3312999999999993e-05 2024-04-20T01:48:48.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.711273+0000", 2024-04-20T01:48:48.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5701000000000001e-05 2024-04-20T01:48:48.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717738+0000", 2024-04-20T01:48:48.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064653339999999997 2024-04-20T01:48:48.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:47:49.717757+0000", 2024-04-20T01:48:48.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8972999999999999e-05 2024-04-20T01:48:48.741 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.741 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.741 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.742 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 63/30 ECSubWrite(tid=295, reqid=client.4643.0:574, at_version=63'79, trim_to=0'0, roll_forward_to=62'77))", 2024-04-20T01:48:48.742 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:04.795161+0000", 2024-04-20T01:48:48.742 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 43.919352367000002, 2024-04-20T01:48:48.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0075404740000000001, 2024-04-20T01:48:48.742 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.742 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.742 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795161+0000", 2024-04-20T01:48:48.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795161+0000", 2024-04-20T01:48:48.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795159+0000", 2024-04-20T01:48:48.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:48.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795678+0000", 2024-04-20T01:48:48.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00051905099999999997 2024-04-20T01:48:48.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795694+0000", 2024-04-20T01:48:48.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6076e-05 2024-04-20T01:48:48.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795704+0000", 2024-04-20T01:48:48.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0078000000000001e-05 2024-04-20T01:48:48.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795753+0000", 2024-04-20T01:48:48.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9055999999999999e-05 2024-04-20T01:48:48.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.795763+0000", 2024-04-20T01:48:48.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6430000000000007e-06 2024-04-20T01:48:48.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.802666+0000", 2024-04-20T01:48:48.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069029149999999999 2024-04-20T01:48:48.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.802701+0000", 2024-04-20T01:48:48.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5664999999999998e-05 2024-04-20T01:48:48.745 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.746 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.746 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.746 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 63/30 ECSubWrite(tid=296, reqid=client.4643.0:575, at_version=63'80, trim_to=0'0, roll_forward_to=62'77))", 2024-04-20T01:48:48.746 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:04.798493+0000", 2024-04-20T01:48:48.746 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 43.916019761999998, 2024-04-20T01:48:48.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069235080000000001, 2024-04-20T01:48:48.746 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.746 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.746 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798493+0000", 2024-04-20T01:48:48.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798493+0000", 2024-04-20T01:48:48.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798493+0000", 2024-04-20T01:48:48.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:48.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798862+0000", 2024-04-20T01:48:48.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00036897900000000002 2024-04-20T01:48:48.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798876+0000", 2024-04-20T01:48:48.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4321999999999999e-05 2024-04-20T01:48:48.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798884+0000", 2024-04-20T01:48:48.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.157e-06 2024-04-20T01:48:48.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798936+0000", 2024-04-20T01:48:48.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1879000000000003e-05 2024-04-20T01:48:48.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.798949+0000", 2024-04-20T01:48:48.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3359e-05 2024-04-20T01:48:48.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.805386+0000", 2024-04-20T01:48:48.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064363470000000002 2024-04-20T01:48:48.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:04.805417+0000", 2024-04-20T01:48:48.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1260999999999998e-05 2024-04-20T01:48:48.750 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.750 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.750 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.750 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:717 3.5s0 3:ae9f92b8:::smithi16579500-22:head [read 0~2129920,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:48.750 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.113707+0000", 2024-04-20T01:48:48.750 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.600805932999997, 2024-04-20T01:48:48.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083649959999999995, 2024-04-20T01:48:48.750 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.750 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:48.750 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:48.751 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:48.751 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:48.751 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 717 2024-04-20T01:48:48.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.751 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113707+0000", 2024-04-20T01:48:48.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113707+0000", 2024-04-20T01:48:48.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113706+0000", 2024-04-20T01:48:48.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:48.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113714+0000", 2024-04-20T01:48:48.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.371e-06 2024-04-20T01:48:48.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113726+0000", 2024-04-20T01:48:48.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1572e-05 2024-04-20T01:48:48.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113732+0000", 2024-04-20T01:48:48.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2870000000000001e-06 2024-04-20T01:48:48.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113785+0000", 2024-04-20T01:48:48.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2648000000000001e-05 2024-04-20T01:48:48.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:48.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113900+0000", 2024-04-20T01:48:48.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011510699999999999 2024-04-20T01:48:48.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.122072+0000", 2024-04-20T01:48:48.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0081722930000000006 2024-04-20T01:48:48.754 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.754 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.754 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.754 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:718 3.5s0 3:ae9f92b8:::smithi16579500-22:head [read 0~2129920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:48.754 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.113742+0000", 2024-04-20T01:48:48.754 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.600770861999997, 2024-04-20T01:48:48.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095008990000000001, 2024-04-20T01:48:48.755 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.755 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:48.755 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:48.755 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:48.755 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:48.755 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 718 2024-04-20T01:48:48.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.755 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113742+0000", 2024-04-20T01:48:48.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113742+0000", 2024-04-20T01:48:48.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113742+0000", 2024-04-20T01:48:48.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:48.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113744+0000", 2024-04-20T01:48:48.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4219999999999999e-06 2024-04-20T01:48:48.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113749+0000", 2024-04-20T01:48:48.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.313e-06 2024-04-20T01:48:48.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113752+0000", 2024-04-20T01:48:48.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2449999999999998e-06 2024-04-20T01:48:48.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.117251+0000", 2024-04-20T01:48:48.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034990360000000001 2024-04-20T01:48:48.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:48.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.117402+0000", 2024-04-20T01:48:48.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000151099 2024-04-20T01:48:48.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.123243+0000", 2024-04-20T01:48:48.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058410800000000002 2024-04-20T01:48:48.758 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.761 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.761 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.762 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4643.0:719 3.5s0 3:ae9f92b8:::smithi16579500-22:head [read 0~2129920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-04-20T01:48:48.762 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:10.113770+0000", 2024-04-20T01:48:48.762 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.600743244, 2024-04-20T01:48:48.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010243805999999999, 2024-04-20T01:48:48.762 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.762 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-04-20T01:48:48.762 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-04-20T01:48:48.762 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4643", 2024-04-20T01:48:48.762 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:48.762 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 719 2024-04-20T01:48:48.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.763 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113770+0000", 2024-04-20T01:48:48.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113770+0000", 2024-04-20T01:48:48.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113770+0000", 2024-04-20T01:48:48.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:48.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113772+0000", 2024-04-20T01:48:48.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1270000000000001e-06 2024-04-20T01:48:48.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113777+0000", 2024-04-20T01:48:48.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4410000000000001e-06 2024-04-20T01:48:48.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.113780+0000", 2024-04-20T01:48:48.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.869e-06 2024-04-20T01:48:48.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.120794+0000", 2024-04-20T01:48:48.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070135659999999997 2024-04-20T01:48:48.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-04-20T01:48:48.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.120909+0000", 2024-04-20T01:48:48.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011497099999999999 2024-04-20T01:48:48.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:10.124014+0000", 2024-04-20T01:48:48.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00310509 2024-04-20T01:48:48.766 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.766 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.766 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.766 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.2s3 75/30 ECSubWrite(tid=357, reqid=osd.13.0:357, at_version=75'127, trim_to=0'0, roll_forward_to=75'123))", 2024-04-20T01:48:48.766 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:16.739742+0000", 2024-04-20T01:48:48.766 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 31.974771320999999, 2024-04-20T01:48:48.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006963365, 2024-04-20T01:48:48.766 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.766 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.766 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739742+0000", 2024-04-20T01:48:48.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739742+0000", 2024-04-20T01:48:48.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739741+0000", 2024-04-20T01:48:48.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:48.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739764+0000", 2024-04-20T01:48:48.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2824e-05 2024-04-20T01:48:48.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739776+0000", 2024-04-20T01:48:48.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1806e-05 2024-04-20T01:48:48.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739785+0000", 2024-04-20T01:48:48.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1500000000000005e-06 2024-04-20T01:48:48.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739842+0000", 2024-04-20T01:48:48.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7034000000000003e-05 2024-04-20T01:48:48.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.739853+0000", 2024-04-20T01:48:48.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0556e-05 2024-04-20T01:48:48.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.770 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.770 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.746679+0000", 2024-04-20T01:48:48.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068263880000000001 2024-04-20T01:48:48.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.770 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.770 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.746705+0000", 2024-04-20T01:48:48.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6219000000000001e-05 2024-04-20T01:48:48.770 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.770 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.770 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.771 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s6 75/30 ECSubWrite(tid=363, reqid=client.4643.0:790, at_version=75'30, trim_to=0'0, roll_forward_to=75'28))", 2024-04-20T01:48:48.771 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:16.766298+0000", 2024-04-20T01:48:48.771 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 31.948215480000002, 2024-04-20T01:48:48.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0078250469999999999, 2024-04-20T01:48:48.771 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.771 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.771 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.771 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.771 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.766298+0000", 2024-04-20T01:48:48.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.771 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.771 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.766298+0000", 2024-04-20T01:48:48.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.766297+0000", 2024-04-20T01:48:48.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:48.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.767936+0000", 2024-04-20T01:48:48.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016393880000000001 2024-04-20T01:48:48.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.767947+0000", 2024-04-20T01:48:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0061000000000001e-05 2024-04-20T01:48:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.767953+0000", 2024-04-20T01:48:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.083e-06 2024-04-20T01:48:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.767982+0000", 2024-04-20T01:48:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9203000000000001e-05 2024-04-20T01:48:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.767988+0000", 2024-04-20T01:48:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9669999999999998e-06 2024-04-20T01:48:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.774100+0000", 2024-04-20T01:48:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061125729999999996 2024-04-20T01:48:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:16.774123+0000", 2024-04-20T01:48:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2404000000000001e-05 2024-04-20T01:48:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.as1 80/30 ECSubWrite(tid=44, reqid=client.4643.0:907, at_version=80'23, trim_to=0'0, roll_forward_to=66'20))", 2024-04-20T01:48:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:21.742360+0000", 2024-04-20T01:48:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.972153007999999, 2024-04-20T01:48:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068772989999999999, 2024-04-20T01:48:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.742360+0000", 2024-04-20T01:48:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.742360+0000", 2024-04-20T01:48:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.742359+0000", 2024-04-20T01:48:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.742381+0000", 2024-04-20T01:48:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1467999999999998e-05 2024-04-20T01:48:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.742395+0000", 2024-04-20T01:48:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3939e-05 2024-04-20T01:48:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.742404+0000", 2024-04-20T01:48:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8810000000000002e-06 2024-04-20T01:48:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.742456+0000", 2024-04-20T01:48:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2024000000000002e-05 2024-04-20T01:48:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.742467+0000", 2024-04-20T01:48:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1106e-05 2024-04-20T01:48:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.749208+0000", 2024-04-20T01:48:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0067413129999999996 2024-04-20T01:48:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:21.749237+0000", 2024-04-20T01:48:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9439000000000001e-05 2024-04-20T01:48:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s6 86/30 ECSubWrite(tid=392, reqid=client.4643.0:960, at_version=86'41, trim_to=0'0, roll_forward_to=80'39))", 2024-04-20T01:48:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:27.784783+0000", 2024-04-20T01:48:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.929729783999999, 2024-04-20T01:48:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070335490000000001, 2024-04-20T01:48:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.784783+0000", 2024-04-20T01:48:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.784783+0000", 2024-04-20T01:48:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.784783+0000", 2024-04-20T01:48:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.784839+0000", 2024-04-20T01:48:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6625000000000001e-05 2024-04-20T01:48:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.784874+0000", 2024-04-20T01:48:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4425999999999999e-05 2024-04-20T01:48:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.784900+0000", 2024-04-20T01:48:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6333e-05 2024-04-20T01:48:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.785044+0000", 2024-04-20T01:48:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014331700000000001 2024-04-20T01:48:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.785055+0000", 2024-04-20T01:48:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1483e-05 2024-04-20T01:48:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.791786+0000", 2024-04-20T01:48:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0067313520000000003 2024-04-20T01:48:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.791817+0000", 2024-04-20T01:48:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0608999999999998e-05 2024-04-20T01:48:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.3s6 86/30 ECSubWrite(tid=393, reqid=client.4643.0:961, at_version=86'42, trim_to=0'0, roll_forward_to=80'39))", 2024-04-20T01:48:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:27.786036+0000", 2024-04-20T01:48:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.928476961000001, 2024-04-20T01:48:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006157226, 2024-04-20T01:48:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.786036+0000", 2024-04-20T01:48:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.786036+0000", 2024-04-20T01:48:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.786035+0000", 2024-04-20T01:48:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.786057+0000", 2024-04-20T01:48:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1311999999999999e-05 2024-04-20T01:48:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.786073+0000", 2024-04-20T01:48:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6728e-05 2024-04-20T01:48:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.786087+0000", 2024-04-20T01:48:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4136e-05 2024-04-20T01:48:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.786140+0000", 2024-04-20T01:48:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2845e-05 2024-04-20T01:48:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.786151+0000", 2024-04-20T01:48:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0937e-05 2024-04-20T01:48:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.792171+0000", 2024-04-20T01:48:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060194319999999999 2024-04-20T01:48:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:27.792193+0000", 2024-04-20T01:48:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2775000000000001e-05 2024-04-20T01:48:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.bs5 87/30 ECSubWrite(tid=332, reqid=client.4643.0:967, at_version=0'0, trim_to=0'0, roll_forward_to=87'164))", 2024-04-20T01:48:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:28.789649+0000", 2024-04-20T01:48:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.924864212999999, 2024-04-20T01:48:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061559609999999997, 2024-04-20T01:48:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:28.789649+0000", 2024-04-20T01:48:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:28.789649+0000", 2024-04-20T01:48:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:28.789648+0000", 2024-04-20T01:48:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.999999 2024-04-20T01:48:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:28.789662+0000", 2024-04-20T01:48:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3991e-05 2024-04-20T01:48:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:28.789674+0000", 2024-04-20T01:48:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2228e-05 2024-04-20T01:48:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:28.789684+0000", 2024-04-20T01:48:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2779999999999999e-06 2024-04-20T01:48:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:28.789732+0000", 2024-04-20T01:48:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8593e-05 2024-04-20T01:48:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:28.789743+0000", 2024-04-20T01:48:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0946e-05 2024-04-20T01:48:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:28.795783+0000", 2024-04-20T01:48:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006039928 2024-04-20T01:48:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:28.795805+0000", 2024-04-20T01:48:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1843000000000001e-05 2024-04-20T01:48:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "MOSDECSubOpWrite(3.bs5 91/30 ECSubWrite(tid=343, reqid=osd.4.0:343, at_version=91'174, trim_to=0'0, roll_forward_to=89'172))", 2024-04-20T01:48:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-04-20T01:48:32.720076+0000", 2024-04-20T01:48:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.994437035000001, 2024-04-20T01:48:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058909519999999996, 2024-04-20T01:48:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-04-20T01:48:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-04-20T01:48:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-04-20T01:48:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-04-20T01:48:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:32.720076+0000", 2024-04-20T01:48:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-04-20T01:48:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:32.720076+0000", 2024-04-20T01:48:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-04-20T01:48:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-04-20T01:48:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:32.720076+0000", 2024-04-20T01:48:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-04-20T01:48:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:32.720099+0000", 2024-04-20T01:48:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3176999999999998e-05 2024-04-20T01:48:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-04-20T01:48:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:32.720110+0000", 2024-04-20T01:48:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1574e-05 2024-04-20T01:48:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-04-20T01:48:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:32.720121+0000", 2024-04-20T01:48:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1066999999999999e-05 2024-04-20T01:48:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-04-20T01:48:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:32.720359+0000", 2024-04-20T01:48:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023742199999999999 2024-04-20T01:48:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_started", 2024-04-20T01:48:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:32.720370+0000", 2024-04-20T01:48:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1279e-05 2024-04-20T01:48:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_committed", 2024-04-20T01:48:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:32.725944+0000", 2024-04-20T01:48:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055740529999999998 2024-04-20T01:48:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T01:48:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-04-20T01:48:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-04-20T01:48:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-04-20T01:48:32.725967+0000", 2024-04-20T01:48:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2977e-05 2024-04-20T01:48:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T01:48:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-04-20T01:48:48.795 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:48.797 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-20T01:48:48.836 INFO:teuthology.orchestra.run.smithi044.stderr:nodeep-scrub is unset 2024-04-20T01:48:48.938 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:48.939 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:48.939 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-04-20T01:48:48.939 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:48.949 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-20T01:48:49.077 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:49.078 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:49.078 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:48:49.078 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:48:49.078 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:49.088 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-20T01:48:49.219 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:49.219 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:48:49.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:48:49.220 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:48:49.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.220 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:234 3.0s0 3:0c1ed9f1:::smithi16579500-46:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:49.220 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.787453+0000", 2024-04-20T01:48:49.220 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.445428151000002, 2024-04-20T01:48:49.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016795529, 2024-04-20T01:48:49.220 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.220 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:49.220 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:49.220 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:49.220 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:49.220 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 234 2024-04-20T01:48:49.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.221 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787453+0000", 2024-04-20T01:48:49.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787453+0000", 2024-04-20T01:48:49.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787451+0000", 2024-04-20T01:48:49.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:49.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787462+0000", 2024-04-20T01:48:49.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1083e-05 2024-04-20T01:48:49.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787483+0000", 2024-04-20T01:48:49.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1656999999999999e-05 2024-04-20T01:48:49.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787503+0000", 2024-04-20T01:48:49.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9623000000000002e-05 2024-04-20T01:48:49.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787595+0000", 2024-04-20T01:48:49.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2417000000000006e-05 2024-04-20T01:48:49.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:49.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787604+0000", 2024-04-20T01:48:49.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.952e-06 2024-04-20T01:48:49.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.791584+0000", 2024-04-20T01:48:49.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039792250000000003 2024-04-20T01:48:49.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:49.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.791699+0000", 2024-04-20T01:48:49.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115692 2024-04-20T01:48:49.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.792530+0000", 2024-04-20T01:48:49.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083077900000000002 2024-04-20T01:48:49.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.798777+0000", 2024-04-20T01:48:49.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062472379999999996 2024-04-20T01:48:49.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:49.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.804175+0000", 2024-04-20T01:48:49.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005397158 2024-04-20T01:48:49.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.804248+0000", 2024-04-20T01:48:49.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3682000000000004e-05 2024-04-20T01:48:49.225 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.225 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.225 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.226 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:235 3.0s0 3:0c1ed9f1:::smithi16579500-46:head [rollback 5] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:49.226 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.787517+0000", 2024-04-20T01:48:49.226 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.445364322000003, 2024-04-20T01:48:49.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019202061999999999, 2024-04-20T01:48:49.226 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.226 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:49.226 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:49.226 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:49.226 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:49.226 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 235 2024-04-20T01:48:49.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.226 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787517+0000", 2024-04-20T01:48:49.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787517+0000", 2024-04-20T01:48:49.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787516+0000", 2024-04-20T01:48:49.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:49.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787521+0000", 2024-04-20T01:48:49.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2599999999999996e-06 2024-04-20T01:48:49.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787534+0000", 2024-04-20T01:48:49.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2717e-05 2024-04-20T01:48:49.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787538+0000", 2024-04-20T01:48:49.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3540000000000002e-06 2024-04-20T01:48:49.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787673+0000", 2024-04-20T01:48:49.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013480799999999999 2024-04-20T01:48:49.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:49.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787678+0000", 2024-04-20T01:48:49.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5229999999999997e-06 2024-04-20T01:48:49.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793075+0000", 2024-04-20T01:48:49.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053977549999999997 2024-04-20T01:48:49.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:49.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793190+0000", 2024-04-20T01:48:49.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114775 2024-04-20T01:48:49.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793760+0000", 2024-04-20T01:48:49.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00056945599999999998 2024-04-20T01:48:49.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799198+0000", 2024-04-20T01:48:49.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0054385589999999999 2024-04-20T01:48:49.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:49.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806653+0000", 2024-04-20T01:48:49.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0074547739999999999 2024-04-20T01:48:49.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806719+0000", 2024-04-20T01:48:49.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5732e-05 2024-04-20T01:48:49.231 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.231 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.231 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.232 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:236 3.0s0 3:0c1ed9f1:::smithi16579500-46:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-20T01:48:49.232 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:48:49.232 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.445304707999995, 2024-04-20T01:48:49.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019273109, 2024-04-20T01:48:49.232 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.232 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:49.232 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:49.232 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:49.232 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:49.232 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 236 2024-04-20T01:48:49.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.232 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:48:49.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:48:49.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787576+0000", 2024-04-20T01:48:49.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:49.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787579+0000", 2024-04-20T01:48:49.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6380000000000002e-06 2024-04-20T01:48:49.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787586+0000", 2024-04-20T01:48:49.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.81e-06 2024-04-20T01:48:49.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787592+0000", 2024-04-20T01:48:49.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5180000000000004e-06 2024-04-20T01:48:49.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787705+0000", 2024-04-20T01:48:49.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011387000000000001 2024-04-20T01:48:49.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:49.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.787712+0000", 2024-04-20T01:48:49.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6229999999999999e-06 2024-04-20T01:48:49.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794765+0000", 2024-04-20T01:48:49.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0070530519999999998 2024-04-20T01:48:49.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:49.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.794873+0000", 2024-04-20T01:48:49.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107861 2024-04-20T01:48:49.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.795365+0000", 2024-04-20T01:48:49.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049192300000000001 2024-04-20T01:48:49.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799201+0000", 2024-04-20T01:48:49.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003836454 2024-04-20T01:48:49.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:49.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806808+0000", 2024-04-20T01:48:49.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076070119999999998 2024-04-20T01:48:49.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.806849+0000", 2024-04-20T01:48:49.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0852999999999997e-05 2024-04-20T01:48:49.237 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.237 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.237 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.237 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:443 3.0s0 3:06dd530f:::smithi16579500-30:head [append 0~720896] snapc 15=[15,14,13,b] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:49.237 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.714463+0000", 2024-04-20T01:48:49.237 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.518417599999999, 2024-04-20T01:48:49.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014549549, 2024-04-20T01:48:49.238 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.238 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:49.238 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:49.238 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:49.238 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:49.238 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 443 2024-04-20T01:48:49.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.238 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714463+0000", 2024-04-20T01:48:49.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714463+0000", 2024-04-20T01:48:49.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714461+0000", 2024-04-20T01:48:49.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:49.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714916+0000", 2024-04-20T01:48:49.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00045487499999999998 2024-04-20T01:48:49.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.240 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.240 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714934+0000", 2024-04-20T01:48:49.240 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7586000000000001e-05 2024-04-20T01:48:49.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.240 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.240 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.240 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.714962+0000", 2024-04-20T01:48:49.240 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7976000000000001e-05 2024-04-20T01:48:49.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.240 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.240 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.715010+0000", 2024-04-20T01:48:49.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8535999999999997e-05 2024-04-20T01:48:49.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:49.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.715016+0000", 2024-04-20T01:48:49.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8309999999999997e-06 2024-04-20T01:48:49.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717469+0000", 2024-04-20T01:48:49.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024527529999999998 2024-04-20T01:48:49.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:49.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717568+0000", 2024-04-20T01:48:49.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8708000000000004e-05 2024-04-20T01:48:49.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718717+0000", 2024-04-20T01:48:49.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0011493510000000001 2024-04-20T01:48:49.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724443+0000", 2024-04-20T01:48:49.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057254280000000003 2024-04-20T01:48:49.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:49.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.728929+0000", 2024-04-20T01:48:49.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044864919999999999 2024-04-20T01:48:49.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.729013+0000", 2024-04-20T01:48:49.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3770999999999995e-05 2024-04-20T01:48:49.244 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.244 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.244 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.244 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:444 3.0s0 3:06dd530f:::smithi16579500-30:head [append 0~442368] snapc 15=[15,14,13,b] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:49.244 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.716986+0000", 2024-04-20T01:48:49.244 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.515895336, 2024-04-20T01:48:49.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014745457999999999, 2024-04-20T01:48:49.244 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.244 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:49.244 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:49.244 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:49.244 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:49.245 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 444 2024-04-20T01:48:49.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.245 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.716986+0000", 2024-04-20T01:48:49.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.716986+0000", 2024-04-20T01:48:49.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.716984+0000", 2024-04-20T01:48:49.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:49.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717309+0000", 2024-04-20T01:48:49.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032479500000000001 2024-04-20T01:48:49.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717321+0000", 2024-04-20T01:48:49.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2177e-05 2024-04-20T01:48:49.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.717330+0000", 2024-04-20T01:48:49.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0399999999999998e-06 2024-04-20T01:48:49.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.720236+0000", 2024-04-20T01:48:49.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002905992 2024-04-20T01:48:49.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:49.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.720367+0000", 2024-04-20T01:48:49.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000130488 2024-04-20T01:48:49.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721328+0000", 2024-04-20T01:48:49.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00096118699999999996 2024-04-20T01:48:49.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.729125+0000", 2024-04-20T01:48:49.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077972429999999997 2024-04-20T01:48:49.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:49.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731687+0000", 2024-04-20T01:48:49.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00256201 2024-04-20T01:48:49.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731731+0000", 2024-04-20T01:48:49.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4005999999999998e-05 2024-04-20T01:48:49.249 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.249 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.249 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.249 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:445 3.0s0 3:06dd530f:::smithi16579500-30:head [append 0~327680] snapc 15=[15,14,13,b] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:49.249 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.718952+0000", 2024-04-20T01:48:49.249 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.513928671999999, 2024-04-20T01:48:49.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012930084, 2024-04-20T01:48:49.249 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.250 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:49.250 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:49.250 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:49.250 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:49.250 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 445 2024-04-20T01:48:49.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.250 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.250 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.250 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718952+0000", 2024-04-20T01:48:49.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.250 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718952+0000", 2024-04-20T01:48:49.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718950+0000", 2024-04-20T01:48:49.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:49.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719225+0000", 2024-04-20T01:48:49.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00027498300000000002 2024-04-20T01:48:49.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719241+0000", 2024-04-20T01:48:49.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6036999999999999e-05 2024-04-20T01:48:49.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719251+0000", 2024-04-20T01:48:49.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.027e-05 2024-04-20T01:48:49.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722012+0000", 2024-04-20T01:48:49.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027604309999999998 2024-04-20T01:48:49.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:49.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722121+0000", 2024-04-20T01:48:49.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109207 2024-04-20T01:48:49.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722804+0000", 2024-04-20T01:48:49.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00068250299999999999 2024-04-20T01:48:49.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.729128+0000", 2024-04-20T01:48:49.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0063243079999999998 2024-04-20T01:48:49.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:49.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731817+0000", 2024-04-20T01:48:49.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0026888239999999998 2024-04-20T01:48:49.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731882+0000", 2024-04-20T01:48:49.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5499000000000004e-05 2024-04-20T01:48:49.254 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.254 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.255 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.255 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:446 3.0s0 3:06dd530f:::smithi16579500-30:head [setxattr _header (57) in=7b] snapc 15=[15,14,13,b] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:49.255 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.719261+0000", 2024-04-20T01:48:49.255 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.513620334999999, 2024-04-20T01:48:49.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012992831, 2024-04-20T01:48:49.255 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.255 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:49.255 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:49.255 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:49.255 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:49.255 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 446 2024-04-20T01:48:49.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.256 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719261+0000", 2024-04-20T01:48:49.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719261+0000", 2024-04-20T01:48:49.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719260+0000", 2024-04-20T01:48:49.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:49.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719264+0000", 2024-04-20T01:48:49.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6179999999999997e-06 2024-04-20T01:48:49.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719271+0000", 2024-04-20T01:48:49.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9949999999999999e-06 2024-04-20T01:48:49.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719277+0000", 2024-04-20T01:48:49.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6999999999999996e-06 2024-04-20T01:48:49.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723396+0000", 2024-04-20T01:48:49.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0041186599999999997 2024-04-20T01:48:49.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:49.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723458+0000", 2024-04-20T01:48:49.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2786000000000005e-05 2024-04-20T01:48:49.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723867+0000", 2024-04-20T01:48:49.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040805300000000002 2024-04-20T01:48:49.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731913+0000", 2024-04-20T01:48:49.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080461330000000005 2024-04-20T01:48:49.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:49.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731951+0000", 2024-04-20T01:48:49.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8707000000000001e-05 2024-04-20T01:48:49.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.732253+0000", 2024-04-20T01:48:49.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030198999999999997 2024-04-20T01:48:49.260 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.260 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.260 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.260 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:447 3.0s0 3:06dd530f:::smithi16579500-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e53)", 2024-04-20T01:48:49.260 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.719282+0000", 2024-04-20T01:48:49.260 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.513598768000001, 2024-04-20T01:48:49.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01483317, 2024-04-20T01:48:49.260 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.260 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:49.260 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:49.260 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:49.261 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:49.261 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 447 2024-04-20T01:48:49.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.261 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.261 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719282+0000", 2024-04-20T01:48:49.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.261 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719282+0000", 2024-04-20T01:48:49.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719282+0000", 2024-04-20T01:48:49.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:49.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719294+0000", 2024-04-20T01:48:49.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2747e-05 2024-04-20T01:48:49.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719302+0000", 2024-04-20T01:48:49.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0960000000000001e-06 2024-04-20T01:48:49.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719308+0000", 2024-04-20T01:48:49.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.066e-06 2024-04-20T01:48:49.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724217+0000", 2024-04-20T01:48:49.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0049083549999999997 2024-04-20T01:48:49.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:49.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724333+0000", 2024-04-20T01:48:49.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011662099999999999 2024-04-20T01:48:49.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.732307+0000", 2024-04-20T01:48:49.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0079733059999999994 2024-04-20T01:48:49.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:49.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.732409+0000", 2024-04-20T01:48:49.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010193599999999999 2024-04-20T01:48:49.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.734115+0000", 2024-04-20T01:48:49.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0017065520000000001 2024-04-20T01:48:49.265 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.265 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.265 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.265 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:781 3.0s0 3:06dd530f:::smithi16579500-30:head [copy-from ver 10] snapc 2a=[2a,28] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-04-20T01:48:49.265 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.717982+0000", 2024-04-20T01:48:49.265 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 32.514899006999997, 2024-04-20T01:48:49.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.036464862000000001, 2024-04-20T01:48:49.266 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.266 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:49.266 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:49.266 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:49.266 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:49.266 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 781 2024-04-20T01:48:49.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.266 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.717982+0000", 2024-04-20T01:48:49.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.717982+0000", 2024-04-20T01:48:49.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.717980+0000", 2024-04-20T01:48:49.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:49.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.717990+0000", 2024-04-20T01:48:49.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9790000000000005e-06 2024-04-20T01:48:49.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718003+0000", 2024-04-20T01:48:49.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.342e-05 2024-04-20T01:48:49.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718014+0000", 2024-04-20T01:48:49.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0258000000000001e-05 2024-04-20T01:48:49.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718122+0000", 2024-04-20T01:48:49.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108208 2024-04-20T01:48:49.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:49.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718130+0000", 2024-04-20T01:48:49.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2120000000000003e-06 2024-04-20T01:48:49.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722371+0000", 2024-04-20T01:48:49.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042406049999999997 2024-04-20T01:48:49.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:49.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722507+0000", 2024-04-20T01:48:49.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000136136 2024-04-20T01:48:49.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.734773+0000", 2024-04-20T01:48:49.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012266325999999999 2024-04-20T01:48:49.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.742440+0000", 2024-04-20T01:48:49.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076668489999999999 2024-04-20T01:48:49.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:49.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.754352+0000", 2024-04-20T01:48:49.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011911880999999999 2024-04-20T01:48:49.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.754447+0000", 2024-04-20T01:48:49.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4827999999999994e-05 2024-04-20T01:48:49.271 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.271 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.271 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.271 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:782 3.0s0 3:06dd530f:::smithi16579500-30:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e75)", 2024-04-20T01:48:49.271 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.718032+0000", 2024-04-20T01:48:49.271 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 32.514849011000003, 2024-04-20T01:48:49.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020311961999999999, 2024-04-20T01:48:49.271 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.272 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:49.272 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:49.272 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:49.272 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:49.272 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 782 2024-04-20T01:48:49.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.272 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718032+0000", 2024-04-20T01:48:49.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718032+0000", 2024-04-20T01:48:49.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718031+0000", 2024-04-20T01:48:49.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:49.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718038+0000", 2024-04-20T01:48:49.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0990000000000001e-06 2024-04-20T01:48:49.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718047+0000", 2024-04-20T01:48:49.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2520000000000005e-06 2024-04-20T01:48:49.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.274 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718059+0000", 2024-04-20T01:48:49.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1965e-05 2024-04-20T01:48:49.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.274 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718210+0000", 2024-04-20T01:48:49.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150519 2024-04-20T01:48:49.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.274 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:49.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718216+0000", 2024-04-20T01:48:49.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3539999999999996e-06 2024-04-20T01:48:49.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722660+0000", 2024-04-20T01:48:49.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.004443687 2024-04-20T01:48:49.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:49.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722692+0000", 2024-04-20T01:48:49.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1761000000000003e-05 2024-04-20T01:48:49.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.738085+0000", 2024-04-20T01:48:49.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015393333 2024-04-20T01:48:49.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:49.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.738203+0000", 2024-04-20T01:48:49.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011827100000000001 2024-04-20T01:48:49.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.738344+0000", 2024-04-20T01:48:49.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014056600000000001 2024-04-20T01:48:49.276 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.276 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.277 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.277 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs1 75/30 ECSubWrite(tid=65, reqid=osd.2.0:65, at_version=75'50, trim_to=0'0, roll_forward_to=73'46))", 2024-04-20T01:48:49.277 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.720810+0000", 2024-04-20T01:48:49.277 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 32.512070672, 2024-04-20T01:48:49.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013232499, 2024-04-20T01:48:49.277 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.277 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.277 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720810+0000", 2024-04-20T01:48:49.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720810+0000", 2024-04-20T01:48:49.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720809+0000", 2024-04-20T01:48:49.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:49.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720840+0000", 2024-04-20T01:48:49.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0954999999999998e-05 2024-04-20T01:48:49.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720853+0000", 2024-04-20T01:48:49.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2748000000000001e-05 2024-04-20T01:48:49.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720863+0000", 2024-04-20T01:48:49.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4409999999999999e-06 2024-04-20T01:48:49.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722832+0000", 2024-04-20T01:48:49.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019689899999999999 2024-04-20T01:48:49.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722841+0000", 2024-04-20T01:48:49.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8320000000000001e-06 2024-04-20T01:48:49.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.734017+0000", 2024-04-20T01:48:49.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011175314 2024-04-20T01:48:49.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.734043+0000", 2024-04-20T01:48:49.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5962999999999999e-05 2024-04-20T01:48:49.281 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.281 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.281 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.281 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s1 75/30 ECSubWrite(tid=351, reqid=osd.13.0:351, at_version=75'121, trim_to=0'0, roll_forward_to=73'119))", 2024-04-20T01:48:49.281 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.721483+0000", 2024-04-20T01:48:49.281 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 32.511398161999999, 2024-04-20T01:48:49.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012358233, 2024-04-20T01:48:49.281 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.281 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.281 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721483+0000", 2024-04-20T01:48:49.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721483+0000", 2024-04-20T01:48:49.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721482+0000", 2024-04-20T01:48:49.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:49.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721517+0000", 2024-04-20T01:48:49.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4520999999999997e-05 2024-04-20T01:48:49.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721530+0000", 2024-04-20T01:48:49.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2928000000000001e-05 2024-04-20T01:48:49.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721540+0000", 2024-04-20T01:48:49.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0276e-05 2024-04-20T01:48:49.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721654+0000", 2024-04-20T01:48:49.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113496 2024-04-20T01:48:49.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.721665+0000", 2024-04-20T01:48:49.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1796e-05 2024-04-20T01:48:49.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.733806+0000", 2024-04-20T01:48:49.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012140656999999999 2024-04-20T01:48:49.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.733841+0000", 2024-04-20T01:48:49.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4820999999999997e-05 2024-04-20T01:48:49.285 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.285 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.285 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.285 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s1 75/30 ECSubWrite(tid=352, reqid=osd.13.0:352, at_version=75'123, trim_to=0'0, roll_forward_to=73'119))", 2024-04-20T01:48:49.285 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.723743+0000", 2024-04-20T01:48:49.285 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 32.509137500999998, 2024-04-20T01:48:49.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013134636999999999, 2024-04-20T01:48:49.285 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.285 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.285 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723743+0000", 2024-04-20T01:48:49.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723743+0000", 2024-04-20T01:48:49.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723742+0000", 2024-04-20T01:48:49.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:49.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723769+0000", 2024-04-20T01:48:49.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6086999999999999e-05 2024-04-20T01:48:49.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723782+0000", 2024-04-20T01:48:49.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3675e-05 2024-04-20T01:48:49.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723794+0000", 2024-04-20T01:48:49.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1965e-05 2024-04-20T01:48:49.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725734+0000", 2024-04-20T01:48:49.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019396439999999999 2024-04-20T01:48:49.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725743+0000", 2024-04-20T01:48:49.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8659999999999995e-06 2024-04-20T01:48:49.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736860+0000", 2024-04-20T01:48:49.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011117311 2024-04-20T01:48:49.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736878+0000", 2024-04-20T01:48:49.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7938999999999999e-05 2024-04-20T01:48:49.289 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.289 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.289 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.289 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs3 75/30 ECSubWrite(tid=112, reqid=osd.14.0:112, at_version=75'91, trim_to=0'0, roll_forward_to=74'89))", 2024-04-20T01:48:49.289 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.724406+0000", 2024-04-20T01:48:49.289 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 32.508474812999999, 2024-04-20T01:48:49.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012721514999999999, 2024-04-20T01:48:49.290 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.290 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.290 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724406+0000", 2024-04-20T01:48:49.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724406+0000", 2024-04-20T01:48:49.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724406+0000", 2024-04-20T01:48:49.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:49.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724444+0000", 2024-04-20T01:48:49.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8659999999999999e-05 2024-04-20T01:48:49.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724455+0000", 2024-04-20T01:48:49.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0468999999999999e-05 2024-04-20T01:48:49.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724469+0000", 2024-04-20T01:48:49.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4498000000000001e-05 2024-04-20T01:48:49.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724894+0000", 2024-04-20T01:48:49.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00042428799999999999 2024-04-20T01:48:49.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724905+0000", 2024-04-20T01:48:49.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1415e-05 2024-04-20T01:48:49.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.737103+0000", 2024-04-20T01:48:49.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012198293000000001 2024-04-20T01:48:49.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.737128+0000", 2024-04-20T01:48:49.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4178000000000001e-05 2024-04-20T01:48:49.293 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.293 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.293 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.294 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs3 75/30 ECSubWrite(tid=113, reqid=osd.14.0:113, at_version=75'93, trim_to=0'0, roll_forward_to=74'89))", 2024-04-20T01:48:49.294 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.724486+0000", 2024-04-20T01:48:49.294 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 32.508395120000003, 2024-04-20T01:48:49.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012679546999999999, 2024-04-20T01:48:49.294 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.294 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.294 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724486+0000", 2024-04-20T01:48:49.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724486+0000", 2024-04-20T01:48:49.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724485+0000", 2024-04-20T01:48:49.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:49.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724545+0000", 2024-04-20T01:48:49.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9892000000000002e-05 2024-04-20T01:48:49.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724556+0000", 2024-04-20T01:48:49.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0706999999999999e-05 2024-04-20T01:48:49.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724564+0000", 2024-04-20T01:48:49.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5129999999999996e-06 2024-04-20T01:48:49.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725568+0000", 2024-04-20T01:48:49.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001003547 2024-04-20T01:48:49.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725593+0000", 2024-04-20T01:48:49.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4604e-05 2024-04-20T01:48:49.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.737155+0000", 2024-04-20T01:48:49.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011562915 2024-04-20T01:48:49.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.737165+0000", 2024-04-20T01:48:49.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7909999999999999e-06 2024-04-20T01:48:49.298 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.298 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.298 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.298 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s3 75/30 ECSubWrite(tid=101, reqid=osd.5.0:101, at_version=75'82, trim_to=0'0, roll_forward_to=68'78))", 2024-04-20T01:48:49.298 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.724669+0000", 2024-04-20T01:48:49.298 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 32.508212186000002, 2024-04-20T01:48:49.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012107346, 2024-04-20T01:48:49.298 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.298 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.299 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724669+0000", 2024-04-20T01:48:49.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724669+0000", 2024-04-20T01:48:49.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724668+0000", 2024-04-20T01:48:49.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:49.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724693+0000", 2024-04-20T01:48:49.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4289000000000001e-05 2024-04-20T01:48:49.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724707+0000", 2024-04-20T01:48:49.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4489e-05 2024-04-20T01:48:49.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724715+0000", 2024-04-20T01:48:49.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.174e-06 2024-04-20T01:48:49.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725729+0000", 2024-04-20T01:48:49.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0010136870000000001 2024-04-20T01:48:49.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725739+0000", 2024-04-20T01:48:49.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0064000000000001e-05 2024-04-20T01:48:49.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736749+0000", 2024-04-20T01:48:49.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011009778 2024-04-20T01:48:49.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736776+0000", 2024-04-20T01:48:49.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7220999999999999e-05 2024-04-20T01:48:49.302 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.302 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.302 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.302 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs3 80/30 ECSubWrite(tid=132, reqid=osd.14.0:132, at_version=80'111, trim_to=0'0, roll_forward_to=77'109))", 2024-04-20T01:48:49.302 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:21.729572+0000", 2024-04-20T01:48:49.302 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.503308595, 2024-04-20T01:48:49.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020941090999999998, 2024-04-20T01:48:49.303 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.303 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.303 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729572+0000", 2024-04-20T01:48:49.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729572+0000", 2024-04-20T01:48:49.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729572+0000", 2024-04-20T01:48:49.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:49.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729602+0000", 2024-04-20T01:48:49.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0369e-05 2024-04-20T01:48:49.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729615+0000", 2024-04-20T01:48:49.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3158e-05 2024-04-20T01:48:49.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729626+0000", 2024-04-20T01:48:49.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0056e-05 2024-04-20T01:48:49.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729699+0000", 2024-04-20T01:48:49.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3857999999999997e-05 2024-04-20T01:48:49.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.729712+0000", 2024-04-20T01:48:49.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3088000000000001e-05 2024-04-20T01:48:49.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.750484+0000", 2024-04-20T01:48:49.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020771511999999999 2024-04-20T01:48:49.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.750513+0000", 2024-04-20T01:48:49.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9363e-05 2024-04-20T01:48:49.306 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.306 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.306 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.307 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.es5 80/30 ECSubWrite(tid=112, reqid=osd.11.0:112, at_version=80'88, trim_to=0'0, roll_forward_to=77'86))", 2024-04-20T01:48:49.307 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:21.733917+0000", 2024-04-20T01:48:49.307 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.498963691, 2024-04-20T01:48:49.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015930045, 2024-04-20T01:48:49.307 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.307 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.307 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.733917+0000", 2024-04-20T01:48:49.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.733917+0000", 2024-04-20T01:48:49.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.733917+0000", 2024-04-20T01:48:49.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:49.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.733943+0000", 2024-04-20T01:48:49.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6764e-05 2024-04-20T01:48:49.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.733956+0000", 2024-04-20T01:48:49.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2537999999999999e-05 2024-04-20T01:48:49.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.733966+0000", 2024-04-20T01:48:49.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0145e-05 2024-04-20T01:48:49.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.734023+0000", 2024-04-20T01:48:49.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7231000000000001e-05 2024-04-20T01:48:49.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.734035+0000", 2024-04-20T01:48:49.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1163e-05 2024-04-20T01:48:49.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.749818+0000", 2024-04-20T01:48:49.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015783452 2024-04-20T01:48:49.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.749847+0000", 2024-04-20T01:48:49.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9235e-05 2024-04-20T01:48:49.310 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.310 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.310 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.311 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s5 80/57 ECSubWrite(tid=205, reqid=osd.8.0:205, at_version=80'147, trim_to=0'0, roll_forward_to=79'145))", 2024-04-20T01:48:49.311 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:21.737407+0000", 2024-04-20T01:48:49.311 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.495474077000001, 2024-04-20T01:48:49.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013126919000000001, 2024-04-20T01:48:49.311 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.311 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.311 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737407+0000", 2024-04-20T01:48:49.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737407+0000", 2024-04-20T01:48:49.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737406+0000", 2024-04-20T01:48:49.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:49.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737434+0000", 2024-04-20T01:48:49.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7623999999999998e-05 2024-04-20T01:48:49.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737448+0000", 2024-04-20T01:48:49.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3811e-05 2024-04-20T01:48:49.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737458+0000", 2024-04-20T01:48:49.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0426e-05 2024-04-20T01:48:49.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737568+0000", 2024-04-20T01:48:49.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109221 2024-04-20T01:48:49.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.737579+0000", 2024-04-20T01:48:49.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1895e-05 2024-04-20T01:48:49.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.750521+0000", 2024-04-20T01:48:49.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012941998 2024-04-20T01:48:49.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.750534+0000", 2024-04-20T01:48:49.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2267e-05 2024-04-20T01:48:49.315 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.315 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.315 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.315 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s5 80/57 ECSubWrite(tid=206, reqid=osd.8.0:206, at_version=80'149, trim_to=0'0, roll_forward_to=79'145))", 2024-04-20T01:48:49.315 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:21.738507+0000", 2024-04-20T01:48:49.315 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.494373876000001, 2024-04-20T01:48:49.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014548627999999999, 2024-04-20T01:48:49.315 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.315 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.315 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.738507+0000", 2024-04-20T01:48:49.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.738507+0000", 2024-04-20T01:48:49.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.738506+0000", 2024-04-20T01:48:49.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:49.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.738540+0000", 2024-04-20T01:48:49.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4316000000000001e-05 2024-04-20T01:48:49.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.738553+0000", 2024-04-20T01:48:49.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2755000000000001e-05 2024-04-20T01:48:49.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.738564+0000", 2024-04-20T01:48:49.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.066e-05 2024-04-20T01:48:49.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.739602+0000", 2024-04-20T01:48:49.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001038186 2024-04-20T01:48:49.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.739612+0000", 2024-04-20T01:48:49.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0145e-05 2024-04-20T01:48:49.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.753027+0000", 2024-04-20T01:48:49.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013415152 2024-04-20T01:48:49.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:21.753056+0000", 2024-04-20T01:48:49.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8393000000000001e-05 2024-04-20T01:48:49.319 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.319 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.319 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.319 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.319 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.319 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:49.321 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-20T01:48:49.449 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:49.450 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:49.450 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-04-20T01:48:49.450 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:49.459 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-20T01:48:49.589 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:49.589 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:49.589 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:48:49.589 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:48:49.589 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:49.599 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-20T01:48:49.729 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:49.730 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:48:49.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:48:49.730 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:48:49.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.730 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:289 3.9s0 3:94399c02:::smithi16579500-3:head [copy-from ver 6] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:49.730 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:43.807269+0000", 2024-04-20T01:48:49.730 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 65.936020948000007, 2024-04-20T01:48:49.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016170202000000002, 2024-04-20T01:48:49.730 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.730 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:49.730 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:49.731 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:49.731 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:49.731 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 289 2024-04-20T01:48:49.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.731 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807269+0000", 2024-04-20T01:48:49.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807269+0000", 2024-04-20T01:48:49.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807267+0000", 2024-04-20T01:48:49.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:49.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807280+0000", 2024-04-20T01:48:49.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3227999999999999e-05 2024-04-20T01:48:49.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807303+0000", 2024-04-20T01:48:49.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2991000000000001e-05 2024-04-20T01:48:49.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807318+0000", 2024-04-20T01:48:49.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4321999999999999e-05 2024-04-20T01:48:49.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807385+0000", 2024-04-20T01:48:49.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7403000000000005e-05 2024-04-20T01:48:49.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:49.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807394+0000", 2024-04-20T01:48:49.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7010000000000002e-06 2024-04-20T01:48:49.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.809246+0000", 2024-04-20T01:48:49.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0018521 2024-04-20T01:48:49.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:49.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.809314+0000", 2024-04-20T01:48:49.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7818999999999996e-05 2024-04-20T01:48:49.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.817661+0000", 2024-04-20T01:48:49.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0083471710000000005 2024-04-20T01:48:49.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.819085+0000", 2024-04-20T01:48:49.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0014239739999999999 2024-04-20T01:48:49.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:49.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.823350+0000", 2024-04-20T01:48:49.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042652990000000002 2024-04-20T01:48:49.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.823439+0000", 2024-04-20T01:48:49.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9424000000000003e-05 2024-04-20T01:48:49.736 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.736 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.736 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.736 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:290 3.9s0 3:94399c02:::smithi16579500-3:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-04-20T01:48:49.736 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:43.807329+0000", 2024-04-20T01:48:49.736 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 65.935961027000005, 2024-04-20T01:48:49.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011459461000000001, 2024-04-20T01:48:49.736 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.736 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:49.736 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:49.737 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:49.737 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:49.737 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 290 2024-04-20T01:48:49.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.737 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807329+0000", 2024-04-20T01:48:49.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807329+0000", 2024-04-20T01:48:49.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807329+0000", 2024-04-20T01:48:49.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:49.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807332+0000", 2024-04-20T01:48:49.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.349e-06 2024-04-20T01:48:49.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807343+0000", 2024-04-20T01:48:49.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1270999999999999e-05 2024-04-20T01:48:49.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807347+0000", 2024-04-20T01:48:49.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-04-20T01:48:49.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807453+0000", 2024-04-20T01:48:49.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106162 2024-04-20T01:48:49.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:49.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.807460+0000", 2024-04-20T01:48:49.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9809999999999997e-06 2024-04-20T01:48:49.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.809441+0000", 2024-04-20T01:48:49.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019806060000000002 2024-04-20T01:48:49.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:49.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.809484+0000", 2024-04-20T01:48:49.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3019000000000003e-05 2024-04-20T01:48:49.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.818614+0000", 2024-04-20T01:48:49.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091302699999999994 2024-04-20T01:48:49.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:49.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.818713+0000", 2024-04-20T01:48:49.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8776000000000004e-05 2024-04-20T01:48:49.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:43.818789+0000", 2024-04-20T01:48:49.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5944000000000003e-05 2024-04-20T01:48:49.741 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.741 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.741 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.741 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 47/30 ECSubWrite(tid=38, reqid=osd.14.0:38, at_version=47'31, trim_to=0'0, roll_forward_to=46'29))", 2024-04-20T01:48:49.742 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.711869+0000", 2024-04-20T01:48:49.742 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 61.031421139000003, 2024-04-20T01:48:49.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016896644999999998, 2024-04-20T01:48:49.742 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.742 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.742 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711869+0000", 2024-04-20T01:48:49.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711869+0000", 2024-04-20T01:48:49.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711868+0000", 2024-04-20T01:48:49.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:49.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711907+0000", 2024-04-20T01:48:49.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8828000000000002e-05 2024-04-20T01:48:49.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711927+0000", 2024-04-20T01:48:49.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9656e-05 2024-04-20T01:48:49.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711937+0000", 2024-04-20T01:48:49.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0277e-05 2024-04-20T01:48:49.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.711991+0000", 2024-04-20T01:48:49.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3980000000000002e-05 2024-04-20T01:48:49.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.712002+0000", 2024-04-20T01:48:49.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719e-05 2024-04-20T01:48:49.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.728747+0000", 2024-04-20T01:48:49.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016745466000000001 2024-04-20T01:48:49.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.728766+0000", 2024-04-20T01:48:49.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8607999999999999e-05 2024-04-20T01:48:49.745 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.745 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.745 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.746 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 47/30 ECSubWrite(tid=39, reqid=osd.14.0:39, at_version=47'33, trim_to=0'0, roll_forward_to=46'29))", 2024-04-20T01:48:49.746 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.713021+0000", 2024-04-20T01:48:49.746 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 61.030268931999998, 2024-04-20T01:48:49.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020945089, 2024-04-20T01:48:49.746 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.746 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.746 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713021+0000", 2024-04-20T01:48:49.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713021+0000", 2024-04-20T01:48:49.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713020+0000", 2024-04-20T01:48:49.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:49.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713046+0000", 2024-04-20T01:48:49.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5554e-05 2024-04-20T01:48:49.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713064+0000", 2024-04-20T01:48:49.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7635999999999999e-05 2024-04-20T01:48:49.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713074+0000", 2024-04-20T01:48:49.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9529999999999994e-06 2024-04-20T01:48:49.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713140+0000", 2024-04-20T01:48:49.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6235999999999994e-05 2024-04-20T01:48:49.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713155+0000", 2024-04-20T01:48:49.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5647000000000001e-05 2024-04-20T01:48:49.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733953+0000", 2024-04-20T01:48:49.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020797936 2024-04-20T01:48:49.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733966+0000", 2024-04-20T01:48:49.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3067999999999999e-05 2024-04-20T01:48:49.750 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.750 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.750 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.750 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s1 47/30 ECSubWrite(tid=30, reqid=osd.3.0:30, at_version=47'19, trim_to=0'0, roll_forward_to=44'17))", 2024-04-20T01:48:49.750 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.713663+0000", 2024-04-20T01:48:49.750 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 61.029627171999998, 2024-04-20T01:48:49.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0202712, 2024-04-20T01:48:49.750 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.750 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.750 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713663+0000", 2024-04-20T01:48:49.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713663+0000", 2024-04-20T01:48:49.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713662+0000", 2024-04-20T01:48:49.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:49.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713693+0000", 2024-04-20T01:48:49.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1031999999999997e-05 2024-04-20T01:48:49.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713712+0000", 2024-04-20T01:48:49.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8615999999999999e-05 2024-04-20T01:48:49.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713726+0000", 2024-04-20T01:48:49.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4477e-05 2024-04-20T01:48:49.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713779+0000", 2024-04-20T01:48:49.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3090999999999997e-05 2024-04-20T01:48:49.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.713791+0000", 2024-04-20T01:48:49.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1921e-05 2024-04-20T01:48:49.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733906+0000", 2024-04-20T01:48:49.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020114522999999999 2024-04-20T01:48:49.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733934+0000", 2024-04-20T01:48:49.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8357999999999999e-05 2024-04-20T01:48:49.754 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.754 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.754 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.754 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 47/30 ECSubWrite(tid=39, reqid=osd.5.0:39, at_version=47'27, trim_to=0'0, roll_forward_to=46'25))", 2024-04-20T01:48:49.754 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.718432+0000", 2024-04-20T01:48:49.754 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 61.024858514999998, 2024-04-20T01:48:49.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015468671, 2024-04-20T01:48:49.754 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.754 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.755 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718432+0000", 2024-04-20T01:48:49.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718432+0000", 2024-04-20T01:48:49.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718431+0000", 2024-04-20T01:48:49.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:49.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718455+0000", 2024-04-20T01:48:49.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3085e-05 2024-04-20T01:48:49.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718471+0000", 2024-04-20T01:48:49.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6172999999999999e-05 2024-04-20T01:48:49.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718480+0000", 2024-04-20T01:48:49.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5719999999999992e-06 2024-04-20T01:48:49.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718592+0000", 2024-04-20T01:48:49.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111517 2024-04-20T01:48:49.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.718602+0000", 2024-04-20T01:48:49.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.064e-05 2024-04-20T01:48:49.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733869+0000", 2024-04-20T01:48:49.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015266798 2024-04-20T01:48:49.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733900+0000", 2024-04-20T01:48:49.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.116e-05 2024-04-20T01:48:49.758 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.758 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.758 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.758 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 47/30 ECSubWrite(tid=40, reqid=osd.5.0:40, at_version=47'29, trim_to=0'0, roll_forward_to=46'25))", 2024-04-20T01:48:49.758 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.719277+0000", 2024-04-20T01:48:49.758 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 61.024012818999999, 2024-04-20T01:48:49.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017279452000000001, 2024-04-20T01:48:49.759 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.759 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.759 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719277+0000", 2024-04-20T01:48:49.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719277+0000", 2024-04-20T01:48:49.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719277+0000", 2024-04-20T01:48:49.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:49.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719306+0000", 2024-04-20T01:48:49.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9771999999999999e-05 2024-04-20T01:48:49.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719322+0000", 2024-04-20T01:48:49.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6035000000000001e-05 2024-04-20T01:48:49.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.719331+0000", 2024-04-20T01:48:49.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1130000000000007e-06 2024-04-20T01:48:49.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.728810+0000", 2024-04-20T01:48:49.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094789090000000006 2024-04-20T01:48:49.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.728819+0000", 2024-04-20T01:48:49.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8890000000000008e-06 2024-04-20T01:48:49.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.736530+0000", 2024-04-20T01:48:49.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077106129999999998 2024-04-20T01:48:49.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.736557+0000", 2024-04-20T01:48:49.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6974999999999998e-05 2024-04-20T01:48:49.762 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.762 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.762 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.762 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 47/30 ECSubWrite(tid=240, reqid=osd.13.0:240, at_version=47'33, trim_to=0'0, roll_forward_to=44'31))", 2024-04-20T01:48:49.762 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.722777+0000", 2024-04-20T01:48:49.762 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 61.020513104999999, 2024-04-20T01:48:49.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008649894, 2024-04-20T01:48:49.763 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.763 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.763 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722777+0000", 2024-04-20T01:48:49.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722777+0000", 2024-04-20T01:48:49.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722777+0000", 2024-04-20T01:48:49.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:49.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722805+0000", 2024-04-20T01:48:49.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7881e-05 2024-04-20T01:48:49.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722820+0000", 2024-04-20T01:48:49.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5387e-05 2024-04-20T01:48:49.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722831+0000", 2024-04-20T01:48:49.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813e-05 2024-04-20T01:48:49.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722915+0000", 2024-04-20T01:48:49.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3578000000000005e-05 2024-04-20T01:48:49.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.722929+0000", 2024-04-20T01:48:49.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4256999999999999e-05 2024-04-20T01:48:49.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.731402+0000", 2024-04-20T01:48:49.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008472785 2024-04-20T01:48:49.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.731427+0000", 2024-04-20T01:48:49.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5355e-05 2024-04-20T01:48:49.766 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.766 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.766 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.766 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 47/30 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=47'35, trim_to=0'0, roll_forward_to=44'31))", 2024-04-20T01:48:49.767 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.723066+0000", 2024-04-20T01:48:49.767 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 61.020224358999997, 2024-04-20T01:48:49.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010874728, 2024-04-20T01:48:49.767 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.767 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.767 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723066+0000", 2024-04-20T01:48:49.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723066+0000", 2024-04-20T01:48:49.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723065+0000", 2024-04-20T01:48:49.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:49.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723089+0000", 2024-04-20T01:48:49.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3529000000000001e-05 2024-04-20T01:48:49.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723101+0000", 2024-04-20T01:48:49.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2143e-05 2024-04-20T01:48:49.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.723110+0000", 2024-04-20T01:48:49.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9809999999999992e-06 2024-04-20T01:48:49.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.725642+0000", 2024-04-20T01:48:49.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002532337 2024-04-20T01:48:49.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.725655+0000", 2024-04-20T01:48:49.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2707e-05 2024-04-20T01:48:49.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733911+0000", 2024-04-20T01:48:49.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082559369999999997 2024-04-20T01:48:49.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.733941+0000", 2024-04-20T01:48:49.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9798000000000002e-05 2024-04-20T01:48:49.770 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.770 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.770 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.771 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:396 3.9s0 3:9a2fbd15:::smithi16579500-1:head [copy-from ver 32] snapc d=[c,b] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-20T01:48:49.771 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:48.746088+0000", 2024-04-20T01:48:49.771 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 60.997202180999999, 2024-04-20T01:48:49.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01355685, 2024-04-20T01:48:49.771 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.771 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:49.771 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:49.771 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:49.771 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:49.771 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 396 2024-04-20T01:48:49.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.771 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746088+0000", 2024-04-20T01:48:49.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746088+0000", 2024-04-20T01:48:49.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746086+0000", 2024-04-20T01:48:49.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:49.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746094+0000", 2024-04-20T01:48:49.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.272e-06 2024-04-20T01:48:49.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746109+0000", 2024-04-20T01:48:49.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5563999999999999e-05 2024-04-20T01:48:49.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746131+0000", 2024-04-20T01:48:49.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1515999999999999e-05 2024-04-20T01:48:49.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746178+0000", 2024-04-20T01:48:49.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7472000000000002e-05 2024-04-20T01:48:49.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:49.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.746261+0000", 2024-04-20T01:48:49.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2412999999999999e-05 2024-04-20T01:48:49.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.752312+0000", 2024-04-20T01:48:49.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0060515120000000002 2024-04-20T01:48:49.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.754299+0000", 2024-04-20T01:48:49.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019870690000000002 2024-04-20T01:48:49.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:49.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.759594+0000", 2024-04-20T01:48:49.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0052944070000000001 2024-04-20T01:48:49.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:48.759645+0000", 2024-04-20T01:48:49.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1236000000000002e-05 2024-04-20T01:48:49.776 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.776 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.776 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.776 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 48/30 ECSubWrite(tid=44, reqid=osd.5.0:44, at_version=48'33, trim_to=0'0, roll_forward_to=47'31))", 2024-04-20T01:48:49.777 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.716311+0000", 2024-04-20T01:48:49.777 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 60.026978776, 2024-04-20T01:48:49.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093701210000000004, 2024-04-20T01:48:49.777 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.777 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.777 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.777 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716311+0000", 2024-04-20T01:48:49.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.777 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716311+0000", 2024-04-20T01:48:49.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716311+0000", 2024-04-20T01:48:49.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:49.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716336+0000", 2024-04-20T01:48:49.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5216e-05 2024-04-20T01:48:49.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716348+0000", 2024-04-20T01:48:49.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1369000000000001e-05 2024-04-20T01:48:49.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716358+0000", 2024-04-20T01:48:49.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9019999999999994e-06 2024-04-20T01:48:49.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716410+0000", 2024-04-20T01:48:49.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1974000000000001e-05 2024-04-20T01:48:49.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.716419+0000", 2024-04-20T01:48:49.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4439999999999997e-06 2024-04-20T01:48:49.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.725652+0000", 2024-04-20T01:48:49.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092323909999999995 2024-04-20T01:48:49.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.725682+0000", 2024-04-20T01:48:49.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0049999999999999e-05 2024-04-20T01:48:49.780 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.780 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.780 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.781 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 48/30 ECSubWrite(tid=256, reqid=client.4643.0:403, at_version=48'46, trim_to=0'0, roll_forward_to=48'44))", 2024-04-20T01:48:49.781 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.789372+0000", 2024-04-20T01:48:49.781 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.953918199999997, 2024-04-20T01:48:49.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086922490000000008, 2024-04-20T01:48:49.781 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.781 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.781 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789372+0000", 2024-04-20T01:48:49.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789372+0000", 2024-04-20T01:48:49.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789371+0000", 2024-04-20T01:48:49.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:49.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789968+0000", 2024-04-20T01:48:49.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00059678300000000004 2024-04-20T01:48:49.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789979+0000", 2024-04-20T01:48:49.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1499e-05 2024-04-20T01:48:49.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.789986+0000", 2024-04-20T01:48:49.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2179999999999996e-06 2024-04-20T01:48:49.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.790029+0000", 2024-04-20T01:48:49.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2876000000000001e-05 2024-04-20T01:48:49.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.790036+0000", 2024-04-20T01:48:49.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7870000000000002e-06 2024-04-20T01:48:49.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.798040+0000", 2024-04-20T01:48:49.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080038720000000004 2024-04-20T01:48:49.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.798064+0000", 2024-04-20T01:48:49.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4037999999999999e-05 2024-04-20T01:48:49.786 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.786 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.787 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.787 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.2s6 48/30 ECSubWrite(tid=257, reqid=client.4643.0:404, at_version=48'47, trim_to=0'0, roll_forward_to=48'44))", 2024-04-20T01:48:49.787 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:49.791622+0000", 2024-04-20T01:48:49.787 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.951668214000001, 2024-04-20T01:48:49.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0088453630000000002, 2024-04-20T01:48:49.787 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.787 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.787 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.791622+0000", 2024-04-20T01:48:49.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.791622+0000", 2024-04-20T01:48:49.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.791621+0000", 2024-04-20T01:48:49.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:49.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792215+0000", 2024-04-20T01:48:49.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000593689 2024-04-20T01:48:49.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792227+0000", 2024-04-20T01:48:49.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2037e-05 2024-04-20T01:48:49.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792236+0000", 2024-04-20T01:48:49.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8240000000000006e-06 2024-04-20T01:48:49.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792538+0000", 2024-04-20T01:48:49.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000301668 2024-04-20T01:48:49.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.792549+0000", 2024-04-20T01:48:49.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0915e-05 2024-04-20T01:48:49.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.800443+0000", 2024-04-20T01:48:49.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0078945300000000003 2024-04-20T01:48:49.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:49.800467+0000", 2024-04-20T01:48:49.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4309000000000001e-05 2024-04-20T01:48:49.790 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.790 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.791 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.791 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s1 53/30 ECSubWrite(tid=33, reqid=client.4643.0:444, at_version=53'22, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:49.791 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.721415+0000", 2024-04-20T01:48:49.791 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 55.021875436999998, 2024-04-20T01:48:49.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097484879999999996, 2024-04-20T01:48:49.791 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.791 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.791 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721415+0000", 2024-04-20T01:48:49.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721415+0000", 2024-04-20T01:48:49.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721413+0000", 2024-04-20T01:48:49.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999986 2024-04-20T01:48:49.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721522+0000", 2024-04-20T01:48:49.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108445 2024-04-20T01:48:49.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721540+0000", 2024-04-20T01:48:49.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8128e-05 2024-04-20T01:48:49.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721555+0000", 2024-04-20T01:48:49.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5228999999999999e-05 2024-04-20T01:48:49.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721639+0000", 2024-04-20T01:48:49.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3803999999999999e-05 2024-04-20T01:48:49.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721650+0000", 2024-04-20T01:48:49.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0702e-05 2024-04-20T01:48:49.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731131+0000", 2024-04-20T01:48:49.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094816929999999994 2024-04-20T01:48:49.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731163+0000", 2024-04-20T01:48:49.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1946999999999998e-05 2024-04-20T01:48:49.794 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.794 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.795 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.795 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 58/30 ECSubWrite(tid=52, reqid=client.4643.0:489, at_version=58'41, trim_to=0'0, roll_forward_to=55'37))", 2024-04-20T01:48:49.795 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:59.769343+0000", 2024-04-20T01:48:49.795 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.973947099, 2024-04-20T01:48:49.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094380780000000008, 2024-04-20T01:48:49.795 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.795 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.795 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769343+0000", 2024-04-20T01:48:49.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769343+0000", 2024-04-20T01:48:49.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769343+0000", 2024-04-20T01:48:49.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:49.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769462+0000", 2024-04-20T01:48:49.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011929 2024-04-20T01:48:49.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769474+0000", 2024-04-20T01:48:49.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1857e-05 2024-04-20T01:48:49.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769482+0000", 2024-04-20T01:48:49.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0809999999999994e-06 2024-04-20T01:48:49.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769531+0000", 2024-04-20T01:48:49.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9168000000000001e-05 2024-04-20T01:48:49.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.769538+0000", 2024-04-20T01:48:49.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5930000000000002e-06 2024-04-20T01:48:49.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.778749+0000", 2024-04-20T01:48:49.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092115670000000004 2024-04-20T01:48:49.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.778781+0000", 2024-04-20T01:48:49.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2140000000000001e-05 2024-04-20T01:48:49.798 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.798 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.798 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.799 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:733 3.9s0 3:94399c02:::smithi16579500-3:head [rmxattr Pp[amim]nQkdfb][ca]Rhonh_mr`cpapUabYUfb\\ in=40b,rmxattr QP`igcZgYZ`X]j_`ObrUNcR_gQZW^mQs_T`XhhQp in=40b,rmxattr QXqRngrniZSrdNdQaffOsm_g_pYQoedrXoNqT[mn in=40b,rmxattr V[thp]nUrl^Xata^al[oh[d`ddg`p[siljQWZ^\\X in=40b,rmxattr XgcaedbaNamUmoa^d[dhg[arrjeikfUlToSjsbY` in=40b,rmxattr ZcXlR_O_^fXfTQrt`cZfOgVlmhepod\\^\\`socsW` in=40b,rmxattr ][ifNPWgaZd[^RnfpRhWbfS\\^a]m[\\[ienetlUUZ in=40b,rmxattr `U`PSXddYml`lYXX[sSNnPfg`VgWbXsYnftUQ`it in=40b,rmxattr aR in=2b,rmxattr eNsVcjeRcj\\m]Q[VhYdSqNcmPhhR[UPR^ORSk_Vd in=40b,rmxattr eRqQZNtlPRVjcgdjmbSmOsUSjSlU]tSoVjr`^oRN in=40b,rmxattr fVRh^qlag^rjpdaqVk`^r]eR`P[XUlr]]agZjWq[ in=40b,rmxattr frZXQX\\isoq]gmjs`rk]PiQ\\fV[RTNPNYN]YV[mq in=40b,rmxattr h\\bUdlZ]kknhkaaYeTcclaXatghTgbVrhNQblto_ in=40b,rmxattr hgq\\iqsggNPTRPQok in=17b,rmxattr jecnmafmfg__r^\\S_sU]bi]kYaplTcttSSX`\\prR in=40b,rmxattr jmliR in=5b,rmxattr mnhUacnNajpOrNecmPZT[OUj[XaWmU in=30b,rmxattr peUYXiNb_UsOaXnhlX_s_[QTW_SoUaN]Wal`rkao in=40b,rmxattr rZrXsckW]Y`iWQiRn in=17b] snapc 26=[26,25,1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-20T01:48:49.799 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:11.823678+0000", 2024-04-20T01:48:49.799 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.919612733000001, 2024-04-20T01:48:49.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0098521919999999992, 2024-04-20T01:48:49.799 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.799 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:49.799 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:49.799 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:49.799 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:49.799 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 733 2024-04-20T01:48:49.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.799 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823678+0000", 2024-04-20T01:48:49.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823678+0000", 2024-04-20T01:48:49.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823675+0000", 2024-04-20T01:48:49.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:49.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823691+0000", 2024-04-20T01:48:49.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5954999999999999e-05 2024-04-20T01:48:49.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823715+0000", 2024-04-20T01:48:49.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4113000000000001e-05 2024-04-20T01:48:49.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823730+0000", 2024-04-20T01:48:49.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4694e-05 2024-04-20T01:48:49.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823786+0000", 2024-04-20T01:48:49.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6419000000000003e-05 2024-04-20T01:48:49.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:49.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823913+0000", 2024-04-20T01:48:49.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012677500000000001 2024-04-20T01:48:49.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.824989+0000", 2024-04-20T01:48:49.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0010762199999999999 2024-04-20T01:48:49.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.828014+0000", 2024-04-20T01:48:49.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0030246190000000001 2024-04-20T01:48:49.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:49.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.833442+0000", 2024-04-20T01:48:49.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0054285490000000004 2024-04-20T01:48:49.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.833530+0000", 2024-04-20T01:48:49.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7462999999999999e-05 2024-04-20T01:48:49.803 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.803 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.803 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.804 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 75/30 ECSubWrite(tid=100, reqid=osd.5.0:100, at_version=75'80, trim_to=0'0, roll_forward_to=68'78))", 2024-04-20T01:48:49.804 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.720432+0000", 2024-04-20T01:48:49.804 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 33.022857987000002, 2024-04-20T01:48:49.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010779983, 2024-04-20T01:48:49.804 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.804 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.804 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720432+0000", 2024-04-20T01:48:49.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720432+0000", 2024-04-20T01:48:49.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720432+0000", 2024-04-20T01:48:49.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:49.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720457+0000", 2024-04-20T01:48:49.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5272000000000001e-05 2024-04-20T01:48:49.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720469+0000", 2024-04-20T01:48:49.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2193e-05 2024-04-20T01:48:49.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720480+0000", 2024-04-20T01:48:49.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0291e-05 2024-04-20T01:48:49.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723263+0000", 2024-04-20T01:48:49.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002783003 2024-04-20T01:48:49.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723273+0000", 2024-04-20T01:48:49.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.01e-05 2024-04-20T01:48:49.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731180+0000", 2024-04-20T01:48:49.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0079069929999999993 2024-04-20T01:48:49.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731212+0000", 2024-04-20T01:48:49.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2421e-05 2024-04-20T01:48:49.807 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.807 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.807 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.808 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 75/30 ECSubWrite(tid=112, reqid=osd.14.0:112, at_version=75'91, trim_to=0'0, roll_forward_to=74'89))", 2024-04-20T01:48:49.808 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.720882+0000", 2024-04-20T01:48:49.808 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 33.022408458000001, 2024-04-20T01:48:49.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010362481999999999, 2024-04-20T01:48:49.808 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.808 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.808 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720882+0000", 2024-04-20T01:48:49.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720882+0000", 2024-04-20T01:48:49.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.809 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720881+0000", 2024-04-20T01:48:49.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:49.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.809 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720905+0000", 2024-04-20T01:48:49.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3683e-05 2024-04-20T01:48:49.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.809 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720917+0000", 2024-04-20T01:48:49.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1651000000000001e-05 2024-04-20T01:48:49.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720926+0000", 2024-04-20T01:48:49.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2040000000000003e-06 2024-04-20T01:48:49.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724280+0000", 2024-04-20T01:48:49.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003354363 2024-04-20T01:48:49.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724291+0000", 2024-04-20T01:48:49.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1004e-05 2024-04-20T01:48:49.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731213+0000", 2024-04-20T01:48:49.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069213670000000003 2024-04-20T01:48:49.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731244+0000", 2024-04-20T01:48:49.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1563000000000003e-05 2024-04-20T01:48:49.811 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.811 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.811 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.812 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.8s6 75/30 ECSubWrite(tid=101, reqid=osd.5.0:101, at_version=75'82, trim_to=0'0, roll_forward_to=68'78))", 2024-04-20T01:48:49.812 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.723236+0000", 2024-04-20T01:48:49.812 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 33.020054086000002, 2024-04-20T01:48:49.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010789544, 2024-04-20T01:48:49.812 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.812 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.812 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723236+0000", 2024-04-20T01:48:49.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723236+0000", 2024-04-20T01:48:49.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723235+0000", 2024-04-20T01:48:49.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:49.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723261+0000", 2024-04-20T01:48:49.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5381e-05 2024-04-20T01:48:49.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723279+0000", 2024-04-20T01:48:49.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7796e-05 2024-04-20T01:48:49.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.723290+0000", 2024-04-20T01:48:49.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1739000000000001e-05 2024-04-20T01:48:49.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726374+0000", 2024-04-20T01:48:49.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003084023 2024-04-20T01:48:49.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726389+0000", 2024-04-20T01:48:49.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5041e-05 2024-04-20T01:48:49.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.733997+0000", 2024-04-20T01:48:49.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076079329999999999 2024-04-20T01:48:49.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.734026+0000", 2024-04-20T01:48:49.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8394e-05 2024-04-20T01:48:49.815 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.815 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.815 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.816 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.cs4 75/30 ECSubWrite(tid=113, reqid=osd.14.0:113, at_version=75'93, trim_to=0'0, roll_forward_to=74'89))", 2024-04-20T01:48:49.816 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.724246+0000", 2024-04-20T01:48:49.816 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 33.019044242, 2024-04-20T01:48:49.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097217950000000001, 2024-04-20T01:48:49.816 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:49.816 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:49.816 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:49.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:49.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724246+0000", 2024-04-20T01:48:49.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:49.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724246+0000", 2024-04-20T01:48:49.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:49.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:49.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724245+0000", 2024-04-20T01:48:49.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:49.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:49.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724270+0000", 2024-04-20T01:48:49.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4536999999999999e-05 2024-04-20T01:48:49.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:49.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724284+0000", 2024-04-20T01:48:49.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4421e-05 2024-04-20T01:48:49.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:49.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.724292+0000", 2024-04-20T01:48:49.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6009999999999997e-06 2024-04-20T01:48:49.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:49.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726433+0000", 2024-04-20T01:48:49.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021418399999999999 2024-04-20T01:48:49.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:49.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726444+0000", 2024-04-20T01:48:49.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0671e-05 2024-04-20T01:48:49.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:49.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.733939+0000", 2024-04-20T01:48:49.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0074953249999999997 2024-04-20T01:48:49.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:49.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:49.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:49.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.733968+0000", 2024-04-20T01:48:49.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8368e-05 2024-04-20T01:48:49.819 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.819 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.819 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.819 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:49.819 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:49.819 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:49.821 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-20T01:48:49.950 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:49.950 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:49.950 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-04-20T01:48:49.950 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:49.960 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-20T01:48:50.088 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:50.088 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:50.088 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:48:50.088 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:48:50.088 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:50.098 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-20T01:48:50.229 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:48:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:48:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:48:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:104 3.es0 3:78f9000f:::smithi16579500-35:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:48:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 75.280133006, 2024-04-20T01:48:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012120811, 2024-04-20T01:48:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 104 2024-04-20T01:48:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:48:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:48:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962442+0000", 2024-04-20T01:48:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962445+0000", 2024-04-20T01:48:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3210000000000001e-06 2024-04-20T01:48:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962451+0000", 2024-04-20T01:48:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.31e-06 2024-04-20T01:48:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962457+0000", 2024-04-20T01:48:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4819999999999999e-06 2024-04-20T01:48:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966434+0000", 2024-04-20T01:48:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039772239999999997 2024-04-20T01:48:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966501+0000", 2024-04-20T01:48:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6947999999999997e-05 2024-04-20T01:48:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966766+0000", 2024-04-20T01:48:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00026468999999999999 2024-04-20T01:48:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970614+0000", 2024-04-20T01:48:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0038481330000000001 2024-04-20T01:48:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974486+0000", 2024-04-20T01:48:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003871799 2024-04-20T01:48:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974563+0000", 2024-04-20T01:48:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7416999999999994e-05 2024-04-20T01:48:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:105 3.es0 3:78f9000f:::smithi16579500-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962461+0000", 2024-04-20T01:48:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 75.280114796000007, 2024-04-20T01:48:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014134643000000001, 2024-04-20T01:48:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 105 2024-04-20T01:48:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962461+0000", 2024-04-20T01:48:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962461+0000", 2024-04-20T01:48:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962460+0000", 2024-04-20T01:48:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962462+0000", 2024-04-20T01:48:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.897e-06 2024-04-20T01:48:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962466+0000", 2024-04-20T01:48:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4520000000000002e-06 2024-04-20T01:48:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962468+0000", 2024-04-20T01:48:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6570000000000001e-06 2024-04-20T01:48:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.966956+0000", 2024-04-20T01:48:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044879730000000001 2024-04-20T01:48:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.967029+0000", 2024-04-20T01:48:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2657999999999995e-05 2024-04-20T01:48:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974602+0000", 2024-04-20T01:48:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075729370000000001 2024-04-20T01:48:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974650+0000", 2024-04-20T01:48:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8000000000000001e-05 2024-04-20T01:48:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976595+0000", 2024-04-20T01:48:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019453789999999999 2024-04-20T01:48:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:109 3.es0 3:7f925225:::smithi16579500-37:head [append 0~606208,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:48:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 75.280104365, 2024-04-20T01:48:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012453573000000001, 2024-04-20T01:48:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 109 2024-04-20T01:48:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:48:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:48:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962471+0000", 2024-04-20T01:48:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962732+0000", 2024-04-20T01:48:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00026073200000000001 2024-04-20T01:48:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962742+0000", 2024-04-20T01:48:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0735e-05 2024-04-20T01:48:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962749+0000", 2024-04-20T01:48:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.365e-06 2024-04-20T01:48:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.967197+0000", 2024-04-20T01:48:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044487119999999996 2024-04-20T01:48:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.967324+0000", 2024-04-20T01:48:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126575 2024-04-20T01:48:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968044+0000", 2024-04-20T01:48:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00072015399999999998 2024-04-20T01:48:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970795+0000", 2024-04-20T01:48:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027507650000000001 2024-04-20T01:48:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974884+0000", 2024-04-20T01:48:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040886469999999999 2024-04-20T01:48:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974925+0000", 2024-04-20T01:48:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1094000000000003e-05 2024-04-20T01:48:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:110 3.es0 3:7f925225:::smithi16579500-37:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962755+0000", 2024-04-20T01:48:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 75.279820494999996, 2024-04-20T01:48:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012356723999999999, 2024-04-20T01:48:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 110 2024-04-20T01:48:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962755+0000", 2024-04-20T01:48:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962755+0000", 2024-04-20T01:48:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962754+0000", 2024-04-20T01:48:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962758+0000", 2024-04-20T01:48:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8750000000000002e-06 2024-04-20T01:48:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962765+0000", 2024-04-20T01:48:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2370000000000003e-06 2024-04-20T01:48:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962770+0000", 2024-04-20T01:48:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9030000000000003e-06 2024-04-20T01:48:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968614+0000", 2024-04-20T01:48:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0058437619999999997 2024-04-20T01:48:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968676+0000", 2024-04-20T01:48:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1592000000000003e-05 2024-04-20T01:48:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.968935+0000", 2024-04-20T01:48:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025953599999999999 2024-04-20T01:48:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970828+0000", 2024-04-20T01:48:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0018928510000000001 2024-04-20T01:48:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975036+0000", 2024-04-20T01:48:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042079680000000003 2024-04-20T01:48:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975112+0000", 2024-04-20T01:48:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5558999999999999e-05 2024-04-20T01:48:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:111 3.es0 3:7f925225:::smithi16579500-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962775+0000", 2024-04-20T01:48:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 75.279800739999999, 2024-04-20T01:48:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014337756, 2024-04-20T01:48:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 111 2024-04-20T01:48:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962775+0000", 2024-04-20T01:48:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962775+0000", 2024-04-20T01:48:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962774+0000", 2024-04-20T01:48:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962777+0000", 2024-04-20T01:48:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7839999999999999e-06 2024-04-20T01:48:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962784+0000", 2024-04-20T01:48:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9809999999999997e-06 2024-04-20T01:48:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962790+0000", 2024-04-20T01:48:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5509999999999999e-06 2024-04-20T01:48:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970564+0000", 2024-04-20T01:48:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077740680000000003 2024-04-20T01:48:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.970609+0000", 2024-04-20T01:48:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5059999999999999e-05 2024-04-20T01:48:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975165+0000", 2024-04-20T01:48:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0045564999999999998 2024-04-20T01:48:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.975263+0000", 2024-04-20T01:48:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7689000000000006e-05 2024-04-20T01:48:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977112+0000", 2024-04-20T01:48:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001849552 2024-04-20T01:48:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:142 3.es0 3:797661d0:::smithi16579500-48:head [append 0~425984,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.962794+0000", 2024-04-20T01:48:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 75.279781538999998, 2024-04-20T01:48:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013969364, 2024-04-20T01:48:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 142 2024-04-20T01:48:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962794+0000", 2024-04-20T01:48:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962794+0000", 2024-04-20T01:48:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.962793+0000", 2024-04-20T01:48:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963014+0000", 2024-04-20T01:48:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000221015 2024-04-20T01:48:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963025+0000", 2024-04-20T01:48:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0739e-05 2024-04-20T01:48:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963032+0000", 2024-04-20T01:48:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7580000000000003e-06 2024-04-20T01:48:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.971338+0000", 2024-04-20T01:48:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008305659 2024-04-20T01:48:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.971477+0000", 2024-04-20T01:48:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013920100000000001 2024-04-20T01:48:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.972169+0000", 2024-04-20T01:48:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00069257800000000001 2024-04-20T01:48:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974577+0000", 2024-04-20T01:48:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024072820000000002 2024-04-20T01:48:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976694+0000", 2024-04-20T01:48:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021172629999999999 2024-04-20T01:48:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976763+0000", 2024-04-20T01:48:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9259999999999998e-05 2024-04-20T01:48:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.263 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.263 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:143 3.es0 3:797661d0:::smithi16579500-48:head [append 0~376832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:50.263 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.963039+0000", 2024-04-20T01:48:50.263 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 75.279536784000001, 2024-04-20T01:48:50.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013923471999999999, 2024-04-20T01:48:50.263 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.263 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:50.263 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:50.263 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:50.263 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:50.263 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 143 2024-04-20T01:48:50.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.263 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963039+0000", 2024-04-20T01:48:50.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963039+0000", 2024-04-20T01:48:50.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963038+0000", 2024-04-20T01:48:50.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:50.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963210+0000", 2024-04-20T01:48:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017155800000000001 2024-04-20T01:48:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963217+0000", 2024-04-20T01:48:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7309999999999993e-06 2024-04-20T01:48:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963230+0000", 2024-04-20T01:48:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.295e-05 2024-04-20T01:48:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.972654+0000", 2024-04-20T01:48:50.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094237369999999997 2024-04-20T01:48:50.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:50.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.972720+0000", 2024-04-20T01:48:50.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5958999999999996e-05 2024-04-20T01:48:50.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.973225+0000", 2024-04-20T01:48:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00050500399999999998 2024-04-20T01:48:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974578+0000", 2024-04-20T01:48:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001353295 2024-04-20T01:48:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976919+0000", 2024-04-20T01:48:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023405840000000002 2024-04-20T01:48:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976962+0000", 2024-04-20T01:48:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3226000000000003e-05 2024-04-20T01:48:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:144 3.es0 3:797661d0:::smithi16579500-48:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:48:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 75.279338150000001, 2024-04-20T01:48:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014456992, 2024-04-20T01:48:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 144 2024-04-20T01:48:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:48:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:48:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963237+0000", 2024-04-20T01:48:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963240+0000", 2024-04-20T01:48:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6459999999999999e-06 2024-04-20T01:48:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963247+0000", 2024-04-20T01:48:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9650000000000002e-06 2024-04-20T01:48:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963253+0000", 2024-04-20T01:48:50.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1900000000000003e-06 2024-04-20T01:48:50.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.973649+0000", 2024-04-20T01:48:50.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010396865999999999 2024-04-20T01:48:50.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:50.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.973745+0000", 2024-04-20T01:48:50.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5345000000000003e-05 2024-04-20T01:48:50.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.974151+0000", 2024-04-20T01:48:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000405893 2024-04-20T01:48:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.976606+0000", 2024-04-20T01:48:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024551730000000002 2024-04-20T01:48:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977622+0000", 2024-04-20T01:48:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0010159570000000001 2024-04-20T01:48:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977694+0000", 2024-04-20T01:48:50.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2469e-05 2024-04-20T01:48:50.273 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.273 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.273 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.273 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:145 3.es0 3:797661d0:::smithi16579500-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-20T01:48:50.273 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:48:50.273 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 75.279316328999997, 2024-04-20T01:48:50.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016339408999999999, 2024-04-20T01:48:50.273 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.273 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:50.273 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 145 2024-04-20T01:48:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:48:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:48:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963259+0000", 2024-04-20T01:48:50.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:50.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963267+0000", 2024-04-20T01:48:50.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2490000000000001e-06 2024-04-20T01:48:50.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963274+0000", 2024-04-20T01:48:50.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7599999999999997e-06 2024-04-20T01:48:50.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.963279+0000", 2024-04-20T01:48:50.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5400000000000003e-06 2024-04-20T01:48:50.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977300+0000", 2024-04-20T01:48:50.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014020608 2024-04-20T01:48:50.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.977365+0000", 2024-04-20T01:48:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5617999999999994e-05 2024-04-20T01:48:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.978057+0000", 2024-04-20T01:48:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00069179099999999998 2024-04-20T01:48:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.978136+0000", 2024-04-20T01:48:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9132000000000001e-05 2024-04-20T01:48:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:34.979598+0000", 2024-04-20T01:48:50.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0014621599999999999 2024-04-20T01:48:50.278 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.278 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.278 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.278 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:245 3.es0 3:78f9000f:::smithi16579500-35:head [copy-from ver 10] snapc 5=[2] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:50.278 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785668+0000", 2024-04-20T01:48:50.278 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.456907142999995, 2024-04-20T01:48:50.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025155522, 2024-04-20T01:48:50.278 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.278 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 245 2024-04-20T01:48:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785668+0000", 2024-04-20T01:48:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785668+0000", 2024-04-20T01:48:50.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785666+0000", 2024-04-20T01:48:50.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999976 2024-04-20T01:48:50.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785676+0000", 2024-04-20T01:48:50.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0139999999999999e-05 2024-04-20T01:48:50.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785692+0000", 2024-04-20T01:48:50.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5894999999999999e-05 2024-04-20T01:48:50.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785715+0000", 2024-04-20T01:48:50.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2654999999999999e-05 2024-04-20T01:48:50.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785771+0000", 2024-04-20T01:48:50.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6331e-05 2024-04-20T01:48:50.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:50.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785779+0000", 2024-04-20T01:48:50.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4789999999999999e-06 2024-04-20T01:48:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791776+0000", 2024-04-20T01:48:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059978840000000002 2024-04-20T01:48:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.791905+0000", 2024-04-20T01:48:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000128147 2024-04-20T01:48:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.805603+0000", 2024-04-20T01:48:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013698824 2024-04-20T01:48:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.808612+0000", 2024-04-20T01:48:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003008541 2024-04-20T01:48:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.810754+0000", 2024-04-20T01:48:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021421840000000001 2024-04-20T01:48:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.810824+0000", 2024-04-20T01:48:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9627000000000003e-05 2024-04-20T01:48:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:246 3.es0 3:78f9000f:::smithi16579500-35:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:48:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.456846963999993, 2024-04-20T01:48:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020980167000000001, 2024-04-20T01:48:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 246 2024-04-20T01:48:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:48:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:48:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785728+0000", 2024-04-20T01:48:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785731+0000", 2024-04-20T01:48:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3550000000000001e-06 2024-04-20T01:48:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785738+0000", 2024-04-20T01:48:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2080000000000003e-06 2024-04-20T01:48:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785747+0000", 2024-04-20T01:48:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.388e-06 2024-04-20T01:48:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785859+0000", 2024-04-20T01:48:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112432 2024-04-20T01:48:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785866+0000", 2024-04-20T01:48:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0890000000000002e-06 2024-04-20T01:48:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792104+0000", 2024-04-20T01:48:50.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062376970000000004 2024-04-20T01:48:50.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for blocked object", 2024-04-20T01:48:50.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.792157+0000", 2024-04-20T01:48:50.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3269000000000001e-05 2024-04-20T01:48:50.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.806521+0000", 2024-04-20T01:48:50.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014363651999999999 2024-04-20T01:48:50.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:50.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.806639+0000", 2024-04-20T01:48:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011755400000000001 2024-04-20T01:48:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.806709+0000", 2024-04-20T01:48:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0037999999999996e-05 2024-04-20T01:48:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:248 3.es0 3:79bd6b86:::smithi16579500-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:48:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.456791203999998, 2024-04-20T01:48:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011602988, 2024-04-20T01:48:50.290 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.290 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:50.290 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:50.290 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:50.290 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:50.290 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 248 2024-04-20T01:48:50.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.290 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:48:50.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:48:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785784+0000", 2024-04-20T01:48:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785787+0000", 2024-04-20T01:48:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4079999999999998e-06 2024-04-20T01:48:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785791+0000", 2024-04-20T01:48:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9929999999999997e-06 2024-04-20T01:48:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785795+0000", 2024-04-20T01:48:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1559999999999997e-06 2024-04-20T01:48:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785936+0000", 2024-04-20T01:48:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014018 2024-04-20T01:48:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785939+0000", 2024-04-20T01:48:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-04-20T01:48:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794183+0000", 2024-04-20T01:48:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082434339999999991 2024-04-20T01:48:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794272+0000", 2024-04-20T01:48:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9246000000000006e-05 2024-04-20T01:48:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.797387+0000", 2024-04-20T01:48:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0031150269999999998 2024-04-20T01:48:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:249 3.es0 3:79bd6b86:::smithi16579500-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-20T01:48:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:48:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.456776907000005, 2024-04-20T01:48:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011999312, 2024-04-20T01:48:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 249 2024-04-20T01:48:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:48:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:48:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785798+0000", 2024-04-20T01:48:50.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967296 2024-04-20T01:48:50.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785800+0000", 2024-04-20T01:48:50.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.995e-06 2024-04-20T01:48:50.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785804+0000", 2024-04-20T01:48:50.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2990000000000001e-06 2024-04-20T01:48:50.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785807+0000", 2024-04-20T01:48:50.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.715e-06 2024-04-20T01:48:50.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785955+0000", 2024-04-20T01:48:50.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000147923 2024-04-20T01:48:50.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting_for_map not empty", 2024-04-20T01:48:50.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.785958+0000", 2024-04-20T01:48:50.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8940000000000002e-06 2024-04-20T01:48:50.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.794609+0000", 2024-04-20T01:48:50.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086509289999999999 2024-04-20T01:48:50.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:50.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.795125+0000", 2024-04-20T01:48:50.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00051564999999999998 2024-04-20T01:48:50.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.797798+0000", 2024-04-20T01:48:50.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0026731250000000002 2024-04-20T01:48:50.300 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.300 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.300 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.300 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:586 3.es0 3:79bd6b86:::smithi16579500-12:head [delete] snapc 20=[20,1f,1a,b] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e65)", 2024-04-20T01:48:50.300 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:06.762289+0000", 2024-04-20T01:48:50.300 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 43.480286827999997, 2024-04-20T01:48:50.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011713963000000001, 2024-04-20T01:48:50.301 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.301 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:50.301 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:50.301 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:50.301 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:50.301 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 586 2024-04-20T01:48:50.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.301 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762289+0000", 2024-04-20T01:48:50.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762289+0000", 2024-04-20T01:48:50.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762287+0000", 2024-04-20T01:48:50.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:50.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762294+0000", 2024-04-20T01:48:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6450000000000005e-06 2024-04-20T01:48:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762308+0000", 2024-04-20T01:48:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3652000000000001e-05 2024-04-20T01:48:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762318+0000", 2024-04-20T01:48:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6679999999999996e-06 2024-04-20T01:48:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762391+0000", 2024-04-20T01:48:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3823000000000001e-05 2024-04-20T01:48:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.762400+0000", 2024-04-20T01:48:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6750000000000008e-06 2024-04-20T01:48:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.767267+0000", 2024-04-20T01:48:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0048666300000000003 2024-04-20T01:48:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.767389+0000", 2024-04-20T01:48:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012267699999999999 2024-04-20T01:48:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.774003+0000", 2024-04-20T01:48:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066131640000000004 2024-04-20T01:48:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:727 3.es0 3:797661d0:::smithi16579500-48:head [append 0~622592] snapc 26=[26,25,1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-20T01:48:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:11.814331+0000", 2024-04-20T01:48:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 38.428244796000001, 2024-04-20T01:48:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013760674000000001, 2024-04-20T01:48:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-20T01:48:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 727 2024-04-20T01:48:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814331+0000", 2024-04-20T01:48:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814331+0000", 2024-04-20T01:48:50.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814329+0000", 2024-04-20T01:48:50.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:50.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814688+0000", 2024-04-20T01:48:50.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00035975800000000001 2024-04-20T01:48:50.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814706+0000", 2024-04-20T01:48:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7248000000000002e-05 2024-04-20T01:48:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814716+0000", 2024-04-20T01:48:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9620000000000005e-06 2024-04-20T01:48:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814789+0000", 2024-04-20T01:48:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3734999999999998e-05 2024-04-20T01:48:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-04-20T01:48:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.814800+0000", 2024-04-20T01:48:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0898e-05 2024-04-20T01:48:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.816951+0000", 2024-04-20T01:48:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021503379999999999 2024-04-20T01:48:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.817082+0000", 2024-04-20T01:48:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013159599999999999 2024-04-20T01:48:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.818406+0000", 2024-04-20T01:48:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013237749999999999 2024-04-20T01:48:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.822671+0000", 2024-04-20T01:48:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042655460000000003 2024-04-20T01:48:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-04-20T01:48:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.828006+0000", 2024-04-20T01:48:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005334999 2024-04-20T01:48:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.828091+0000", 2024-04-20T01:48:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4857000000000004e-05 2024-04-20T01:48:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:732 3.es0 3:797661d0:::smithi16579500-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e70)", 2024-04-20T01:48:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:11.823728+0000", 2024-04-20T01:48:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 38.418847645, 2024-04-20T01:48:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012035161000000001, 2024-04-20T01:48:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:50.312 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:50.312 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:50.312 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 732 2024-04-20T01:48:50.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.312 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823728+0000", 2024-04-20T01:48:50.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823728+0000", 2024-04-20T01:48:50.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823727+0000", 2024-04-20T01:48:50.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:50.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823731+0000", 2024-04-20T01:48:50.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8349999999999997e-06 2024-04-20T01:48:50.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823745+0000", 2024-04-20T01:48:50.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4211e-05 2024-04-20T01:48:50.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.823756+0000", 2024-04-20T01:48:50.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0705e-05 2024-04-20T01:48:50.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.827650+0000", 2024-04-20T01:48:50.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003894163 2024-04-20T01:48:50.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-04-20T01:48:50.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.827758+0000", 2024-04-20T01:48:50.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107759 2024-04-20T01:48:50.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.834824+0000", 2024-04-20T01:48:50.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.007066587 2024-04-20T01:48:50.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:50.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.834920+0000", 2024-04-20T01:48:50.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5639999999999999e-05 2024-04-20T01:48:50.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:11.835763+0000", 2024-04-20T01:48:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000842835 2024-04-20T01:48:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:1007 3.es0 3:797661d0:::smithi16579500-48:head [read 0~6258688,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:31.807916+0000", 2024-04-20T01:48:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.434659890999999, 2024-04-20T01:48:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011898517000000001, 2024-04-20T01:48:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:50.317 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:50.317 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1007 2024-04-20T01:48:50.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.317 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807916+0000", 2024-04-20T01:48:50.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807916+0000", 2024-04-20T01:48:50.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807913+0000", 2024-04-20T01:48:50.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:50.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807935+0000", 2024-04-20T01:48:50.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1943e-05 2024-04-20T01:48:50.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807962+0000", 2024-04-20T01:48:50.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6199000000000001e-05 2024-04-20T01:48:50.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807976+0000", 2024-04-20T01:48:50.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4029000000000001e-05 2024-04-20T01:48:50.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808105+0000", 2024-04-20T01:48:50.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012951299999999999 2024-04-20T01:48:50.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808234+0000", 2024-04-20T01:48:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000129068 2024-04-20T01:48:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.819814+0000", 2024-04-20T01:48:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011579789 2024-04-20T01:48:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:1008 3.es0 3:797661d0:::smithi16579500-48:head [read 0~6258688,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:31.807999+0000", 2024-04-20T01:48:50.321 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.434576842999999, 2024-04-20T01:48:50.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020846726999999999, 2024-04-20T01:48:50.321 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.321 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:50.321 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:50.321 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:50.321 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:50.321 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1008 2024-04-20T01:48:50.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.321 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807999+0000", 2024-04-20T01:48:50.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807999+0000", 2024-04-20T01:48:50.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.807998+0000", 2024-04-20T01:48:50.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:50.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808003+0000", 2024-04-20T01:48:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0919999999999997e-06 2024-04-20T01:48:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808014+0000", 2024-04-20T01:48:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1331999999999999e-05 2024-04-20T01:48:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808021+0000", 2024-04-20T01:48:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0559999999999999e-06 2024-04-20T01:48:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808626+0000", 2024-04-20T01:48:50.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000604493 2024-04-20T01:48:50.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:50.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808697+0000", 2024-04-20T01:48:50.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1711000000000005e-05 2024-04-20T01:48:50.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.828845+0000", 2024-04-20T01:48:50.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.020147805000000001 2024-04-20T01:48:50.324 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.324 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4643.0:1009 3.es0 3:797661d0:::smithi16579500-48:head [read 0~6258688,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-04-20T01:48:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:31.808028+0000", 2024-04-20T01:48:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.434547372000001, 2024-04-20T01:48:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.026288720000000002, 2024-04-20T01:48:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-04-20T01:48:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-04-20T01:48:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4643", 2024-04-20T01:48:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.165:0/121778565", 2024-04-20T01:48:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1009 2024-04-20T01:48:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808028+0000", 2024-04-20T01:48:50.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808028+0000", 2024-04-20T01:48:50.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808027+0000", 2024-04-20T01:48:50.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808032+0000", 2024-04-20T01:48:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1459999999999998e-06 2024-04-20T01:48:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808038+0000", 2024-04-20T01:48:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.494e-06 2024-04-20T01:48:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808044+0000", 2024-04-20T01:48:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2770000000000002e-06 2024-04-20T01:48:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.808958+0000", 2024-04-20T01:48:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00091398300000000005 2024-04-20T01:48:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-04-20T01:48:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.809027+0000", 2024-04-20T01:48:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8675000000000002e-05 2024-04-20T01:48:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.834317+0000", 2024-04-20T01:48:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025289696 2024-04-20T01:48:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpReadReply(3.es0 90/30 ECSubReadReply(tid=129, attrs_read=0))", 2024-04-20T01:48:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:31.816185+0000", 2024-04-20T01:48:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.426390148999999, 2024-04-20T01:48:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012799764, 2024-04-20T01:48:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.816185+0000", 2024-04-20T01:48:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.816185+0000", 2024-04-20T01:48:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.816185+0000", 2024-04-20T01:48:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.824675+0000", 2024-04-20T01:48:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0084907639999999996 2024-04-20T01:48:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.824690+0000", 2024-04-20T01:48:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4503e-05 2024-04-20T01:48:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.824700+0000", 2024-04-20T01:48:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7820000000000005e-06 2024-04-20T01:48:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.824734+0000", 2024-04-20T01:48:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4072000000000002e-05 2024-04-20T01:48:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:31.828985+0000", 2024-04-20T01:48:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042512629999999999 2024-04-20T01:48:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.332 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:50.334 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-20T01:48:50.462 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:50.463 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:50.463 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-04-20T01:48:50.463 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:50.473 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-20T01:48:50.601 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:50.601 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-04-20T01:48:50.601 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-04-20T01:48:50.601 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-04-20T01:48:50.601 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:50.611 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-20T01:48:50.742 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-20T01:48:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-04-20T01:48:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-04-20T01:48:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-04-20T01:48:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 37/30 ECSubWrite(tid=43, reqid=client.4643.0:241, at_version=37'31, trim_to=0'0, roll_forward_to=36'28))", 2024-04-20T01:48:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.793501+0000", 2024-04-20T01:48:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 71.961788045000006, 2024-04-20T01:48:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005847517, 2024-04-20T01:48:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793501+0000", 2024-04-20T01:48:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793501+0000", 2024-04-20T01:48:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793500+0000", 2024-04-20T01:48:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793525+0000", 2024-04-20T01:48:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4641999999999999e-05 2024-04-20T01:48:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793537+0000", 2024-04-20T01:48:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2626000000000001e-05 2024-04-20T01:48:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793547+0000", 2024-04-20T01:48:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.842e-06 2024-04-20T01:48:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793599+0000", 2024-04-20T01:48:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2301e-05 2024-04-20T01:48:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793607+0000", 2024-04-20T01:48:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9070000000000007e-06 2024-04-20T01:48:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799321+0000", 2024-04-20T01:48:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005713358 2024-04-20T01:48:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799349+0000", 2024-04-20T01:48:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7832999999999999e-05 2024-04-20T01:48:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 37/30 ECSubWrite(tid=19, reqid=client.4643.0:235, at_version=37'9, trim_to=0'0, roll_forward_to=33'6))", 2024-04-20T01:48:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:38.793816+0000", 2024-04-20T01:48:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 71.961472569999998, 2024-04-20T01:48:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0055550790000000001, 2024-04-20T01:48:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793816+0000", 2024-04-20T01:48:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793816+0000", 2024-04-20T01:48:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793816+0000", 2024-04-20T01:48:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793838+0000", 2024-04-20T01:48:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2075000000000001e-05 2024-04-20T01:48:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793849+0000", 2024-04-20T01:48:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1433999999999999e-05 2024-04-20T01:48:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793859+0000", 2024-04-20T01:48:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4849999999999998e-06 2024-04-20T01:48:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793956+0000", 2024-04-20T01:48:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7114000000000006e-05 2024-04-20T01:48:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.793965+0000", 2024-04-20T01:48:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0820000000000005e-06 2024-04-20T01:48:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799344+0000", 2024-04-20T01:48:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053794400000000001 2024-04-20T01:48:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:38.799372+0000", 2024-04-20T01:48:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7231000000000001e-05 2024-04-20T01:48:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 38/30 ECSubWrite(tid=23, reqid=osd.3.0:22, at_version=0'0, trim_to=0'0, roll_forward_to=38'12))", 2024-04-20T01:48:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:39.788077+0000", 2024-04-20T01:48:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.967212516000004, 2024-04-20T01:48:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057844480000000002, 2024-04-20T01:48:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788077+0000", 2024-04-20T01:48:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788077+0000", 2024-04-20T01:48:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788076+0000", 2024-04-20T01:48:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788088+0000", 2024-04-20T01:48:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2672e-05 2024-04-20T01:48:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788101+0000", 2024-04-20T01:48:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2468e-05 2024-04-20T01:48:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788121+0000", 2024-04-20T01:48:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0315000000000001e-05 2024-04-20T01:48:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788181+0000", 2024-04-20T01:48:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0257999999999998e-05 2024-04-20T01:48:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.788194+0000", 2024-04-20T01:48:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2714000000000001e-05 2024-04-20T01:48:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.793811+0000", 2024-04-20T01:48:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056169649999999998 2024-04-20T01:48:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:39.793861+0000", 2024-04-20T01:48:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9978000000000002e-05 2024-04-20T01:48:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 53/30 ECSubWrite(tid=32, reqid=client.4643.0:443, at_version=53'21, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.718947+0000", 2024-04-20T01:48:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 56.036341548000003, 2024-04-20T01:48:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009785129, 2024-04-20T01:48:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718947+0000", 2024-04-20T01:48:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718947+0000", 2024-04-20T01:48:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.718947+0000", 2024-04-20T01:48:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719111+0000", 2024-04-20T01:48:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000164227 2024-04-20T01:48:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719125+0000", 2024-04-20T01:48:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.454e-05 2024-04-20T01:48:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719137+0000", 2024-04-20T01:48:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1783999999999999e-05 2024-04-20T01:48:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719206+0000", 2024-04-20T01:48:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9123999999999998e-05 2024-04-20T01:48:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.719221+0000", 2024-04-20T01:48:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4963999999999999e-05 2024-04-20T01:48:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.728704+0000", 2024-04-20T01:48:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094822859999999995 2024-04-20T01:48:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.728733+0000", 2024-04-20T01:48:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8972e-05 2024-04-20T01:48:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 53/30 ECSubWrite(tid=33, reqid=client.4643.0:444, at_version=53'22, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.721427+0000", 2024-04-20T01:48:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 56.033862298000003, 2024-04-20T01:48:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010045943999999999, 2024-04-20T01:48:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721427+0000", 2024-04-20T01:48:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721427+0000", 2024-04-20T01:48:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721424+0000", 2024-04-20T01:48:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999967 2024-04-20T01:48:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721541+0000", 2024-04-20T01:48:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117309 2024-04-20T01:48:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721560+0000", 2024-04-20T01:48:50.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8559999999999998e-05 2024-04-20T01:48:50.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721571+0000", 2024-04-20T01:48:50.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1612e-05 2024-04-20T01:48:50.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721648+0000", 2024-04-20T01:48:50.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6534999999999998e-05 2024-04-20T01:48:50.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.721660+0000", 2024-04-20T01:48:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2396e-05 2024-04-20T01:48:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731447+0000", 2024-04-20T01:48:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097865039999999997 2024-04-20T01:48:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731473+0000", 2024-04-20T01:48:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6129999999999999e-05 2024-04-20T01:48:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 53/30 ECSubWrite(tid=34, reqid=client.4643.0:445, at_version=53'23, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.722888+0000", 2024-04-20T01:48:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 56.032400817999999, 2024-04-20T01:48:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086321880000000007, 2024-04-20T01:48:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722888+0000", 2024-04-20T01:48:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722888+0000", 2024-04-20T01:48:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722887+0000", 2024-04-20T01:48:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722958+0000", 2024-04-20T01:48:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0167999999999996e-05 2024-04-20T01:48:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722970+0000", 2024-04-20T01:48:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2393e-05 2024-04-20T01:48:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.722979+0000", 2024-04-20T01:48:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1509999999999993e-06 2024-04-20T01:48:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723935+0000", 2024-04-20T01:48:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00095590900000000001 2024-04-20T01:48:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.723947+0000", 2024-04-20T01:48:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2405999999999999e-05 2024-04-20T01:48:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731503+0000", 2024-04-20T01:48:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075554990000000002 2024-04-20T01:48:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731520+0000", 2024-04-20T01:48:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7490999999999999e-05 2024-04-20T01:48:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 53/30 ECSubWrite(tid=35, reqid=client.4643.0:446, at_version=53'24, trim_to=0'0, roll_forward_to=47'19))", 2024-04-20T01:48:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:54.724462+0000", 2024-04-20T01:48:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 56.030826738999998, 2024-04-20T01:48:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0070986110000000003, 2024-04-20T01:48:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724462+0000", 2024-04-20T01:48:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724462+0000", 2024-04-20T01:48:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724461+0000", 2024-04-20T01:48:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724482+0000", 2024-04-20T01:48:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0557999999999999e-05 2024-04-20T01:48:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724502+0000", 2024-04-20T01:48:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9704999999999999e-05 2024-04-20T01:48:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724511+0000", 2024-04-20T01:48:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8699999999999998e-06 2024-04-20T01:48:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724537+0000", 2024-04-20T01:48:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.688e-05 2024-04-20T01:48:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.724543+0000", 2024-04-20T01:48:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9909999999999999e-06 2024-04-20T01:48:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731548+0000", 2024-04-20T01:48:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0070045990000000002 2024-04-20T01:48:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:54.731561+0000", 2024-04-20T01:48:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2857000000000001e-05 2024-04-20T01:48:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs2 55/30 ECSubWrite(tid=42, reqid=osd.2.0:42, at_version=55'26, trim_to=0'0, roll_forward_to=52'21))", 2024-04-20T01:48:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:56.731259+0000", 2024-04-20T01:48:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.024030011999997, 2024-04-20T01:48:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0064248020000000003, 2024-04-20T01:48:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731259+0000", 2024-04-20T01:48:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731259+0000", 2024-04-20T01:48:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731259+0000", 2024-04-20T01:48:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731283+0000", 2024-04-20T01:48:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4437e-05 2024-04-20T01:48:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731292+0000", 2024-04-20T01:48:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0880000000000001e-06 2024-04-20T01:48:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731300+0000", 2024-04-20T01:48:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8599999999999993e-06 2024-04-20T01:48:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731742+0000", 2024-04-20T01:48:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044220700000000002 2024-04-20T01:48:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.731753+0000", 2024-04-20T01:48:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0968000000000001e-05 2024-04-20T01:48:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.737642+0000", 2024-04-20T01:48:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0058890239999999996 2024-04-20T01:48:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.737684+0000", 2024-04-20T01:48:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1683000000000001e-05 2024-04-20T01:48:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 55/30 ECSubWrite(tid=41, reqid=osd.3.0:41, at_version=55'30, trim_to=0'0, roll_forward_to=54'26))", 2024-04-20T01:48:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:56.732670+0000", 2024-04-20T01:48:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.022619001000002, 2024-04-20T01:48:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080432620000000007, 2024-04-20T01:48:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732670+0000", 2024-04-20T01:48:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732670+0000", 2024-04-20T01:48:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732669+0000", 2024-04-20T01:48:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732698+0000", 2024-04-20T01:48:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8818000000000001e-05 2024-04-20T01:48:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732711+0000", 2024-04-20T01:48:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2696e-05 2024-04-20T01:48:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732721+0000", 2024-04-20T01:48:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0006e-05 2024-04-20T01:48:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732805+0000", 2024-04-20T01:48:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4152000000000003e-05 2024-04-20T01:48:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.732814+0000", 2024-04-20T01:48:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7110000000000007e-06 2024-04-20T01:48:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.740690+0000", 2024-04-20T01:48:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0078757670000000005 2024-04-20T01:48:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.740713+0000", 2024-04-20T01:48:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3141000000000001e-05 2024-04-20T01:48:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 55/30 ECSubWrite(tid=42, reqid=client.4643.0:451, at_version=55'31, trim_to=0'0, roll_forward_to=55'28))", 2024-04-20T01:48:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:56.739643+0000", 2024-04-20T01:48:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.015646306999997, 2024-04-20T01:48:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056333429999999999, 2024-04-20T01:48:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739643+0000", 2024-04-20T01:48:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739643+0000", 2024-04-20T01:48:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739641+0000", 2024-04-20T01:48:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999981 2024-04-20T01:48:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739813+0000", 2024-04-20T01:48:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017258700000000001 2024-04-20T01:48:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739833+0000", 2024-04-20T01:48:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9312000000000001e-05 2024-04-20T01:48:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.739844+0000", 2024-04-20T01:48:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1303e-05 2024-04-20T01:48:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.740000+0000", 2024-04-20T01:48:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015592899999999999 2024-04-20T01:48:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.740013+0000", 2024-04-20T01:48:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2659e-05 2024-04-20T01:48:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.745246+0000", 2024-04-20T01:48:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0052337290000000003 2024-04-20T01:48:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:56.745276+0000", 2024-04-20T01:48:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9688000000000001e-05 2024-04-20T01:48:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 58/57 ECSubWrite(tid=121, reqid=client.4643.0:485, at_version=58'81, trim_to=0'0, roll_forward_to=58'78))", 2024-04-20T01:48:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:47:59.819732+0000", 2024-04-20T01:48:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 50.935557529999997, 2024-04-20T01:48:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053001919999999996, 2024-04-20T01:48:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819732+0000", 2024-04-20T01:48:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819732+0000", 2024-04-20T01:48:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819731+0000", 2024-04-20T01:48:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819761+0000", 2024-04-20T01:48:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0620999999999997e-05 2024-04-20T01:48:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819775+0000", 2024-04-20T01:48:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4042999999999999e-05 2024-04-20T01:48:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819785+0000", 2024-04-20T01:48:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9369999999999999e-06 2024-04-20T01:48:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819822+0000", 2024-04-20T01:48:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6430000000000002e-05 2024-04-20T01:48:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.819829+0000", 2024-04-20T01:48:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0299999999999996e-06 2024-04-20T01:48:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.825010+0000", 2024-04-20T01:48:50.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0051810900000000002 2024-04-20T01:48:50.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:47:59.825032+0000", 2024-04-20T01:48:50.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1759e-05 2024-04-20T01:48:50.796 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.796 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.796 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.796 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 60/57 ECSubWrite(tid=139, reqid=osd.8.0:138, at_version=0'0, trim_to=0'0, roll_forward_to=60'83))", 2024-04-20T01:48:50.796 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:01.785138+0000", 2024-04-20T01:48:50.796 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 48.970151016999999, 2024-04-20T01:48:50.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0054622710000000003, 2024-04-20T01:48:50.797 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.797 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.797 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785138+0000", 2024-04-20T01:48:50.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785138+0000", 2024-04-20T01:48:50.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785137+0000", 2024-04-20T01:48:50.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:50.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785149+0000", 2024-04-20T01:48:50.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1198e-05 2024-04-20T01:48:50.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785160+0000", 2024-04-20T01:48:50.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066e-05 2024-04-20T01:48:50.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785168+0000", 2024-04-20T01:48:50.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4880000000000007e-06 2024-04-20T01:48:50.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785217+0000", 2024-04-20T01:48:50.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8952000000000002e-05 2024-04-20T01:48:50.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.785228+0000", 2024-04-20T01:48:50.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0915e-05 2024-04-20T01:48:50.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.790582+0000", 2024-04-20T01:48:50.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053536649999999996 2024-04-20T01:48:50.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:01.790600+0000", 2024-04-20T01:48:50.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8677000000000001e-05 2024-04-20T01:48:50.800 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.800 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.800 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.801 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 65/57 ECSubWrite(tid=165, reqid=osd.8.0:165, at_version=65'111, trim_to=0'0, roll_forward_to=65'107))", 2024-04-20T01:48:50.801 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:06.783738+0000", 2024-04-20T01:48:50.801 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 43.971551114, 2024-04-20T01:48:50.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053461910000000001, 2024-04-20T01:48:50.801 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.801 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.801 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783738+0000", 2024-04-20T01:48:50.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783738+0000", 2024-04-20T01:48:50.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783737+0000", 2024-04-20T01:48:50.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:50.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783756+0000", 2024-04-20T01:48:50.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8454e-05 2024-04-20T01:48:50.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783768+0000", 2024-04-20T01:48:50.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1950000000000001e-05 2024-04-20T01:48:50.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783777+0000", 2024-04-20T01:48:50.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0850000000000003e-06 2024-04-20T01:48:50.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783815+0000", 2024-04-20T01:48:50.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7857e-05 2024-04-20T01:48:50.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.783822+0000", 2024-04-20T01:48:50.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1130000000000003e-06 2024-04-20T01:48:50.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.789064+0000", 2024-04-20T01:48:50.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0052421009999999999 2024-04-20T01:48:50.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:06.789084+0000", 2024-04-20T01:48:50.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0395e-05 2024-04-20T01:48:50.805 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.805 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.805 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.806 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 75/57 ECSubWrite(tid=185, reqid=osd.8.0:185, at_version=75'128, trim_to=0'0, roll_forward_to=68'123))", 2024-04-20T01:48:50.806 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.718372+0000", 2024-04-20T01:48:50.806 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 34.036916570999999, 2024-04-20T01:48:50.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.006709179, 2024-04-20T01:48:50.806 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.806 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.806 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718372+0000", 2024-04-20T01:48:50.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718372+0000", 2024-04-20T01:48:50.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718372+0000", 2024-04-20T01:48:50.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:50.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718398+0000", 2024-04-20T01:48:50.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5945e-05 2024-04-20T01:48:50.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718408+0000", 2024-04-20T01:48:50.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5540000000000003e-06 2024-04-20T01:48:50.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718416+0000", 2024-04-20T01:48:50.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4030000000000007e-06 2024-04-20T01:48:50.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.809 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718988+0000", 2024-04-20T01:48:50.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00057154199999999997 2024-04-20T01:48:50.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.809 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.718999+0000", 2024-04-20T01:48:50.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1499e-05 2024-04-20T01:48:50.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725052+0000", 2024-04-20T01:48:50.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0060528350000000003 2024-04-20T01:48:50.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.725082+0000", 2024-04-20T01:48:50.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9816999999999999e-05 2024-04-20T01:48:50.810 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.810 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.811 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.811 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs2 75/30 ECSubWrite(tid=64, reqid=osd.2.0:64, at_version=75'48, trim_to=0'0, roll_forward_to=73'46))", 2024-04-20T01:48:50.811 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.720225+0000", 2024-04-20T01:48:50.811 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 34.035064525000003, 2024-04-20T01:48:50.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0058277789999999999, 2024-04-20T01:48:50.811 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.811 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.811 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720225+0000", 2024-04-20T01:48:50.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720225+0000", 2024-04-20T01:48:50.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720224+0000", 2024-04-20T01:48:50.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:50.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720253+0000", 2024-04-20T01:48:50.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9485e-05 2024-04-20T01:48:50.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720280+0000", 2024-04-20T01:48:50.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.69e-05 2024-04-20T01:48:50.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720291+0000", 2024-04-20T01:48:50.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0589e-05 2024-04-20T01:48:50.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720522+0000", 2024-04-20T01:48:50.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023163799999999999 2024-04-20T01:48:50.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720532+0000", 2024-04-20T01:48:50.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7499999999999998e-06 2024-04-20T01:48:50.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726006+0000", 2024-04-20T01:48:50.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0054741770000000002 2024-04-20T01:48:50.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726052+0000", 2024-04-20T01:48:50.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.613e-05 2024-04-20T01:48:50.816 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.816 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.816 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.816 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.fs2 75/30 ECSubWrite(tid=65, reqid=osd.2.0:65, at_version=75'50, trim_to=0'0, roll_forward_to=73'46))", 2024-04-20T01:48:50.816 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.720305+0000", 2024-04-20T01:48:50.816 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 34.034983889000003, 2024-04-20T01:48:50.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0060264209999999997, 2024-04-20T01:48:50.816 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.816 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.816 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720305+0000", 2024-04-20T01:48:50.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720305+0000", 2024-04-20T01:48:50.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720305+0000", 2024-04-20T01:48:50.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:50.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720327+0000", 2024-04-20T01:48:50.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2181e-05 2024-04-20T01:48:50.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720335+0000", 2024-04-20T01:48:50.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7710000000000006e-06 2024-04-20T01:48:50.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.720341+0000", 2024-04-20T01:48:50.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4690000000000002e-06 2024-04-20T01:48:50.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722262+0000", 2024-04-20T01:48:50.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001920843 2024-04-20T01:48:50.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.722272+0000", 2024-04-20T01:48:50.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0481e-05 2024-04-20T01:48:50.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.820 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.820 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726304+0000", 2024-04-20T01:48:50.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040320979999999996 2024-04-20T01:48:50.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.820 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.820 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.726332+0000", 2024-04-20T01:48:50.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7148000000000002e-05 2024-04-20T01:48:50.820 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.820 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.820 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.821 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.4s1 75/57 ECSubWrite(tid=188, reqid=osd.8.0:188, at_version=75'132, trim_to=0'0, roll_forward_to=75'128))", 2024-04-20T01:48:50.821 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.730972+0000", 2024-04-20T01:48:50.821 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 34.024317181000001, 2024-04-20T01:48:50.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059973420000000001, 2024-04-20T01:48:50.821 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.821 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.821 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.730972+0000", 2024-04-20T01:48:50.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.822 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.822 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.730972+0000", 2024-04-20T01:48:50.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.822 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.822 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.730971+0000", 2024-04-20T01:48:50.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:50.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.822 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.822 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731000+0000", 2024-04-20T01:48:50.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8756e-05 2024-04-20T01:48:50.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731009+0000", 2024-04-20T01:48:50.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6780000000000006e-06 2024-04-20T01:48:50.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.731017+0000", 2024-04-20T01:48:50.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9880000000000004e-06 2024-04-20T01:48:50.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.732017+0000", 2024-04-20T01:48:50.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00099972800000000007 2024-04-20T01:48:50.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.732026+0000", 2024-04-20T01:48:50.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0299999999999999e-06 2024-04-20T01:48:50.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736942+0000", 2024-04-20T01:48:50.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0049165370000000003 2024-04-20T01:48:50.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.736969+0000", 2024-04-20T01:48:50.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7005e-05 2024-04-20T01:48:50.825 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.825 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.825 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.825 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 75/30 ECSubWrite(tid=55, reqid=client.4643.0:781, at_version=75'41, trim_to=0'0, roll_forward_to=66'39))", 2024-04-20T01:48:50.825 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:16.735309+0000", 2024-04-20T01:48:50.825 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 34.019980070000003, 2024-04-20T01:48:50.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005306926, 2024-04-20T01:48:50.825 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.825 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.826 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735309+0000", 2024-04-20T01:48:50.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735309+0000", 2024-04-20T01:48:50.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735309+0000", 2024-04-20T01:48:50.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:50.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735445+0000", 2024-04-20T01:48:50.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013668299999999999 2024-04-20T01:48:50.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735459+0000", 2024-04-20T01:48:50.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3370000000000001e-05 2024-04-20T01:48:50.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735469+0000", 2024-04-20T01:48:50.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8700000000000004e-06 2024-04-20T01:48:50.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735568+0000", 2024-04-20T01:48:50.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9122000000000004e-05 2024-04-20T01:48:50.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.735579+0000", 2024-04-20T01:48:50.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1294999999999999e-05 2024-04-20T01:48:50.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.829 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.829 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.740581+0000", 2024-04-20T01:48:50.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0050022410000000002 2024-04-20T01:48:50.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.829 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.829 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:16.740616+0000", 2024-04-20T01:48:50.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4706000000000003e-05 2024-04-20T01:48:50.829 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.829 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.829 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.830 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 87/30 ECSubWrite(tid=83, reqid=osd.3.0:83, at_version=87'59, trim_to=0'0, roll_forward_to=85'57))", 2024-04-20T01:48:50.830 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:28.775123+0000", 2024-04-20T01:48:50.830 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 21.980165587999998, 2024-04-20T01:48:50.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075716259999999997, 2024-04-20T01:48:50.830 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.830 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.830 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.775123+0000", 2024-04-20T01:48:50.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.775123+0000", 2024-04-20T01:48:50.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.775123+0000", 2024-04-20T01:48:50.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.9999995 2024-04-20T01:48:50.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.775159+0000", 2024-04-20T01:48:50.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6084999999999998e-05 2024-04-20T01:48:50.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.775174+0000", 2024-04-20T01:48:50.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4921e-05 2024-04-20T01:48:50.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.775185+0000", 2024-04-20T01:48:50.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1277999999999999e-05 2024-04-20T01:48:50.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.775257+0000", 2024-04-20T01:48:50.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1247000000000004e-05 2024-04-20T01:48:50.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.775268+0000", 2024-04-20T01:48:50.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.137e-05 2024-04-20T01:48:50.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.782664+0000", 2024-04-20T01:48:50.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0073960780000000004 2024-04-20T01:48:50.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.782695+0000", 2024-04-20T01:48:50.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0978000000000001e-05 2024-04-20T01:48:50.834 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.834 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.834 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.834 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDECSubOpWrite(3.0s5 87/30 ECSubWrite(tid=84, reqid=osd.3.0:84, at_version=87'61, trim_to=0'0, roll_forward_to=85'57))", 2024-04-20T01:48:50.835 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-04-20T01:48:28.776211+0000", 2024-04-20T01:48:50.835 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 21.979078462, 2024-04-20T01:48:50.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0096891890000000008, 2024-04-20T01:48:50.835 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-04-20T01:48:50.835 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-04-20T01:48:50.835 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-04-20T01:48:50.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-04-20T01:48:50.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.776211+0000", 2024-04-20T01:48:50.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-04-20T01:48:50.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.776211+0000", 2024-04-20T01:48:50.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-04-20T01:48:50.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-04-20T01:48:50.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.776210+0000", 2024-04-20T01:48:50.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4294967295.999999 2024-04-20T01:48:50.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-04-20T01:48:50.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.776231+0000", 2024-04-20T01:48:50.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0735999999999999e-05 2024-04-20T01:48:50.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-04-20T01:48:50.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.776248+0000", 2024-04-20T01:48:50.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7459999999999999e-05 2024-04-20T01:48:50.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-04-20T01:48:50.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.776258+0000", 2024-04-20T01:48:50.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0044000000000001e-05 2024-04-20T01:48:50.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-04-20T01:48:50.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.776524+0000", 2024-04-20T01:48:50.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000265665 2024-04-20T01:48:50.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_started", 2024-04-20T01:48:50.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.776534+0000", 2024-04-20T01:48:50.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0152e-05 2024-04-20T01:48:50.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_committed", 2024-04-20T01:48:50.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.785821+0000", 2024-04-20T01:48:50.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092868589999999997 2024-04-20T01:48:50.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-20T01:48:50.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-20T01:48:50.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-04-20T01:48:50.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-04-20T01:48:28.785900+0000", 2024-04-20T01:48:50.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9059000000000003e-05 2024-04-20T01:48:50.840 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.840 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.840 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.840 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-20T01:48:50.840 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-20T01:48:50.840 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-20T01:48:51.857 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:48:51.871+0000 7fbd5bdee7c0 -1 osd.14 88 log_to_monitors true 2024-04-20T01:48:52.930 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-20T01:48:53.075 INFO:teuthology.orchestra.run.smithi123.stderr:no valid command found; 10 closest matches: 2024-04-20T01:48:53.075 INFO:teuthology.orchestra.run.smithi123.stderr:0 2024-04-20T01:48:53.076 INFO:teuthology.orchestra.run.smithi123.stderr:1 2024-04-20T01:48:53.076 INFO:teuthology.orchestra.run.smithi123.stderr:2 2024-04-20T01:48:53.076 INFO:teuthology.orchestra.run.smithi123.stderr:abort 2024-04-20T01:48:53.076 INFO:teuthology.orchestra.run.smithi123.stderr:assert 2024-04-20T01:48:53.076 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs debug_inject_read_zeros 2024-04-20T01:48:53.076 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs files list 2024-04-20T01:48:53.076 INFO:teuthology.orchestra.run.smithi123.stderr:bluefs stats 2024-04-20T01:48:53.076 INFO:teuthology.orchestra.run.smithi123.stderr:bluestore allocator dump block 2024-04-20T01:48:53.076 INFO:teuthology.orchestra.run.smithi123.stderr:bluestore allocator fragmentation block 2024-04-20T01:48:53.076 INFO:teuthology.orchestra.run.smithi123.stderr:admin_socket: invalid command 2024-04-20T01:48:53.078 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-20T01:48:53.078 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-14, ['dump_ops_in_flight'] 2024-04-20T01:48:53.625 INFO:tasks.ceph.osd.14.smithi123.stderr:2024-04-20T01:48:53.639+0000 7fbd47d80700 -1 osd.14 88 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T01:48:58.079 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-20T01:48:58.226 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T01:48:58.226 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-04-20T01:48:58.226 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-04-20T01:48:58.226 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T01:48:58.236 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-04-20T01:48:58.398 INFO:teuthology.orchestra.run.smithi044.stdout:mon.a: {} 2024-04-20T01:48:58.410 INFO:teuthology.orchestra.run.smithi044.stdout:mon.b: {} 2024-04-20T01:48:58.420 INFO:teuthology.orchestra.run.smithi044.stdout:mon.c: {} 2024-04-20T01:48:58.422 INFO:teuthology.orchestra.run.smithi044.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-20T01:48:58.422 INFO:teuthology.orchestra.run.smithi044.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-20T01:48:58.435 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-20T01:48:58.866 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.0 to 1 (10000) 2024-04-20T01:48:58.880 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-04-20T01:48:59.876 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-04-20T01:48:59.893 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-20T01:49:00.883 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.4 to 1 (10000) 2024-04-20T01:49:00.898 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-04-20T01:49:01.891 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-04-20T01:49:01.905 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-04-20T01:49:02.896 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.8 to 1 (10000) 2024-04-20T01:49:02.911 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 8 1 2024-04-20T01:49:03.901 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.8 primary-affinity to 1 (8655362) 2024-04-20T01:49:03.916 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 12 1 2024-04-20T01:49:04.907 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.12 to 1 (10000) 2024-04-20T01:49:04.923 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 12 1 2024-04-20T01:49:05.913 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.12 primary-affinity to 1 (8655362) 2024-04-20T01:49:05.928 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-20T01:49:06.922 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.1 to 1 (10000) 2024-04-20T01:49:06.937 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-04-20T01:49:07.929 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-04-20T01:49:07.944 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-20T01:49:08.938 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.5 to 1 (10000) 2024-04-20T01:49:08.952 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-04-20T01:49:09.946 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-04-20T01:49:09.960 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-04-20T01:49:10.951 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.9 to 1 (10000) 2024-04-20T01:49:10.966 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 9 1 2024-04-20T01:49:11.957 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.9 primary-affinity to 1 (8655362) 2024-04-20T01:49:11.973 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 13 1 2024-04-20T01:49:12.967 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.13 to 1 (10000) 2024-04-20T01:49:12.982 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 13 1 2024-04-20T01:49:13.977 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.13 primary-affinity to 1 (8655362) 2024-04-20T01:49:13.991 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-20T01:49:14.985 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.2 to 1 (10000) 2024-04-20T01:49:15.000 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-04-20T01:49:15.992 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-04-20T01:49:16.007 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-20T01:49:17.000 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.6 to 1 (10000) 2024-04-20T01:49:17.015 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-04-20T01:49:18.009 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-04-20T01:49:18.024 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-04-20T01:49:19.014 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.10 to 1 (10000) 2024-04-20T01:49:19.029 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 10 1 2024-04-20T01:49:20.021 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.10 primary-affinity to 1 (8655362) 2024-04-20T01:49:20.036 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-20T01:49:21.030 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.3 to 1 (10000) 2024-04-20T01:49:21.044 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-04-20T01:49:22.037 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-04-20T01:49:22.053 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-20T01:49:23.042 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.7 to 1 (10000) 2024-04-20T01:49:23.056 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-04-20T01:49:24.044 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-04-20T01:49:24.059 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-04-20T01:49:25.052 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.11 to 1 (10000) 2024-04-20T01:49:25.067 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 11 1 2024-04-20T01:49:26.060 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.11 primary-affinity to 1 (8655362) 2024-04-20T01:49:26.075 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 15 1 2024-04-20T01:49:27.068 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.15 to 1 (10000) 2024-04-20T01:49:27.083 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 15 1 2024-04-20T01:49:28.075 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.15 primary-affinity to 1 (8655362) 2024-04-20T01:49:28.090 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 14 1 2024-04-20T01:49:29.082 INFO:teuthology.orchestra.run.smithi044.stderr:reweighted osd.14 to 1 (10000) 2024-04-20T01:49:29.097 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 14 1 2024-04-20T01:49:30.090 INFO:teuthology.orchestra.run.smithi044.stderr:set osd.14 primary-affinity to 1 (8655362) 2024-04-20T01:49:30.106 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-04-20T01:49:30.106 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-04-20T01:49:30.106 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-04-20T01:49:30.106 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-04-20T01:49:30.106 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-20T01:49:30.107 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T01:49:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:49:30.481 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":135,"fsid":"9c2f64c1-aa35-4337-9f5c-081b9e3a2938","created":"2024-04-20T01:47:02.934241+0000","modified":"2024-04-20T01:49:30.094179+0000","last_up_change":"2024-04-20T01:48:54.636657+0000","last_in_change":"2024-04-20T01:47:07.894719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T01:47:15.195181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-20T01:47:18.448004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3.9900000095367432,"score_stable":3.9900000095367432,"optimal_score":0.68999999761581421,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3903b88f-ff2b-4252-9480-92b701b1e59b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":43518},{"type":"v1","addr":"172.21.15.44:6825","nonce":43518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":43518},{"type":"v1","addr":"172.21.15.44:6827","nonce":43518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":43518},{"type":"v1","addr":"172.21.15.44:6831","nonce":43518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":43518},{"type":"v1","addr":"172.21.15.44:6829","nonce":43518}]},"public_addr":"172.21.15.44:6825/43518","cluster_addr":"172.21.15.44:6827/43518","heartbeat_back_addr":"172.21.15.44:6831/43518","heartbeat_front_addr":"172.21.15.44:6829/43518","state":["exists","up"]},{"osd":1,"uuid":"c5ff0175-b4f0-40a5-86c7-7a0a351bf070","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":74361},{"type":"v1","addr":"172.21.15.121:6817","nonce":74361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":74361},{"type":"v1","addr":"172.21.15.121:6819","nonce":74361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":74361},{"type":"v1","addr":"172.21.15.121:6823","nonce":74361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":74361},{"type":"v1","addr":"172.21.15.121:6821","nonce":74361}]},"public_addr":"172.21.15.121:6817/74361","cluster_addr":"172.21.15.121:6819/74361","heartbeat_back_addr":"172.21.15.121:6823/74361","heartbeat_front_addr":"172.21.15.121:6821/74361","state":["exists","up"]},{"osd":2,"uuid":"7af325c1-36b5-420c-a2a9-44c00a002977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6809","nonce":43108},{"type":"v1","addr":"172.21.15.123:6811","nonce":43108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":43108},{"type":"v1","addr":"172.21.15.123:6816","nonce":43108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":43108},{"type":"v1","addr":"172.21.15.123:6823","nonce":43108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":43108},{"type":"v1","addr":"172.21.15.123:6821","nonce":43108}]},"public_addr":"172.21.15.123:6811/43108","cluster_addr":"172.21.15.123:6816/43108","heartbeat_back_addr":"172.21.15.123:6823/43108","heartbeat_front_addr":"172.21.15.123:6821/43108","state":["exists","up"]},{"osd":3,"uuid":"99334f43-83ea-4c3f-b91d-ae72c8361b8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":74545},{"type":"v1","addr":"172.21.15.165:6825","nonce":74545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":74545},{"type":"v1","addr":"172.21.15.165:6827","nonce":74545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":74545},{"type":"v1","addr":"172.21.15.165:6831","nonce":74545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":74545},{"type":"v1","addr":"172.21.15.165:6829","nonce":74545}]},"public_addr":"172.21.15.165:6825/74545","cluster_addr":"172.21.15.165:6827/74545","heartbeat_back_addr":"172.21.15.165:6831/74545","heartbeat_front_addr":"172.21.15.165:6829/74545","state":["exists","up"]},{"osd":4,"uuid":"8764d799-af67-4e6c-889f-9fdd91201cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":43517},{"type":"v1","addr":"172.21.15.44:6811","nonce":43517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":43517},{"type":"v1","addr":"172.21.15.44:6815","nonce":43517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":43517},{"type":"v1","addr":"172.21.15.44:6823","nonce":43517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6817","nonce":43517},{"type":"v1","addr":"172.21.15.44:6819","nonce":43517}]},"public_addr":"172.21.15.44:6811/43517","cluster_addr":"172.21.15.44:6815/43517","heartbeat_back_addr":"172.21.15.44:6823/43517","heartbeat_front_addr":"172.21.15.44:6819/43517","state":["exists","up"]},{"osd":5,"uuid":"2556b2fa-4dcc-479c-a379-6f9fe96e9c77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":74354},{"type":"v1","addr":"172.21.15.121:6801","nonce":74354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":74354},{"type":"v1","addr":"172.21.15.121:6803","nonce":74354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":74354},{"type":"v1","addr":"172.21.15.121:6807","nonce":74354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":74354},{"type":"v1","addr":"172.21.15.121:6805","nonce":74354}]},"public_addr":"172.21.15.121:6801/74354","cluster_addr":"172.21.15.121:6803/74354","heartbeat_back_addr":"172.21.15.121:6807/74354","heartbeat_front_addr":"172.21.15.121:6805/74354","state":["exists","up"]},{"osd":6,"uuid":"baeaae3a-fff4-4ccc-a08a-ff32db4531ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":43107},{"type":"v1","addr":"172.21.15.123:6801","nonce":43107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":43107},{"type":"v1","addr":"172.21.15.123:6803","nonce":43107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":43107},{"type":"v1","addr":"172.21.15.123:6807","nonce":43107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":43107},{"type":"v1","addr":"172.21.15.123:6805","nonce":43107}]},"public_addr":"172.21.15.123:6801/43107","cluster_addr":"172.21.15.123:6803/43107","heartbeat_back_addr":"172.21.15.123:6807/43107","heartbeat_front_addr":"172.21.15.123:6805/43107","state":["exists","up"]},{"osd":7,"uuid":"0019f005-3e62-4d9d-b334-f1d347a484ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":74543},{"type":"v1","addr":"172.21.15.165:6801","nonce":74543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":74543},{"type":"v1","addr":"172.21.15.165:6803","nonce":74543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":74543},{"type":"v1","addr":"172.21.15.165:6807","nonce":74543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":74543},{"type":"v1","addr":"172.21.15.165:6805","nonce":74543}]},"public_addr":"172.21.15.165:6801/74543","cluster_addr":"172.21.15.165:6803/74543","heartbeat_back_addr":"172.21.15.165:6807/74543","heartbeat_front_addr":"172.21.15.165:6805/74543","state":["exists","up"]},{"osd":8,"uuid":"6f5200a3-8980-4241-a6fc-ef604fa612b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":43515},{"type":"v1","addr":"172.21.15.44:6810","nonce":43515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":43515},{"type":"v1","addr":"172.21.15.44:6814","nonce":43515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":43515},{"type":"v1","addr":"172.21.15.44:6822","nonce":43515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":43515},{"type":"v1","addr":"172.21.15.44:6818","nonce":43515}]},"public_addr":"172.21.15.44:6810/43515","cluster_addr":"172.21.15.44:6814/43515","heartbeat_back_addr":"172.21.15.44:6822/43515","heartbeat_front_addr":"172.21.15.44:6818/43515","state":["exists","up"]},{"osd":9,"uuid":"29cf9aaa-3a99-4605-92f4-64f1f6a1728d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":74379},{"type":"v1","addr":"172.21.15.121:6825","nonce":74379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":74379},{"type":"v1","addr":"172.21.15.121:6827","nonce":74379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":74379},{"type":"v1","addr":"172.21.15.121:6831","nonce":74379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":74379},{"type":"v1","addr":"172.21.15.121:6829","nonce":74379}]},"public_addr":"172.21.15.121:6825/74379","cluster_addr":"172.21.15.121:6827/74379","heartbeat_back_addr":"172.21.15.121:6831/74379","heartbeat_front_addr":"172.21.15.121:6829/74379","state":["exists","up"]},{"osd":10,"uuid":"705906c4-1514-4754-ac40-45226a837180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":43109},{"type":"v1","addr":"172.21.15.123:6810","nonce":43109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":43109},{"type":"v1","addr":"172.21.15.123:6813","nonce":43109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":43109},{"type":"v1","addr":"172.21.15.123:6820","nonce":43109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":43109},{"type":"v1","addr":"172.21.15.123:6817","nonce":43109}]},"public_addr":"172.21.15.123:6810/43109","cluster_addr":"172.21.15.123:6813/43109","heartbeat_back_addr":"172.21.15.123:6820/43109","heartbeat_front_addr":"172.21.15.123:6817/43109","state":["exists","up"]},{"osd":11,"uuid":"b45f2645-72d1-4d72-89e9-2212f5aa074c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":74544},{"type":"v1","addr":"172.21.15.165:6809","nonce":74544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":74544},{"type":"v1","addr":"172.21.15.165:6811","nonce":74544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":74544},{"type":"v1","addr":"172.21.15.165:6815","nonce":74544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":74544},{"type":"v1","addr":"172.21.15.165:6813","nonce":74544}]},"public_addr":"172.21.15.165:6809/74544","cluster_addr":"172.21.15.165:6811/74544","heartbeat_back_addr":"172.21.15.165:6815/74544","heartbeat_front_addr":"172.21.15.165:6813/74544","state":["exists","up"]},{"osd":12,"uuid":"3ea410b9-a368-467b-a179-2e3aba562d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":43516},{"type":"v1","addr":"172.21.15.44:6801","nonce":43516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":43516},{"type":"v1","addr":"172.21.15.44:6803","nonce":43516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":43516},{"type":"v1","addr":"172.21.15.44:6807","nonce":43516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":43516},{"type":"v1","addr":"172.21.15.44:6805","nonce":43516}]},"public_addr":"172.21.15.44:6801/43516","cluster_addr":"172.21.15.44:6803/43516","heartbeat_back_addr":"172.21.15.44:6807/43516","heartbeat_front_addr":"172.21.15.44:6805/43516","state":["exists","up"]},{"osd":13,"uuid":"15a5154b-e7cb-4b57-8468-dcb048d93f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":74355},{"type":"v1","addr":"172.21.15.121:6809","nonce":74355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":74355},{"type":"v1","addr":"172.21.15.121:6811","nonce":74355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":74355},{"type":"v1","addr":"172.21.15.121:6815","nonce":74355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":74355},{"type":"v1","addr":"172.21.15.121:6813","nonce":74355}]},"public_addr":"172.21.15.121:6809/74355","cluster_addr":"172.21.15.121:6811/74355","heartbeat_back_addr":"172.21.15.121:6815/74355","heartbeat_front_addr":"172.21.15.121:6813/74355","state":["exists","up"]},{"osd":14,"uuid":"23b53da2-f848-4e76-9632-4d9effd5b430","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":21,"last_clean_end":88,"up_from":102,"up_thru":30,"down_at":90,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":53453},{"type":"v1","addr":"172.21.15.123:6825","nonce":53453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":53453},{"type":"v1","addr":"172.21.15.123:6827","nonce":53453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":53453},{"type":"v1","addr":"172.21.15.123:6831","nonce":53453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":53453},{"type":"v1","addr":"172.21.15.123:6829","nonce":53453}]},"public_addr":"172.21.15.123:6825/53453","cluster_addr":"172.21.15.123:6827/53453","heartbeat_back_addr":"172.21.15.123:6831/53453","heartbeat_front_addr":"172.21.15.123:6829/53453","state":["exists","up"]},{"osd":15,"uuid":"546efc0f-82a3-47d2-aeec-2d849eb1faa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":74562},{"type":"v1","addr":"172.21.15.165:6817","nonce":74562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":74562},{"type":"v1","addr":"172.21.15.165:6819","nonce":74562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":74562},{"type":"v1","addr":"172.21.15.165:6823","nonce":74562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":74562},{"type":"v1","addr":"172.21.15.165:6821","nonce":74562}]},"public_addr":"172.21.15.165:6817/74562","cluster_addr":"172.21.15.165:6819/74562","heartbeat_back_addr":"172.21.15.165:6823/74562","heartbeat_front_addr":"172.21.15.165:6821/74562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:32.980703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:19.984157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:41.034618+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:15.114260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:25.217793+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:25.078524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:35.034578+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:26.485365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:28.281278+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:23.704764+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:28.498783+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:49:22.391036+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:28.932481+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:29.119303+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:52.833382+0000","dead_epoch":90},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:17.320410+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":40,"length":1},{"begin":54,"length":1},{"begin":56,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T01:49:30.497 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-20T01:49:30.497 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T01:49:30.862 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:49:30.862 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":135,"fsid":"9c2f64c1-aa35-4337-9f5c-081b9e3a2938","created":"2024-04-20T01:47:02.934241+0000","modified":"2024-04-20T01:49:30.094179+0000","last_up_change":"2024-04-20T01:48:54.636657+0000","last_in_change":"2024-04-20T01:47:07.894719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T01:47:15.195181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-20T01:47:18.448004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3.9900000095367432,"score_stable":3.9900000095367432,"optimal_score":0.68999999761581421,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3903b88f-ff2b-4252-9480-92b701b1e59b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":43518},{"type":"v1","addr":"172.21.15.44:6825","nonce":43518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":43518},{"type":"v1","addr":"172.21.15.44:6827","nonce":43518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":43518},{"type":"v1","addr":"172.21.15.44:6831","nonce":43518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":43518},{"type":"v1","addr":"172.21.15.44:6829","nonce":43518}]},"public_addr":"172.21.15.44:6825/43518","cluster_addr":"172.21.15.44:6827/43518","heartbeat_back_addr":"172.21.15.44:6831/43518","heartbeat_front_addr":"172.21.15.44:6829/43518","state":["exists","up"]},{"osd":1,"uuid":"c5ff0175-b4f0-40a5-86c7-7a0a351bf070","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":74361},{"type":"v1","addr":"172.21.15.121:6817","nonce":74361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":74361},{"type":"v1","addr":"172.21.15.121:6819","nonce":74361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":74361},{"type":"v1","addr":"172.21.15.121:6823","nonce":74361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":74361},{"type":"v1","addr":"172.21.15.121:6821","nonce":74361}]},"public_addr":"172.21.15.121:6817/74361","cluster_addr":"172.21.15.121:6819/74361","heartbeat_back_addr":"172.21.15.121:6823/74361","heartbeat_front_addr":"172.21.15.121:6821/74361","state":["exists","up"]},{"osd":2,"uuid":"7af325c1-36b5-420c-a2a9-44c00a002977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6809","nonce":43108},{"type":"v1","addr":"172.21.15.123:6811","nonce":43108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":43108},{"type":"v1","addr":"172.21.15.123:6816","nonce":43108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":43108},{"type":"v1","addr":"172.21.15.123:6823","nonce":43108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":43108},{"type":"v1","addr":"172.21.15.123:6821","nonce":43108}]},"public_addr":"172.21.15.123:6811/43108","cluster_addr":"172.21.15.123:6816/43108","heartbeat_back_addr":"172.21.15.123:6823/43108","heartbeat_front_addr":"172.21.15.123:6821/43108","state":["exists","up"]},{"osd":3,"uuid":"99334f43-83ea-4c3f-b91d-ae72c8361b8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":74545},{"type":"v1","addr":"172.21.15.165:6825","nonce":74545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":74545},{"type":"v1","addr":"172.21.15.165:6827","nonce":74545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":74545},{"type":"v1","addr":"172.21.15.165:6831","nonce":74545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":74545},{"type":"v1","addr":"172.21.15.165:6829","nonce":74545}]},"public_addr":"172.21.15.165:6825/74545","cluster_addr":"172.21.15.165:6827/74545","heartbeat_back_addr":"172.21.15.165:6831/74545","heartbeat_front_addr":"172.21.15.165:6829/74545","state":["exists","up"]},{"osd":4,"uuid":"8764d799-af67-4e6c-889f-9fdd91201cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":43517},{"type":"v1","addr":"172.21.15.44:6811","nonce":43517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":43517},{"type":"v1","addr":"172.21.15.44:6815","nonce":43517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":43517},{"type":"v1","addr":"172.21.15.44:6823","nonce":43517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6817","nonce":43517},{"type":"v1","addr":"172.21.15.44:6819","nonce":43517}]},"public_addr":"172.21.15.44:6811/43517","cluster_addr":"172.21.15.44:6815/43517","heartbeat_back_addr":"172.21.15.44:6823/43517","heartbeat_front_addr":"172.21.15.44:6819/43517","state":["exists","up"]},{"osd":5,"uuid":"2556b2fa-4dcc-479c-a379-6f9fe96e9c77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":74354},{"type":"v1","addr":"172.21.15.121:6801","nonce":74354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":74354},{"type":"v1","addr":"172.21.15.121:6803","nonce":74354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":74354},{"type":"v1","addr":"172.21.15.121:6807","nonce":74354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":74354},{"type":"v1","addr":"172.21.15.121:6805","nonce":74354}]},"public_addr":"172.21.15.121:6801/74354","cluster_addr":"172.21.15.121:6803/74354","heartbeat_back_addr":"172.21.15.121:6807/74354","heartbeat_front_addr":"172.21.15.121:6805/74354","state":["exists","up"]},{"osd":6,"uuid":"baeaae3a-fff4-4ccc-a08a-ff32db4531ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":43107},{"type":"v1","addr":"172.21.15.123:6801","nonce":43107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":43107},{"type":"v1","addr":"172.21.15.123:6803","nonce":43107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":43107},{"type":"v1","addr":"172.21.15.123:6807","nonce":43107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":43107},{"type":"v1","addr":"172.21.15.123:6805","nonce":43107}]},"public_addr":"172.21.15.123:6801/43107","cluster_addr":"172.21.15.123:6803/43107","heartbeat_back_addr":"172.21.15.123:6807/43107","heartbeat_front_addr":"172.21.15.123:6805/43107","state":["exists","up"]},{"osd":7,"uuid":"0019f005-3e62-4d9d-b334-f1d347a484ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":74543},{"type":"v1","addr":"172.21.15.165:6801","nonce":74543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":74543},{"type":"v1","addr":"172.21.15.165:6803","nonce":74543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":74543},{"type":"v1","addr":"172.21.15.165:6807","nonce":74543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":74543},{"type":"v1","addr":"172.21.15.165:6805","nonce":74543}]},"public_addr":"172.21.15.165:6801/74543","cluster_addr":"172.21.15.165:6803/74543","heartbeat_back_addr":"172.21.15.165:6807/74543","heartbeat_front_addr":"172.21.15.165:6805/74543","state":["exists","up"]},{"osd":8,"uuid":"6f5200a3-8980-4241-a6fc-ef604fa612b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":43515},{"type":"v1","addr":"172.21.15.44:6810","nonce":43515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":43515},{"type":"v1","addr":"172.21.15.44:6814","nonce":43515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":43515},{"type":"v1","addr":"172.21.15.44:6822","nonce":43515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":43515},{"type":"v1","addr":"172.21.15.44:6818","nonce":43515}]},"public_addr":"172.21.15.44:6810/43515","cluster_addr":"172.21.15.44:6814/43515","heartbeat_back_addr":"172.21.15.44:6822/43515","heartbeat_front_addr":"172.21.15.44:6818/43515","state":["exists","up"]},{"osd":9,"uuid":"29cf9aaa-3a99-4605-92f4-64f1f6a1728d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":74379},{"type":"v1","addr":"172.21.15.121:6825","nonce":74379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":74379},{"type":"v1","addr":"172.21.15.121:6827","nonce":74379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":74379},{"type":"v1","addr":"172.21.15.121:6831","nonce":74379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":74379},{"type":"v1","addr":"172.21.15.121:6829","nonce":74379}]},"public_addr":"172.21.15.121:6825/74379","cluster_addr":"172.21.15.121:6827/74379","heartbeat_back_addr":"172.21.15.121:6831/74379","heartbeat_front_addr":"172.21.15.121:6829/74379","state":["exists","up"]},{"osd":10,"uuid":"705906c4-1514-4754-ac40-45226a837180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":43109},{"type":"v1","addr":"172.21.15.123:6810","nonce":43109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":43109},{"type":"v1","addr":"172.21.15.123:6813","nonce":43109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":43109},{"type":"v1","addr":"172.21.15.123:6820","nonce":43109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":43109},{"type":"v1","addr":"172.21.15.123:6817","nonce":43109}]},"public_addr":"172.21.15.123:6810/43109","cluster_addr":"172.21.15.123:6813/43109","heartbeat_back_addr":"172.21.15.123:6820/43109","heartbeat_front_addr":"172.21.15.123:6817/43109","state":["exists","up"]},{"osd":11,"uuid":"b45f2645-72d1-4d72-89e9-2212f5aa074c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":74544},{"type":"v1","addr":"172.21.15.165:6809","nonce":74544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":74544},{"type":"v1","addr":"172.21.15.165:6811","nonce":74544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":74544},{"type":"v1","addr":"172.21.15.165:6815","nonce":74544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":74544},{"type":"v1","addr":"172.21.15.165:6813","nonce":74544}]},"public_addr":"172.21.15.165:6809/74544","cluster_addr":"172.21.15.165:6811/74544","heartbeat_back_addr":"172.21.15.165:6815/74544","heartbeat_front_addr":"172.21.15.165:6813/74544","state":["exists","up"]},{"osd":12,"uuid":"3ea410b9-a368-467b-a179-2e3aba562d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":43516},{"type":"v1","addr":"172.21.15.44:6801","nonce":43516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":43516},{"type":"v1","addr":"172.21.15.44:6803","nonce":43516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":43516},{"type":"v1","addr":"172.21.15.44:6807","nonce":43516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":43516},{"type":"v1","addr":"172.21.15.44:6805","nonce":43516}]},"public_addr":"172.21.15.44:6801/43516","cluster_addr":"172.21.15.44:6803/43516","heartbeat_back_addr":"172.21.15.44:6807/43516","heartbeat_front_addr":"172.21.15.44:6805/43516","state":["exists","up"]},{"osd":13,"uuid":"15a5154b-e7cb-4b57-8468-dcb048d93f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":74355},{"type":"v1","addr":"172.21.15.121:6809","nonce":74355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":74355},{"type":"v1","addr":"172.21.15.121:6811","nonce":74355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":74355},{"type":"v1","addr":"172.21.15.121:6815","nonce":74355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":74355},{"type":"v1","addr":"172.21.15.121:6813","nonce":74355}]},"public_addr":"172.21.15.121:6809/74355","cluster_addr":"172.21.15.121:6811/74355","heartbeat_back_addr":"172.21.15.121:6815/74355","heartbeat_front_addr":"172.21.15.121:6813/74355","state":["exists","up"]},{"osd":14,"uuid":"23b53da2-f848-4e76-9632-4d9effd5b430","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":21,"last_clean_end":88,"up_from":102,"up_thru":30,"down_at":90,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":53453},{"type":"v1","addr":"172.21.15.123:6825","nonce":53453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":53453},{"type":"v1","addr":"172.21.15.123:6827","nonce":53453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":53453},{"type":"v1","addr":"172.21.15.123:6831","nonce":53453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":53453},{"type":"v1","addr":"172.21.15.123:6829","nonce":53453}]},"public_addr":"172.21.15.123:6825/53453","cluster_addr":"172.21.15.123:6827/53453","heartbeat_back_addr":"172.21.15.123:6831/53453","heartbeat_front_addr":"172.21.15.123:6829/53453","state":["exists","up"]},{"osd":15,"uuid":"546efc0f-82a3-47d2-aeec-2d849eb1faa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":74562},{"type":"v1","addr":"172.21.15.165:6817","nonce":74562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":74562},{"type":"v1","addr":"172.21.15.165:6819","nonce":74562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":74562},{"type":"v1","addr":"172.21.15.165:6823","nonce":74562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":74562},{"type":"v1","addr":"172.21.15.165:6821","nonce":74562}]},"public_addr":"172.21.15.165:6817/74562","cluster_addr":"172.21.15.165:6819/74562","heartbeat_back_addr":"172.21.15.165:6823/74562","heartbeat_front_addr":"172.21.15.165:6821/74562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:32.980703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:19.984157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:41.034618+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:15.114260+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:25.217793+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:25.078524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:35.034578+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:26.485365+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:28.281278+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:23.704764+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:28.498783+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:49:22.391036+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:28.932481+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:29.119303+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:52.833382+0000","dead_epoch":90},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:17.320410+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":40,"length":1},{"begin":54,"length":1},{"begin":56,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T01:49:30.878 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-20T01:49:30.879 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-20T01:49:30.879 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-20T01:49:30.880 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-20T01:49:30.880 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-20T01:49:30.880 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-20T01:49:30.881 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-20T01:49:30.881 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-20T01:49:30.882 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-04-20T01:49:30.882 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-04-20T01:49:30.882 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-04-20T01:49:30.883 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-04-20T01:49:30.883 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2024-04-20T01:49:30.884 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2024-04-20T01:49:30.884 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2024-04-20T01:49:30.884 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2024-04-20T01:49:31.412 INFO:teuthology.orchestra.run.smithi044.stdout:90194313248 2024-04-20T01:49:31.412 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-20T01:49:31.456 INFO:teuthology.orchestra.run.smithi044.stdout:90194313247 2024-04-20T01:49:31.457 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-20T01:49:31.457 INFO:teuthology.orchestra.run.smithi044.stdout:90194313247 2024-04-20T01:49:31.458 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-20T01:49:31.458 INFO:teuthology.orchestra.run.smithi044.stdout:90194313246 2024-04-20T01:49:31.458 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-20T01:49:31.461 INFO:teuthology.orchestra.run.smithi044.stdout:90194313248 2024-04-20T01:49:31.461 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-20T01:49:31.471 INFO:teuthology.orchestra.run.smithi044.stdout:90194313247 2024-04-20T01:49:31.471 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-04-20T01:49:31.523 INFO:teuthology.orchestra.run.smithi044.stdout:90194313248 2024-04-20T01:49:31.523 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-04-20T01:49:31.542 INFO:teuthology.orchestra.run.smithi044.stdout:90194313248 2024-04-20T01:49:31.542 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-04-20T01:49:31.543 INFO:teuthology.orchestra.run.smithi044.stdout:90194313248 2024-04-20T01:49:31.544 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-20T01:49:31.579 INFO:teuthology.orchestra.run.smithi044.stdout:90194313249 2024-04-20T01:49:31.580 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-20T01:49:31.581 INFO:teuthology.orchestra.run.smithi044.stdout:94489280542 2024-04-20T01:49:31.581 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-04-20T01:49:31.587 INFO:teuthology.orchestra.run.smithi044.stdout:90194313247 2024-04-20T01:49:31.588 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-04-20T01:49:31.602 INFO:teuthology.orchestra.run.smithi044.stdout:90194313246 2024-04-20T01:49:31.602 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-20T01:49:31.628 INFO:teuthology.orchestra.run.smithi044.stdout:90194313246 2024-04-20T01:49:31.628 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-04-20T01:49:31.680 INFO:teuthology.orchestra.run.smithi044.stdout:90194313247 2024-04-20T01:49:31.680 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-04-20T01:49:31.777 INFO:teuthology.orchestra.run.smithi044.stdout:438086664201 2024-04-20T01:49:31.778 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-04-20T01:49:32.458 INFO:teuthology.orchestra.run.smithi044.stdout:90194313246 2024-04-20T01:49:32.511 INFO:teuthology.orchestra.run.smithi044.stdout:90194313247 2024-04-20T01:49:32.520 INFO:teuthology.orchestra.run.smithi044.stdout:90194313246 2024-04-20T01:49:32.525 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313247 got 90194313246 for osd.5 2024-04-20T01:49:32.598 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313248 got 90194313247 for osd.7 2024-04-20T01:49:32.619 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313247 got 90194313246 for osd.8 2024-04-20T01:49:32.660 INFO:teuthology.orchestra.run.smithi044.stdout:90194313246 2024-04-20T01:49:32.729 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313247 got 90194313246 for osd.1 2024-04-20T01:49:32.745 INFO:teuthology.orchestra.run.smithi044.stdout:90194313247 2024-04-20T01:49:32.830 INFO:teuthology.orchestra.run.smithi044.stdout:90194313247 2024-04-20T01:49:32.831 INFO:teuthology.orchestra.run.smithi044.stdout:90194313245 2024-04-20T01:49:32.831 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313248 got 90194313247 for osd.0 2024-04-20T01:49:32.881 INFO:teuthology.orchestra.run.smithi044.stdout:90194313247 2024-04-20T01:49:32.901 INFO:teuthology.orchestra.run.smithi044.stdout:90194313245 2024-04-20T01:49:32.912 INFO:teuthology.orchestra.run.smithi044.stdout:94489280541 2024-04-20T01:49:32.912 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313246 got 90194313245 for osd.6 2024-04-20T01:49:32.925 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313248 got 90194313247 for osd.9 2024-04-20T01:49:32.939 INFO:teuthology.orchestra.run.smithi044.stdout:90194313246 2024-04-20T01:49:32.942 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313248 got 90194313247 for osd.4 2024-04-20T01:49:32.957 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313246 got 90194313245 for osd.11 2024-04-20T01:49:32.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280542 got 94489280541 for osd.13 2024-04-20T01:49:32.970 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313247 got 90194313246 for osd.12 2024-04-20T01:49:32.999 INFO:teuthology.orchestra.run.smithi044.stdout:90194313248 2024-04-20T01:49:33.022 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313249 got 90194313248 for osd.3 2024-04-20T01:49:33.054 INFO:teuthology.orchestra.run.smithi044.stdout:90194313247 2024-04-20T01:49:33.071 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313248 got 90194313247 for osd.10 2024-04-20T01:49:33.077 INFO:teuthology.orchestra.run.smithi044.stdout:90194313245 2024-04-20T01:49:33.094 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313246 got 90194313245 for osd.2 2024-04-20T01:49:33.095 INFO:teuthology.orchestra.run.smithi044.stdout:90194313246 2024-04-20T01:49:33.112 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313247 got 90194313246 for osd.15 2024-04-20T01:49:33.114 INFO:teuthology.orchestra.run.smithi044.stdout:438086664200 2024-04-20T01:49:33.131 INFO:tasks.ceph.ceph_manager.ceph:need seq 438086664201 got 438086664200 for osd.14 2024-04-20T01:49:33.526 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-20T01:49:33.599 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-20T01:49:33.619 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-04-20T01:49:33.730 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-20T01:49:33.833 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-20T01:49:33.914 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-20T01:49:33.926 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-04-20T01:49:33.944 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-20T01:49:33.946 INFO:teuthology.orchestra.run.smithi044.stdout:90194313247 2024-04-20T01:49:33.958 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-04-20T01:49:33.960 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-04-20T01:49:33.970 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-04-20T01:49:33.973 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313247 got 90194313247 for osd.5 2024-04-20T01:49:33.973 DEBUG:teuthology.parallel:result is None 2024-04-20T01:49:34.023 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-20T01:49:34.072 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-04-20T01:49:34.091 INFO:teuthology.orchestra.run.smithi044.stdout:90194313248 2024-04-20T01:49:34.095 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-20T01:49:34.113 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-04-20T01:49:34.132 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-04-20T01:49:34.181 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313248 got 90194313248 for osd.7 2024-04-20T01:49:34.181 DEBUG:teuthology.parallel:result is None 2024-04-20T01:49:34.189 INFO:teuthology.orchestra.run.smithi044.stdout:90194313247 2024-04-20T01:49:34.227 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313247 got 90194313247 for osd.8 2024-04-20T01:49:34.227 DEBUG:teuthology.parallel:result is None 2024-04-20T01:49:34.404 INFO:teuthology.orchestra.run.smithi044.stdout:90194313247 2024-04-20T01:49:34.447 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313247 got 90194313247 for osd.1 2024-04-20T01:49:34.447 DEBUG:teuthology.parallel:result is None 2024-04-20T01:49:34.606 INFO:teuthology.orchestra.run.smithi044.stdout:90194313248 2024-04-20T01:49:34.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313248 got 90194313248 for osd.0 2024-04-20T01:49:34.711 DEBUG:teuthology.parallel:result is None 2024-04-20T01:49:34.763 INFO:teuthology.orchestra.run.smithi044.stdout:90194313248 2024-04-20T01:49:34.774 INFO:teuthology.orchestra.run.smithi044.stdout:90194313246 2024-04-20T01:49:34.811 INFO:teuthology.orchestra.run.smithi044.stdout:90194313248 2024-04-20T01:49:34.832 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313246 got 90194313246 for osd.6 2024-04-20T01:49:34.832 DEBUG:teuthology.parallel:result is None 2024-04-20T01:49:34.858 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313248 got 90194313248 for osd.9 2024-04-20T01:49:34.858 DEBUG:teuthology.parallel:result is None 2024-04-20T01:49:34.940 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313248 got 90194313248 for osd.4 2024-04-20T01:49:34.941 DEBUG:teuthology.parallel:result is None 2024-04-20T01:49:34.950 INFO:teuthology.orchestra.run.smithi044.stdout:94489280543 2024-04-20T01:49:34.972 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280542 got 94489280543 for osd.13 2024-04-20T01:49:34.973 DEBUG:teuthology.parallel:result is None 2024-04-20T01:49:35.048 INFO:teuthology.orchestra.run.smithi044.stdout:90194313247 2024-04-20T01:49:35.065 INFO:teuthology.orchestra.run.smithi044.stdout:90194313246 2024-04-20T01:49:35.073 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313247 got 90194313247 for osd.12 2024-04-20T01:49:35.073 DEBUG:teuthology.parallel:result is None 2024-04-20T01:49:35.087 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313246 got 90194313246 for osd.11 2024-04-20T01:49:35.088 DEBUG:teuthology.parallel:result is None 2024-04-20T01:49:35.101 INFO:teuthology.orchestra.run.smithi044.stdout:90194313249 2024-04-20T01:49:35.118 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313249 got 90194313249 for osd.3 2024-04-20T01:49:35.118 DEBUG:teuthology.parallel:result is None 2024-04-20T01:49:35.150 INFO:teuthology.orchestra.run.smithi044.stdout:90194313247 2024-04-20T01:49:35.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313247 got 90194313247 for osd.15 2024-04-20T01:49:35.167 DEBUG:teuthology.parallel:result is None 2024-04-20T01:49:35.173 INFO:teuthology.orchestra.run.smithi044.stdout:90194313248 2024-04-20T01:49:35.189 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313248 got 90194313248 for osd.10 2024-04-20T01:49:35.189 DEBUG:teuthology.parallel:result is None 2024-04-20T01:49:35.211 INFO:teuthology.orchestra.run.smithi044.stdout:438086664202 2024-04-20T01:49:35.230 INFO:tasks.ceph.ceph_manager.ceph:need seq 438086664201 got 438086664202 for osd.14 2024-04-20T01:49:35.230 DEBUG:teuthology.parallel:result is None 2024-04-20T01:49:35.234 INFO:teuthology.orchestra.run.smithi044.stdout:90194313246 2024-04-20T01:49:35.251 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313246 got 90194313246 for osd.2 2024-04-20T01:49:35.251 DEBUG:teuthology.parallel:result is None 2024-04-20T01:49:35.251 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-20T01:49:35.251 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T01:49:35.661 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:49:35.662 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T01:49:35.674 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-04-20T01:49:35.166444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":504836,"kb_used_data":33284,"kb_used_omap":38,"kb_used_meta":471193,"kb_avail":1509444604,"statfs":{"total":1546188226560,"available":1545671274496,"internally_reserved":0,"allocated":34082816,"data_stored":20267840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39290,"internal_metadata":482502278},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":128,"apply_latency_ms":128,"commit_latency_ns":128000000,"apply_latency_ns":128000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001507"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":129,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.387881+0000","last_change":"2024-04-20T01:48:41.550435+0000","last_active":"2024-04-20T01:49:30.387881+0000","last_peered":"2024-04-20T01:49:30.387881+0000","last_clean":"2024-04-20T01:49:30.387881+0000","last_became_active":"2024-04-20T01:47:20.071615+0000","last_became_peered":"2024-04-20T01:47:20.071615+0000","last_unstale":"2024-04-20T01:49:30.387881+0000","last_undegraded":"2024-04-20T01:49:30.387881+0000","last_fullsized":"2024-04-20T01:49:30.387881+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:41.550289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:41.550289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:05.389116+0000","scrub_duration":0.0015320100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":130,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.098794+0000","last_change":"2024-04-20T01:48:36.889025+0000","last_active":"2024-04-20T01:49:30.098794+0000","last_peered":"2024-04-20T01:49:30.098794+0000","last_clean":"2024-04-20T01:49:30.098794+0000","last_became_active":"2024-04-20T01:47:20.075314+0000","last_became_peered":"2024-04-20T01:47:20.075314+0000","last_unstale":"2024-04-20T01:49:30.098794+0000","last_undegraded":"2024-04-20T01:49:30.098794+0000","last_fullsized":"2024-04-20T01:49:30.098794+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:36.888923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:36.888923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:45.012481+0000","scrub_duration":0.0018745420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,12],"acting":[2,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":126,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.379548+0000","last_change":"2024-04-20T01:48:26.145238+0000","last_active":"2024-04-20T01:49:30.379548+0000","last_peered":"2024-04-20T01:49:30.379548+0000","last_clean":"2024-04-20T01:49:30.379548+0000","last_became_active":"2024-04-20T01:47:20.038972+0000","last_became_peered":"2024-04-20T01:47:20.038972+0000","last_unstale":"2024-04-20T01:49:30.379548+0000","last_undegraded":"2024-04-20T01:49:30.379548+0000","last_fullsized":"2024-04-20T01:49:30.379548+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:26.145095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:26.145095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:54.362058+0000","scrub_duration":0.0015014830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":140,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.098342+0000","last_change":"2024-04-20T01:49:28.041091+0000","last_active":"2024-04-20T01:49:30.098342+0000","last_peered":"2024-04-20T01:49:30.098342+0000","last_clean":"2024-04-20T01:49:30.098342+0000","last_became_active":"2024-04-20T01:47:20.038764+0000","last_became_peered":"2024-04-20T01:47:20.038764+0000","last_unstale":"2024-04-20T01:49:30.098342+0000","last_undegraded":"2024-04-20T01:49:30.098342+0000","last_fullsized":"2024-04-20T01:49:30.098342+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:49:28.040983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:49:28.040983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:54.519868+0000","scrub_duration":0.0021762000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"25'157","reported_seq":324,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.379550+0000","last_change":"2024-04-20T01:48:36.239045+0000","last_active":"2024-04-20T01:49:30.379550+0000","last_peered":"2024-04-20T01:49:30.379550+0000","last_clean":"2024-04-20T01:49:30.379550+0000","last_became_active":"2024-04-20T01:47:17.302105+0000","last_became_peered":"2024-04-20T01:47:17.302105+0000","last_unstale":"2024-04-20T01:49:30.379550+0000","last_undegraded":"2024-04-20T01:49:30.379550+0000","last_fullsized":"2024-04-20T01:49:30.379550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'157","last_scrub_stamp":"2024-04-20T01:48:36.238909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_clean_scrub_stamp":"2024-04-20T01:48:36.238909+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:57.382841+0000","scrub_duration":0.0025722610000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,15],"acting":[13,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":82,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.098791+0000","last_change":"2024-04-20T01:49:17.016142+0000","last_active":"2024-04-20T01:49:30.098791+0000","last_peered":"2024-04-20T01:49:30.098791+0000","last_clean":"2024-04-20T01:49:30.098791+0000","last_became_active":"2024-04-20T01:49:17.015873+0000","last_became_peered":"2024-04-20T01:49:17.015873+0000","last_unstale":"2024-04-20T01:49:30.098791+0000","last_undegraded":"2024-04-20T01:49:30.098791+0000","last_fullsized":"2024-04-20T01:49:30.098791+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:37.682441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:37.682441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:54.853315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":130,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.379496+0000","last_change":"2024-04-20T01:48:48.595681+0000","last_active":"2024-04-20T01:49:30.379496+0000","last_peered":"2024-04-20T01:49:30.379496+0000","last_clean":"2024-04-20T01:49:30.379496+0000","last_became_active":"2024-04-20T01:47:20.038107+0000","last_became_peered":"2024-04-20T01:47:20.038107+0000","last_unstale":"2024-04-20T01:49:30.379496+0000","last_undegraded":"2024-04-20T01:49:30.379496+0000","last_fullsized":"2024-04-20T01:49:30.379496+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:48.595563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:48.595563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:00.568790+0000","scrub_duration":0.002598885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":128,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.996918+0000","last_change":"2024-04-20T01:48:48.578126+0000","last_active":"2024-04-20T01:49:30.996918+0000","last_peered":"2024-04-20T01:49:30.996918+0000","last_clean":"2024-04-20T01:49:30.996918+0000","last_became_active":"2024-04-20T01:47:20.110260+0000","last_became_peered":"2024-04-20T01:47:20.110260+0000","last_unstale":"2024-04-20T01:49:30.996918+0000","last_undegraded":"2024-04-20T01:49:30.996918+0000","last_fullsized":"2024-04-20T01:49:30.996918+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:48.577995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:48.577995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:53.805538+0000","scrub_duration":0.002163692,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":128,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.993729+0000","last_change":"2024-04-20T01:48:37.432504+0000","last_active":"2024-04-20T01:49:30.993729+0000","last_peered":"2024-04-20T01:49:30.993729+0000","last_clean":"2024-04-20T01:49:30.993729+0000","last_became_active":"2024-04-20T01:47:20.036328+0000","last_became_peered":"2024-04-20T01:47:20.036328+0000","last_unstale":"2024-04-20T01:49:30.993729+0000","last_undegraded":"2024-04-20T01:49:30.993729+0000","last_fullsized":"2024-04-20T01:49:30.993729+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:37.432331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:37.432331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:49.945535+0000","scrub_duration":0.0020345559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,13],"acting":[6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280543,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33652,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338188,"statfs":{"total":96636764160,"available":96602304512,"internally_reserved":0,"allocated":2588672,"data_stored":1726664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[2,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.45100000000000001}]},{"osd":2,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.59799999999999998,"15min":0.61099999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.74299999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.64000000000000001,"15min":0.65100000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84699999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.43099999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.47699999999999998}]},{"osd":4,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":1.0760000000000001,"15min":1.155},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.747,"5min":9.0129999999999999,"15min":9.0129999999999999},"last":0.625},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.65900000000000003,"15min":0.67500000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.747,"5min":1.022,"15min":1.022},"last":0.60099999999999998}]},{"osd":5,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.59199999999999997,"15min":0.60899999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73999999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.56899999999999995,"15min":0.58299999999999996},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72299999999999998,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.379}]},{"osd":6,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.52800000000000002}]},{"osd":7,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.755,"15min":0.76800000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95999999999999996,"5min":1.143,"15min":1.143},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.78100000000000003,"15min":0.80100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97999999999999998,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.51600000000000001}]},{"osd":10,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.497},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.54100000000000004}]},{"osd":11,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.55400000000000005}]},{"osd":12,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.77800000000000002,"15min":0.80000000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.83699999999999997,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":1.141},{"interface":"front","average":{"1min":0.746,"5min":1.196,"15min":1.2709999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93999999999999995,"5min":9.1389999999999993,"15min":9.1389999999999993},"last":0.82199999999999995}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.60999999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.96099999999999997,"15min":0.98999999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.99399999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.88300000000000001,"15min":0.90600000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.94699999999999995,"5min":1.399,"15min":1.399},"last":0.66300000000000003}]}]},{"osd":15,"up_from":21,"seq":90194313247,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29564,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342276,"statfs":{"total":96636764160,"available":96606490624,"internally_reserved":0,"allocated":2592768,"data_stored":1727267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,3,4,5,6,7,8,9,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.34100000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.67300000000000004,"15min":0.70299999999999996},"min":{"1min":0.36499999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72399999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":0.46700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":2,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.68000000000000005}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42399999999999999,"15min":0.42599999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.76100000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.441}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.38800000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.55000000000000004}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84299999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.69099999999999995}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.627,"15min":0.63400000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86499999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63,"15min":0.63300000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.94399999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.71699999999999997}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.93000000000000005,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94899999999999995,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.61099999999999999}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45900000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96099999999999997,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.86199999999999999,"15min":0.90000000000000002},"min":{"1min":0.43099999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97599999999999998,"5min":4.5979999999999999,"15min":4.5979999999999999},"last":0.59699999999999998}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68400000000000005,"15min":0.68700000000000006},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93899999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.748,"15min":0.746},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.016,"5min":1.272,"15min":1.272},"last":0.81499999999999995}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.73699999999999999}]}]},{"osd":14,"up_from":102,"seq":438086664202,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31288,"kb_used_data":2024,"kb_used_omap":15,"kb_used_meta":29232,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":2072576,"data_stored":1207324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15377,"internal_metadata":29934575},"hb_peers":[0,1,2,3,4,5,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":12,"up_from":21,"seq":90194313247,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29040,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342800,"statfs":{"total":96636764160,"available":96607027200,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":1,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.58499999999999996,"15min":0.55100000000000005},"min":{"1min":0.65800000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66800000000000004,"15min":0.66200000000000003},"min":{"1min":0.504,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93799999999999994,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.47099999999999997}]},{"osd":2,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.48499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.73999999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88700000000000001,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.38100000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79800000000000004,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.49399999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.39500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.59999999999999998,"15min":0.61199999999999999},"min":{"1min":0.371,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82399999999999995,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.66200000000000003}]},{"osd":5,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.65500000000000003,"15min":0.63700000000000001},"min":{"1min":0.55200000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0940000000000001,"5min":1.115,"15min":1.115},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.47499999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.0800000000000001,"5min":1.726,"15min":1.726},"last":0.77600000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.66200000000000003,"15min":0.66600000000000004},"min":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.86699999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.74299999999999999,"15min":0.72899999999999998},"min":{"1min":0.63700000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0349999999999999,"5min":1.069,"15min":1.069},"last":0.871}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.76300000000000001,"15min":0.76500000000000001},"min":{"1min":0.54400000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0529999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.95399999999999996,"15min":0.98999999999999999},"min":{"1min":0.53800000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.012,"5min":6.6779999999999999,"15min":6.6779999999999999},"last":0.60499999999999998}]},{"osd":13,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.79800000000000004},"min":{"1min":0.59699999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0189999999999999,"5min":1.827,"15min":1.827},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.80900000000000005,"15min":0.79500000000000004},"min":{"1min":0.753,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1739999999999999,"5min":1.258,"15min":1.258},"last":0.91300000000000003}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":1.135},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.99299999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.79000000000000004},"min":{"1min":0.58199999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0109999999999999,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.82999999999999996,"15min":0.81699999999999995},"min":{"1min":0.753,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.94899999999999995}]}]},{"osd":11,"up_from":21,"seq":90194313246,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33140,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,2,3,4,8,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.58599999999999997,"15min":0.59799999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.83899999999999997,"5min":1.669,"15min":1.669},"last":4.4279999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.67300000000000004,"15min":0.68300000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0900000000000001,"5min":1.75,"15min":1.75},"last":10.49}]},{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.58099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.40300000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.28799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.49099999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.88,"15min":0.93500000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1220000000000001,"5min":8.5969999999999995,"15min":8.5969999999999995},"last":10.468999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.67000000000000004,"15min":0.67900000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.147,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":10.507}]},{"osd":8,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":4.4909999999999997}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.57499999999999996,"15min":0.56200000000000006},"min":{"1min":0.38600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.47299999999999998}]},{"osd":10,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47199999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.50900000000000001}]},{"osd":12,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85699999999999998,"5min":1.875,"15min":1.875},"last":10.619},{"interface":"front","average":{"1min":0.73799999999999999,"5min":1.262,"15min":1.349},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1879999999999999,"5min":12.385,"15min":12.385},"last":10.706}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.54600000000000004}]},{"osd":15,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.75,"15min":0.76100000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.96999999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.61399999999999999}]}]},{"osd":10,"up_from":21,"seq":90194313248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33140,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,4,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.42899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":7,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63300000000000001,"15min":0.626},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.77300000000000002}]},{"osd":8,"last update":"Sat Apr 20 01:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.70399999999999996}]},{"osd":9,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68999999999999995,"15min":0.68500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.76400000000000001}]},{"osd":11,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.42499999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82099999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.90200000000000002,"15min":0.92600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.292,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":0.79000000000000004}]},{"osd":12,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":1.208,"15min":1.29},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1120000000000001,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.997,"15min":1.054},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91000000000000003,"5min":9.7759999999999998,"15min":9.7759999999999998},"last":0.66000000000000003}]},{"osd":13,"last update":"Sat Apr 20 01:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89600000000000002,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.71699999999999997,"15min":0.72299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95799999999999996,"5min":1.05,"15min":1.05},"last":0.749}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.64200000000000002}]},{"osd":15,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.80100000000000005,"15min":0.79700000000000004},"min":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.53400000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.93700000000000006,"5min":1.157,"15min":1.157},"last":0.88400000000000001}]}]},{"osd":9,"up_from":21,"seq":90194313248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":29044,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342796,"statfs":{"total":96636764160,"available":96607023104,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,2,4,6,7,8,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47099999999999997,"15min":0.46300000000000002},"min":{"1min":0.32800000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.52500000000000002,"15min":0.51300000000000001},"min":{"1min":0.40300000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83599999999999997,"5min":0.998,"15min":0.998},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.59399999999999997}]},{"osd":6,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.872,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55700000000000005}]},{"osd":7,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.81000000000000005}]},{"osd":8,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.39600000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.67000000000000004}]},{"osd":10,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.93799999999999994,"5min":1.03,"15min":1.03},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58699999999999997,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76300000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.73899999999999999}]},{"osd":11,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88400000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.81999999999999995}]},{"osd":12,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.74099999999999999,"15min":0.73899999999999999},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97899999999999998,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.89900000000000002,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.85199999999999998}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.56000000000000005}]},{"osd":15,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.745,"15min":0.753},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69699999999999995,"15min":0.70499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88800000000000001,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.76800000000000002}]}]},{"osd":8,"up_from":21,"seq":90194313247,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":2076672,"data_stored":1207927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.34200000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89700000000000002,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.628,"15min":0.64100000000000001},"min":{"1min":0.39500000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.81699999999999995,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.58199999999999996}]},{"osd":3,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.45400000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85799999999999998,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.73599999999999999,"15min":0.753},"min":{"1min":0.44500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91600000000000004,"5min":1.821,"15min":1.821},"last":0.80200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":6,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70199999999999996,"15min":0.70999999999999996},"min":{"1min":0.46500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89900000000000002,"5min":1.784,"15min":1.784},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.44,"5min":0.437,"15min":0.437},"max":{"1min":0.79300000000000004,"5min":1.885,"15min":1.885},"last":0.54700000000000004}]},{"osd":7,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.72599999999999998,"15min":0.73499999999999999},"min":{"1min":0.50800000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93300000000000005,"5min":1.411,"15min":1.411},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.81999999999999995,"15min":0.84399999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95199999999999996,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.75700000000000001}]},{"osd":9,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.72199999999999998,"15min":0.73599999999999999},"min":{"1min":0.436,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91800000000000004,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.139,"5min":1.375,"15min":1.375},"last":0.78200000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":11,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.83899999999999997,"15min":0.84299999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.141,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.92400000000000004,"15min":0.93999999999999995},"min":{"1min":0.66100000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.071,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.65600000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":13,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.63},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.73099999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":15,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.88700000000000001,"15min":0.89100000000000001},"min":{"1min":0.621,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.095,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.95399999999999996,"15min":0.98399999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0229999999999999,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.85199999999999998}]}]},{"osd":7,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29032,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342808,"statfs":{"total":96636764160,"available":96607035392,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,3,4,6,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.38800000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.67000000000000004}]},{"osd":3,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.45900000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.379,"5min":0.372,"15min":0.372},"max":{"1min":0.92600000000000005,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.47199999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94399999999999995,"5min":4.1920000000000002,"15min":4.1920000000000002},"last":0.63800000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.751},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.68899999999999995}]},{"osd":8,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.78900000000000003,"15min":0.81499999999999995},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82599999999999996,"5min":5.7679999999999998,"15min":5.7679999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":1.0189999999999999,"15min":1.077},"min":{"1min":0.46300000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96599999999999997,"5min":9.952,"15min":9.952},"last":0.56799999999999995}]},{"osd":9,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.438,"5min":0.433,"15min":0.433},"max":{"1min":0.751,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.498,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88800000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.65400000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82899999999999996,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.88800000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.57999999999999996}]},{"osd":12,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.84399999999999997,"15min":0.86099999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.99199999999999999,"5min":4.2720000000000002,"15min":4.2720000000000002},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68700000000000006,"15min":0.68000000000000005},"min":{"1min":0.50900000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90300000000000002,"5min":0.995,"15min":0.995},"last":0.77000000000000002}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.79700000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":6,"up_from":21,"seq":90194313246,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":2076672,"data_stored":1207927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[3,4,5,7,8,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.5}]},{"osd":2,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.53700000000000003}]},{"osd":3,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.45600000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":5.141},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":8.0090000000000003}]},{"osd":5,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.38300000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.91700000000000004,"5min":1.083,"15min":1.083},"last":0.59999999999999998}]},{"osd":7,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59999999999999998,"15min":0.60399999999999998},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77600000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.626},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66100000000000003,"15min":0.66400000000000003},"min":{"1min":0.36499999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.89400000000000002,"5min":1.056,"15min":1.056},"last":0.56399999999999995}]},{"osd":8,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.755,"15min":0.77000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88300000000000001,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":6.1130000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.80100000000000005,"15min":0.81200000000000006},"min":{"1min":0.53800000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92000000000000004,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":6.0839999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":10,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67900000000000005,"15min":0.68700000000000006},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92500000000000004,"5min":1.129,"15min":1.129},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.67700000000000005,"15min":0.69099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88,"5min":1.272,"15min":1.272},"last":0.51400000000000001}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.92100000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.78100000000000003,"15min":0.79000000000000004},"min":{"1min":0.52200000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96599999999999997,"5min":1.167,"15min":1.167},"last":0.72099999999999997}]},{"osd":12,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.83499999999999996,"15min":0.84799999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.085,"5min":1.444,"15min":1.444},"last":5.351},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84899999999999998,"15min":0.84699999999999998},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.0089999999999999}]},{"osd":13,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.81699999999999995,"15min":0.82299999999999995},"min":{"1min":0.56899999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0329999999999999,"5min":1.482,"15min":1.482},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.753,"15min":0.76000000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94999999999999996,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.70199999999999996}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.65700000000000003}]},{"osd":15,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.85999999999999999,"15min":0.87},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0049999999999999,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97599999999999998,"5min":1.327,"15min":1.327},"last":0.65200000000000002}]}]},{"osd":1,"up_from":21,"seq":90194313247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33140,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,4,5,6,7,8,9,10,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.75600000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67400000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.41999999999999998}]},{"osd":2,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70199999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79600000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61799999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.76000000000000001}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67800000000000005,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72099999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.39000000000000001}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.497,"15min":0.5},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.499,"15min":0.50700000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.33300000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82599999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.58099999999999996}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66800000000000004,"15min":0.66300000000000003},"min":{"1min":0.48099999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65900000000000003,"15min":0.64900000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.59699999999999998}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.88300000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.72199999999999998,"15min":0.73999999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84899999999999998,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.48099999999999998}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.83499999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.435},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67800000000000005,"15min":0.68400000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92900000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.46200000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.70899999999999996}]}]},{"osd":0,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":2056192,"data_stored":1197753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64500000000000002,"15min":0.63400000000000001},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.76200000000000001}]},{"osd":2,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.90200000000000002,"15min":0.95599999999999996},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.89100000000000001,"5min":8.0619999999999994,"15min":8.0619999999999994},"last":0.626},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.1399999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.53300000000000003}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.80500000000000005,"5min":3.363,"15min":3.363},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.90000000000000002,"15min":0.93400000000000005},"min":{"1min":0.50800000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.998,"5min":7.9710000000000001,"15min":7.9710000000000001},"last":0.878}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66000000000000003,"15min":0.66500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97599999999999998,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90200000000000002,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":0.66700000000000004}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.94199999999999995,"15min":0.98699999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1779999999999999,"5min":8.0239999999999991,"15min":8.0239999999999991},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1739999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.90500000000000003}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.72999999999999998,"15min":0.748},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86299999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.98399999999999999,"15min":1.0229999999999999},"min":{"1min":0.42899999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1200000000000001,"5min":7.8630000000000004,"15min":7.8630000000000004},"last":0.91900000000000004}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.67800000000000005,"15min":0.69499999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.92100000000000004,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.95799999999999996,"15min":1.02},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.93400000000000005,"5min":8.1189999999999998,"15min":8.1189999999999998},"last":0.95099999999999996}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.81200000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.01,"5min":3.5699999999999998,"15min":3.5699999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.55300000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.044,"5min":1.927,"15min":1.927},"last":1.0209999999999999}]},{"osd":13,"last update":"Sat Apr 20 01:48:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.84899999999999998}]},{"osd":15,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.097,"5min":1.833,"15min":1.833},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.86899999999999999,"15min":0.88500000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96699999999999997,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":0.97099999999999997}]}]},{"osd":2,"up_from":21,"seq":90194313246,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29044,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342796,"statfs":{"total":96636764160,"available":96607023104,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.42799999999999999}]},{"osd":1,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54600000000000004,"15min":0.53900000000000003},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.72199999999999998}]},{"osd":3,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82199999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":7,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82799999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":0.60299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72299999999999998,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.499}]},{"osd":8,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65000000000000002,"15min":0.66100000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90100000000000002,"5min":2.512,"15min":2.512},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.68400000000000005,"15min":0.69599999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.77600000000000002,"5min":2.532,"15min":2.532},"last":0.63300000000000001}]},{"osd":9,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.435,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64800000000000002},"min":{"1min":0.43099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93000000000000005,"5min":1.107,"15min":1.107},"last":0.65500000000000003}]},{"osd":11,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0269999999999999,"5min":1.135,"15min":1.135},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68600000000000005,"15min":0.69399999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88900000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70299999999999996}]},{"osd":12,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.75800000000000001,"15min":0.78500000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.872,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.80200000000000005,"15min":0.82899999999999996},"min":{"1min":0.42499999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93899999999999995,"5min":3.492,"15min":3.492},"last":0.59099999999999997}]},{"osd":13,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.748}]},{"osd":15,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.60699999999999998}]}]},{"osd":3,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33140,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,4,5,6,7,8,9,10,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.374}]},{"osd":2,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.36199999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.90500000000000003,"5min":0.996,"15min":0.996},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.85599999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.49199999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.56499999999999995,"15min":0.56899999999999995},"min":{"1min":0.34000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69399999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.94599999999999995,"15min":1.014},"min":{"1min":0.39000000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83899999999999997,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":0.61599999999999999}]},{"osd":5,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.56299999999999994,"15min":0.57199999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66500000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.317},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55600000000000005,"15min":0.56000000000000005},"min":{"1min":0.36399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78700000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.44500000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.38600000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89300000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.432}]},{"osd":7,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66200000000000003,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.54800000000000004,"15min":0.56000000000000005},"min":{"1min":0.30099999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71299999999999997,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.41099999999999998}]},{"osd":8,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.41199999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85099999999999998,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.0780000000000001,"15min":1.1479999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81999999999999995,"5min":9.4700000000000006,"15min":9.4700000000000006},"last":0.54800000000000004}]},{"osd":9,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.622,"15min":0.625},"min":{"1min":0.45200000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84699999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.71199999999999997,"15min":0.72599999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74399999999999999,"5min":1.306,"15min":1.306},"last":0.60399999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":12,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":10.959,"5min":10.959,"15min":10.959},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.49399999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.63800000000000001}]}]},{"osd":4,"up_from":21,"seq":90194313248,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":2076672,"data_stored":1207927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,5,6,7,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.81000000000000005,"15min":0.87},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.93400000000000005,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.46700000000000003,"5min":1.1579999999999999,"15min":1.2729999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.008,"5min":16.27,"15min":16.27},"last":0.38800000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":1.012,"15min":1.081},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.92700000000000005,"5min":10.507999999999999,"15min":10.507999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":1.1559999999999999,"15min":1.23},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6319999999999999,"5min":11.019,"15min":11.019},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":3,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.997,"15min":1.0469999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4750000000000001,"5min":4.9180000000000001,"15min":4.9180000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":1.0449999999999999,"15min":1.095},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.6000000000000001,"5min":10.619,"15min":10.619},"last":0.59699999999999998}]},{"osd":5,"last update":"Sat Apr 20 01:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":1.099,"15min":1.169},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.673,"5min":10.895,"15min":10.895},"last":0.622},{"interface":"front","average":{"1min":0.64200000000000002,"5min":1.012,"15min":1.0740000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1910000000000001,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":0.498}]},{"osd":6,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":1.1890000000000001,"15min":1.2649999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1719999999999999,"5min":11.051,"15min":11.051},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":1.1279999999999999,"15min":1.2},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96199999999999997,"5min":10.589,"15min":10.589},"last":0.76600000000000001}]},{"osd":7,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":1.1950000000000001,"15min":1.2649999999999999},"min":{"1min":0.56000000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.571,"5min":10.957000000000001,"15min":10.957000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.79400000000000004,"5min":1.1799999999999999,"15min":1.244},"min":{"1min":0.47299999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3080000000000001,"5min":10.656000000000001,"15min":10.656000000000001},"last":0.66200000000000003}]},{"osd":8,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.79,"5min":3.79,"15min":3.79},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.0269999999999999}]},{"osd":9,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.385,"15min":1.385},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":10.929,"5min":10.929,"15min":10.929},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":10.555999999999999,"5min":10.555999999999999,"15min":10.555999999999999},"last":1.077}]},{"osd":10,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":1.149,"15min":1.204},"min":{"1min":0.57799999999999996,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.331,"5min":5.0229999999999997,"15min":5.0229999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":1.2729999999999999,"15min":1.351},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2330000000000001,"5min":10.69,"15min":10.69},"last":0.69899999999999995}]},{"osd":11,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.3600000000000001,"15min":1.4410000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.448,"5min":10.987,"15min":10.987},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":1.1879999999999999,"15min":1.244},"min":{"1min":0.63700000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.2290000000000001,"5min":4.9900000000000002,"15min":4.9900000000000002},"last":0.63500000000000001}]},{"osd":12,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.3240000000000001,"15min":1.4059999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.181,"5min":10.720000000000001,"15min":10.720000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.85999999999999999,"5min":1.589,"15min":1.7110000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3149999999999999,"5min":16.617000000000001,"15min":16.617000000000001},"last":0.70899999999999996}]},{"osd":13,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":3.8999999999999999,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":0.745},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":4.5019999999999998,"5min":4.5019999999999998,"15min":4.5019999999999998},"last":0.71799999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]},{"osd":5,"up_from":21,"seq":90194313247,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29044,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342796,"statfs":{"total":96636764160,"available":96607023104,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,8,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.85699999999999998,"15min":0.92100000000000004},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77600000000000002,"5min":10.423999999999999,"15min":10.423999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.92500000000000004,"15min":0.98399999999999999},"min":{"1min":0.374,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73499999999999999,"5min":10.465,"15min":10.465},"last":0.76900000000000002}]},{"osd":2,"last update":"Sat Apr 20 01:49:22 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.33600000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.42799999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.755},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.65900000000000003}]},{"osd":4,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.81999999999999995,"15min":0.86799999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71499999999999997,"5min":8.5269999999999992,"15min":8.5269999999999992},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60999999999999999,"15min":0.61699999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78100000000000003,"5min":2.79,"15min":2.79},"last":0.48699999999999999}]},{"osd":6,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84599999999999997,"5min":1.115,"15min":1.115},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.63100000000000001},"min":{"1min":0.38900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94599999999999995,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.51800000000000002}]},{"osd":7,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78800000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.626},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.49399999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.63800000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":10,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.91300000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.54500000000000004}]},{"osd":11,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71899999999999997,"15min":0.71199999999999997},"min":{"1min":0.53900000000000003,"5min":0.435,"15min":0.435},"max":{"1min":1.242,"5min":1.956,"15min":1.956},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70299999999999996,"15min":0.69199999999999995},"min":{"1min":0.55000000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.72599999999999998}]},{"osd":12,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":1.052,"15min":1.1080000000000001},"min":{"1min":0.55800000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90600000000000003,"5min":10.616,"15min":10.616},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.755,"5min":1.0900000000000001,"15min":1.1459999999999999},"min":{"1min":0.56999999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.98899999999999999,"5min":10.629,"15min":10.629},"last":0.82499999999999996}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.84699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T01:49:35.715 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T01:49:36.089 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:49:36.091 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T01:49:36.104 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-04-20T01:49:35.166444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":504836,"kb_used_data":33284,"kb_used_omap":38,"kb_used_meta":471193,"kb_avail":1509444604,"statfs":{"total":1546188226560,"available":1545671274496,"internally_reserved":0,"allocated":34082816,"data_stored":20267840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39290,"internal_metadata":482502278},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":128,"apply_latency_ms":128,"commit_latency_ns":128000000,"apply_latency_ns":128000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001507"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":129,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.387881+0000","last_change":"2024-04-20T01:48:41.550435+0000","last_active":"2024-04-20T01:49:30.387881+0000","last_peered":"2024-04-20T01:49:30.387881+0000","last_clean":"2024-04-20T01:49:30.387881+0000","last_became_active":"2024-04-20T01:47:20.071615+0000","last_became_peered":"2024-04-20T01:47:20.071615+0000","last_unstale":"2024-04-20T01:49:30.387881+0000","last_undegraded":"2024-04-20T01:49:30.387881+0000","last_fullsized":"2024-04-20T01:49:30.387881+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:41.550289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:41.550289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:05.389116+0000","scrub_duration":0.0015320100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":130,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.098794+0000","last_change":"2024-04-20T01:48:36.889025+0000","last_active":"2024-04-20T01:49:30.098794+0000","last_peered":"2024-04-20T01:49:30.098794+0000","last_clean":"2024-04-20T01:49:30.098794+0000","last_became_active":"2024-04-20T01:47:20.075314+0000","last_became_peered":"2024-04-20T01:47:20.075314+0000","last_unstale":"2024-04-20T01:49:30.098794+0000","last_undegraded":"2024-04-20T01:49:30.098794+0000","last_fullsized":"2024-04-20T01:49:30.098794+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:36.888923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:36.888923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:45.012481+0000","scrub_duration":0.0018745420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,12],"acting":[2,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":126,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.379548+0000","last_change":"2024-04-20T01:48:26.145238+0000","last_active":"2024-04-20T01:49:30.379548+0000","last_peered":"2024-04-20T01:49:30.379548+0000","last_clean":"2024-04-20T01:49:30.379548+0000","last_became_active":"2024-04-20T01:47:20.038972+0000","last_became_peered":"2024-04-20T01:47:20.038972+0000","last_unstale":"2024-04-20T01:49:30.379548+0000","last_undegraded":"2024-04-20T01:49:30.379548+0000","last_fullsized":"2024-04-20T01:49:30.379548+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:26.145095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:26.145095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:54.362058+0000","scrub_duration":0.0015014830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":140,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.098342+0000","last_change":"2024-04-20T01:49:28.041091+0000","last_active":"2024-04-20T01:49:30.098342+0000","last_peered":"2024-04-20T01:49:30.098342+0000","last_clean":"2024-04-20T01:49:30.098342+0000","last_became_active":"2024-04-20T01:47:20.038764+0000","last_became_peered":"2024-04-20T01:47:20.038764+0000","last_unstale":"2024-04-20T01:49:30.098342+0000","last_undegraded":"2024-04-20T01:49:30.098342+0000","last_fullsized":"2024-04-20T01:49:30.098342+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:49:28.040983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:49:28.040983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:54.519868+0000","scrub_duration":0.0021762000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"25'157","reported_seq":324,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.379550+0000","last_change":"2024-04-20T01:48:36.239045+0000","last_active":"2024-04-20T01:49:30.379550+0000","last_peered":"2024-04-20T01:49:30.379550+0000","last_clean":"2024-04-20T01:49:30.379550+0000","last_became_active":"2024-04-20T01:47:17.302105+0000","last_became_peered":"2024-04-20T01:47:17.302105+0000","last_unstale":"2024-04-20T01:49:30.379550+0000","last_undegraded":"2024-04-20T01:49:30.379550+0000","last_fullsized":"2024-04-20T01:49:30.379550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'157","last_scrub_stamp":"2024-04-20T01:48:36.238909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_clean_scrub_stamp":"2024-04-20T01:48:36.238909+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:57.382841+0000","scrub_duration":0.0025722610000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,15],"acting":[13,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":82,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.098791+0000","last_change":"2024-04-20T01:49:17.016142+0000","last_active":"2024-04-20T01:49:30.098791+0000","last_peered":"2024-04-20T01:49:30.098791+0000","last_clean":"2024-04-20T01:49:30.098791+0000","last_became_active":"2024-04-20T01:49:17.015873+0000","last_became_peered":"2024-04-20T01:49:17.015873+0000","last_unstale":"2024-04-20T01:49:30.098791+0000","last_undegraded":"2024-04-20T01:49:30.098791+0000","last_fullsized":"2024-04-20T01:49:30.098791+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:37.682441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:37.682441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:54.853315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":130,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.379496+0000","last_change":"2024-04-20T01:48:48.595681+0000","last_active":"2024-04-20T01:49:30.379496+0000","last_peered":"2024-04-20T01:49:30.379496+0000","last_clean":"2024-04-20T01:49:30.379496+0000","last_became_active":"2024-04-20T01:47:20.038107+0000","last_became_peered":"2024-04-20T01:47:20.038107+0000","last_unstale":"2024-04-20T01:49:30.379496+0000","last_undegraded":"2024-04-20T01:49:30.379496+0000","last_fullsized":"2024-04-20T01:49:30.379496+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:48.595563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:48.595563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:00.568790+0000","scrub_duration":0.002598885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":128,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.996918+0000","last_change":"2024-04-20T01:48:48.578126+0000","last_active":"2024-04-20T01:49:30.996918+0000","last_peered":"2024-04-20T01:49:30.996918+0000","last_clean":"2024-04-20T01:49:30.996918+0000","last_became_active":"2024-04-20T01:47:20.110260+0000","last_became_peered":"2024-04-20T01:47:20.110260+0000","last_unstale":"2024-04-20T01:49:30.996918+0000","last_undegraded":"2024-04-20T01:49:30.996918+0000","last_fullsized":"2024-04-20T01:49:30.996918+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:48.577995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:48.577995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:53.805538+0000","scrub_duration":0.002163692,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":128,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.993729+0000","last_change":"2024-04-20T01:48:37.432504+0000","last_active":"2024-04-20T01:49:30.993729+0000","last_peered":"2024-04-20T01:49:30.993729+0000","last_clean":"2024-04-20T01:49:30.993729+0000","last_became_active":"2024-04-20T01:47:20.036328+0000","last_became_peered":"2024-04-20T01:47:20.036328+0000","last_unstale":"2024-04-20T01:49:30.993729+0000","last_undegraded":"2024-04-20T01:49:30.993729+0000","last_fullsized":"2024-04-20T01:49:30.993729+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:37.432331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:37.432331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:49.945535+0000","scrub_duration":0.0020345559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,13],"acting":[6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280543,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33652,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338188,"statfs":{"total":96636764160,"available":96602304512,"internally_reserved":0,"allocated":2588672,"data_stored":1726664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[2,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.45100000000000001}]},{"osd":2,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.59799999999999998,"15min":0.61099999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.74299999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.64000000000000001,"15min":0.65100000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84699999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.43099999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.47699999999999998}]},{"osd":4,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":1.0760000000000001,"15min":1.155},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.747,"5min":9.0129999999999999,"15min":9.0129999999999999},"last":0.625},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.65900000000000003,"15min":0.67500000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.747,"5min":1.022,"15min":1.022},"last":0.60099999999999998}]},{"osd":5,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.59199999999999997,"15min":0.60899999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73999999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.56899999999999995,"15min":0.58299999999999996},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72299999999999998,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.379}]},{"osd":6,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.52800000000000002}]},{"osd":7,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.755,"15min":0.76800000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95999999999999996,"5min":1.143,"15min":1.143},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.78100000000000003,"15min":0.80100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97999999999999998,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.51600000000000001}]},{"osd":10,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.497},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.54100000000000004}]},{"osd":11,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.55400000000000005}]},{"osd":12,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.77800000000000002,"15min":0.80000000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.83699999999999997,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":1.141},{"interface":"front","average":{"1min":0.746,"5min":1.196,"15min":1.2709999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93999999999999995,"5min":9.1389999999999993,"15min":9.1389999999999993},"last":0.82199999999999995}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.60999999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.96099999999999997,"15min":0.98999999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.99399999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.88300000000000001,"15min":0.90600000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.94699999999999995,"5min":1.399,"15min":1.399},"last":0.66300000000000003}]}]},{"osd":15,"up_from":21,"seq":90194313247,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29564,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342276,"statfs":{"total":96636764160,"available":96606490624,"internally_reserved":0,"allocated":2592768,"data_stored":1727267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,3,4,5,6,7,8,9,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.34100000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.67300000000000004,"15min":0.70299999999999996},"min":{"1min":0.36499999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72399999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":0.46700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":2,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.68000000000000005}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42399999999999999,"15min":0.42599999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.76100000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.441}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.38800000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.55000000000000004}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84299999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.69099999999999995}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.627,"15min":0.63400000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86499999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63,"15min":0.63300000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.94399999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.71699999999999997}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.93000000000000005,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94899999999999995,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.61099999999999999}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45900000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96099999999999997,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.86199999999999999,"15min":0.90000000000000002},"min":{"1min":0.43099999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97599999999999998,"5min":4.5979999999999999,"15min":4.5979999999999999},"last":0.59699999999999998}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68400000000000005,"15min":0.68700000000000006},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93899999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.748,"15min":0.746},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.016,"5min":1.272,"15min":1.272},"last":0.81499999999999995}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.73699999999999999}]}]},{"osd":14,"up_from":102,"seq":438086664202,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31288,"kb_used_data":2024,"kb_used_omap":15,"kb_used_meta":29232,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":2072576,"data_stored":1207324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15377,"internal_metadata":29934575},"hb_peers":[0,1,2,3,4,5,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":12,"up_from":21,"seq":90194313247,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29040,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342800,"statfs":{"total":96636764160,"available":96607027200,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":1,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.58499999999999996,"15min":0.55100000000000005},"min":{"1min":0.65800000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66800000000000004,"15min":0.66200000000000003},"min":{"1min":0.504,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93799999999999994,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.47099999999999997}]},{"osd":2,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.48499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.73999999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88700000000000001,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.38100000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79800000000000004,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.49399999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.39500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.59999999999999998,"15min":0.61199999999999999},"min":{"1min":0.371,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82399999999999995,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.66200000000000003}]},{"osd":5,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.65500000000000003,"15min":0.63700000000000001},"min":{"1min":0.55200000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0940000000000001,"5min":1.115,"15min":1.115},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.47499999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.0800000000000001,"5min":1.726,"15min":1.726},"last":0.77600000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.66200000000000003,"15min":0.66600000000000004},"min":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.86699999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.74299999999999999,"15min":0.72899999999999998},"min":{"1min":0.63700000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0349999999999999,"5min":1.069,"15min":1.069},"last":0.871}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.76300000000000001,"15min":0.76500000000000001},"min":{"1min":0.54400000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0529999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.95399999999999996,"15min":0.98999999999999999},"min":{"1min":0.53800000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.012,"5min":6.6779999999999999,"15min":6.6779999999999999},"last":0.60499999999999998}]},{"osd":13,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.79800000000000004},"min":{"1min":0.59699999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0189999999999999,"5min":1.827,"15min":1.827},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.80900000000000005,"15min":0.79500000000000004},"min":{"1min":0.753,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1739999999999999,"5min":1.258,"15min":1.258},"last":0.91300000000000003}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":1.135},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.99299999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.79000000000000004},"min":{"1min":0.58199999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0109999999999999,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.82999999999999996,"15min":0.81699999999999995},"min":{"1min":0.753,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.94899999999999995}]}]},{"osd":11,"up_from":21,"seq":90194313246,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33140,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,2,3,4,8,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.58599999999999997,"15min":0.59799999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.83899999999999997,"5min":1.669,"15min":1.669},"last":4.4279999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.67300000000000004,"15min":0.68300000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0900000000000001,"5min":1.75,"15min":1.75},"last":10.49}]},{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.58099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.40300000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.28799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.49099999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.88,"15min":0.93500000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1220000000000001,"5min":8.5969999999999995,"15min":8.5969999999999995},"last":10.468999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.67000000000000004,"15min":0.67900000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.147,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":10.507}]},{"osd":8,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":4.4909999999999997}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.57499999999999996,"15min":0.56200000000000006},"min":{"1min":0.38600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.47299999999999998}]},{"osd":10,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47199999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.50900000000000001}]},{"osd":12,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85699999999999998,"5min":1.875,"15min":1.875},"last":10.619},{"interface":"front","average":{"1min":0.73799999999999999,"5min":1.262,"15min":1.349},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1879999999999999,"5min":12.385,"15min":12.385},"last":10.706}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.54600000000000004}]},{"osd":15,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.75,"15min":0.76100000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.96999999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.61399999999999999}]}]},{"osd":10,"up_from":21,"seq":90194313248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33140,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,4,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.42899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":7,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63300000000000001,"15min":0.626},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.77300000000000002}]},{"osd":8,"last update":"Sat Apr 20 01:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.70399999999999996}]},{"osd":9,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68999999999999995,"15min":0.68500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.76400000000000001}]},{"osd":11,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.42499999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82099999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.90200000000000002,"15min":0.92600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.292,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":0.79000000000000004}]},{"osd":12,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":1.208,"15min":1.29},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1120000000000001,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.997,"15min":1.054},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91000000000000003,"5min":9.7759999999999998,"15min":9.7759999999999998},"last":0.66000000000000003}]},{"osd":13,"last update":"Sat Apr 20 01:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89600000000000002,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.71699999999999997,"15min":0.72299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95799999999999996,"5min":1.05,"15min":1.05},"last":0.749}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.64200000000000002}]},{"osd":15,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.80100000000000005,"15min":0.79700000000000004},"min":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.53400000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.93700000000000006,"5min":1.157,"15min":1.157},"last":0.88400000000000001}]}]},{"osd":9,"up_from":21,"seq":90194313248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":29044,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342796,"statfs":{"total":96636764160,"available":96607023104,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,2,4,6,7,8,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47099999999999997,"15min":0.46300000000000002},"min":{"1min":0.32800000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.52500000000000002,"15min":0.51300000000000001},"min":{"1min":0.40300000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83599999999999997,"5min":0.998,"15min":0.998},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.59399999999999997}]},{"osd":6,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.872,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55700000000000005}]},{"osd":7,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.81000000000000005}]},{"osd":8,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.39600000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.67000000000000004}]},{"osd":10,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.93799999999999994,"5min":1.03,"15min":1.03},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58699999999999997,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76300000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.73899999999999999}]},{"osd":11,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88400000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.81999999999999995}]},{"osd":12,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.74099999999999999,"15min":0.73899999999999999},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97899999999999998,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.89900000000000002,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.85199999999999998}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.56000000000000005}]},{"osd":15,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.745,"15min":0.753},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69699999999999995,"15min":0.70499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88800000000000001,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.76800000000000002}]}]},{"osd":8,"up_from":21,"seq":90194313247,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":2076672,"data_stored":1207927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.34200000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89700000000000002,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.628,"15min":0.64100000000000001},"min":{"1min":0.39500000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.81699999999999995,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.58199999999999996}]},{"osd":3,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.45400000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85799999999999998,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.73599999999999999,"15min":0.753},"min":{"1min":0.44500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91600000000000004,"5min":1.821,"15min":1.821},"last":0.80200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":6,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70199999999999996,"15min":0.70999999999999996},"min":{"1min":0.46500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89900000000000002,"5min":1.784,"15min":1.784},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.44,"5min":0.437,"15min":0.437},"max":{"1min":0.79300000000000004,"5min":1.885,"15min":1.885},"last":0.54700000000000004}]},{"osd":7,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.72599999999999998,"15min":0.73499999999999999},"min":{"1min":0.50800000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93300000000000005,"5min":1.411,"15min":1.411},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.81999999999999995,"15min":0.84399999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95199999999999996,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.75700000000000001}]},{"osd":9,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.72199999999999998,"15min":0.73599999999999999},"min":{"1min":0.436,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91800000000000004,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.139,"5min":1.375,"15min":1.375},"last":0.78200000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":11,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.83899999999999997,"15min":0.84299999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.141,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.92400000000000004,"15min":0.93999999999999995},"min":{"1min":0.66100000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.071,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.65600000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":13,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.63},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.73099999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":15,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.88700000000000001,"15min":0.89100000000000001},"min":{"1min":0.621,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.095,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.95399999999999996,"15min":0.98399999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0229999999999999,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.85199999999999998}]}]},{"osd":7,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29032,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342808,"statfs":{"total":96636764160,"available":96607035392,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,3,4,6,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.38800000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.67000000000000004}]},{"osd":3,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.45900000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.379,"5min":0.372,"15min":0.372},"max":{"1min":0.92600000000000005,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.47199999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94399999999999995,"5min":4.1920000000000002,"15min":4.1920000000000002},"last":0.63800000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.751},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.68899999999999995}]},{"osd":8,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.78900000000000003,"15min":0.81499999999999995},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82599999999999996,"5min":5.7679999999999998,"15min":5.7679999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":1.0189999999999999,"15min":1.077},"min":{"1min":0.46300000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96599999999999997,"5min":9.952,"15min":9.952},"last":0.56799999999999995}]},{"osd":9,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.438,"5min":0.433,"15min":0.433},"max":{"1min":0.751,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.498,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88800000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.65400000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82899999999999996,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.88800000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.57999999999999996}]},{"osd":12,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.84399999999999997,"15min":0.86099999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.99199999999999999,"5min":4.2720000000000002,"15min":4.2720000000000002},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68700000000000006,"15min":0.68000000000000005},"min":{"1min":0.50900000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90300000000000002,"5min":0.995,"15min":0.995},"last":0.77000000000000002}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.79700000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":6,"up_from":21,"seq":90194313246,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":2076672,"data_stored":1207927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[3,4,5,7,8,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.5}]},{"osd":2,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.53700000000000003}]},{"osd":3,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.45600000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":5.141},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":8.0090000000000003}]},{"osd":5,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.38300000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.91700000000000004,"5min":1.083,"15min":1.083},"last":0.59999999999999998}]},{"osd":7,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59999999999999998,"15min":0.60399999999999998},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77600000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.626},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66100000000000003,"15min":0.66400000000000003},"min":{"1min":0.36499999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.89400000000000002,"5min":1.056,"15min":1.056},"last":0.56399999999999995}]},{"osd":8,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.755,"15min":0.77000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88300000000000001,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":6.1130000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.80100000000000005,"15min":0.81200000000000006},"min":{"1min":0.53800000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92000000000000004,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":6.0839999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":10,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67900000000000005,"15min":0.68700000000000006},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92500000000000004,"5min":1.129,"15min":1.129},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.67700000000000005,"15min":0.69099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88,"5min":1.272,"15min":1.272},"last":0.51400000000000001}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.92100000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.78100000000000003,"15min":0.79000000000000004},"min":{"1min":0.52200000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96599999999999997,"5min":1.167,"15min":1.167},"last":0.72099999999999997}]},{"osd":12,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.83499999999999996,"15min":0.84799999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.085,"5min":1.444,"15min":1.444},"last":5.351},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84899999999999998,"15min":0.84699999999999998},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.0089999999999999}]},{"osd":13,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.81699999999999995,"15min":0.82299999999999995},"min":{"1min":0.56899999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0329999999999999,"5min":1.482,"15min":1.482},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.753,"15min":0.76000000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94999999999999996,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.70199999999999996}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.65700000000000003}]},{"osd":15,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.85999999999999999,"15min":0.87},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0049999999999999,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97599999999999998,"5min":1.327,"15min":1.327},"last":0.65200000000000002}]}]},{"osd":1,"up_from":21,"seq":90194313247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33140,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,4,5,6,7,8,9,10,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.75600000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67400000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.41999999999999998}]},{"osd":2,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70199999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79600000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61799999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.76000000000000001}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67800000000000005,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72099999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.39000000000000001}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.497,"15min":0.5},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.499,"15min":0.50700000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.33300000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82599999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.58099999999999996}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66800000000000004,"15min":0.66300000000000003},"min":{"1min":0.48099999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65900000000000003,"15min":0.64900000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.59699999999999998}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.88300000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.72199999999999998,"15min":0.73999999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84899999999999998,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.48099999999999998}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.83499999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.435},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67800000000000005,"15min":0.68400000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92900000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.46200000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.70899999999999996}]}]},{"osd":0,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":2056192,"data_stored":1197753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64500000000000002,"15min":0.63400000000000001},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.76200000000000001}]},{"osd":2,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.90200000000000002,"15min":0.95599999999999996},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.89100000000000001,"5min":8.0619999999999994,"15min":8.0619999999999994},"last":0.626},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.1399999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.53300000000000003}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.80500000000000005,"5min":3.363,"15min":3.363},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.90000000000000002,"15min":0.93400000000000005},"min":{"1min":0.50800000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.998,"5min":7.9710000000000001,"15min":7.9710000000000001},"last":0.878}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66000000000000003,"15min":0.66500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97599999999999998,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90200000000000002,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":0.66700000000000004}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.94199999999999995,"15min":0.98699999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1779999999999999,"5min":8.0239999999999991,"15min":8.0239999999999991},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1739999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.90500000000000003}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.72999999999999998,"15min":0.748},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86299999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.98399999999999999,"15min":1.0229999999999999},"min":{"1min":0.42899999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1200000000000001,"5min":7.8630000000000004,"15min":7.8630000000000004},"last":0.91900000000000004}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.67800000000000005,"15min":0.69499999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.92100000000000004,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.95799999999999996,"15min":1.02},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.93400000000000005,"5min":8.1189999999999998,"15min":8.1189999999999998},"last":0.95099999999999996}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.81200000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.01,"5min":3.5699999999999998,"15min":3.5699999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.55300000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.044,"5min":1.927,"15min":1.927},"last":1.0209999999999999}]},{"osd":13,"last update":"Sat Apr 20 01:48:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.84899999999999998}]},{"osd":15,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.097,"5min":1.833,"15min":1.833},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.86899999999999999,"15min":0.88500000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96699999999999997,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":0.97099999999999997}]}]},{"osd":2,"up_from":21,"seq":90194313246,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29044,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342796,"statfs":{"total":96636764160,"available":96607023104,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.42799999999999999}]},{"osd":1,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54600000000000004,"15min":0.53900000000000003},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.72199999999999998}]},{"osd":3,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82199999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":7,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82799999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":0.60299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72299999999999998,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.499}]},{"osd":8,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65000000000000002,"15min":0.66100000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90100000000000002,"5min":2.512,"15min":2.512},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.68400000000000005,"15min":0.69599999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.77600000000000002,"5min":2.532,"15min":2.532},"last":0.63300000000000001}]},{"osd":9,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.435,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64800000000000002},"min":{"1min":0.43099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93000000000000005,"5min":1.107,"15min":1.107},"last":0.65500000000000003}]},{"osd":11,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0269999999999999,"5min":1.135,"15min":1.135},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68600000000000005,"15min":0.69399999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88900000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70299999999999996}]},{"osd":12,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.75800000000000001,"15min":0.78500000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.872,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.80200000000000005,"15min":0.82899999999999996},"min":{"1min":0.42499999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93899999999999995,"5min":3.492,"15min":3.492},"last":0.59099999999999997}]},{"osd":13,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.748}]},{"osd":15,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.60699999999999998}]}]},{"osd":3,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33140,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,4,5,6,7,8,9,10,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.374}]},{"osd":2,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.36199999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.90500000000000003,"5min":0.996,"15min":0.996},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.85599999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.49199999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.56499999999999995,"15min":0.56899999999999995},"min":{"1min":0.34000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69399999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.94599999999999995,"15min":1.014},"min":{"1min":0.39000000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83899999999999997,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":0.61599999999999999}]},{"osd":5,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.56299999999999994,"15min":0.57199999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66500000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.317},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55600000000000005,"15min":0.56000000000000005},"min":{"1min":0.36399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78700000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.44500000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.38600000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89300000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.432}]},{"osd":7,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66200000000000003,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.54800000000000004,"15min":0.56000000000000005},"min":{"1min":0.30099999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71299999999999997,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.41099999999999998}]},{"osd":8,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.41199999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85099999999999998,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.0780000000000001,"15min":1.1479999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81999999999999995,"5min":9.4700000000000006,"15min":9.4700000000000006},"last":0.54800000000000004}]},{"osd":9,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.622,"15min":0.625},"min":{"1min":0.45200000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84699999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.71199999999999997,"15min":0.72599999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74399999999999999,"5min":1.306,"15min":1.306},"last":0.60399999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":12,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":10.959,"5min":10.959,"15min":10.959},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.49399999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.63800000000000001}]}]},{"osd":4,"up_from":21,"seq":90194313248,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":2076672,"data_stored":1207927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,5,6,7,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.81000000000000005,"15min":0.87},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.93400000000000005,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.46700000000000003,"5min":1.1579999999999999,"15min":1.2729999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.008,"5min":16.27,"15min":16.27},"last":0.38800000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":1.012,"15min":1.081},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.92700000000000005,"5min":10.507999999999999,"15min":10.507999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":1.1559999999999999,"15min":1.23},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6319999999999999,"5min":11.019,"15min":11.019},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":3,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.997,"15min":1.0469999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4750000000000001,"5min":4.9180000000000001,"15min":4.9180000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":1.0449999999999999,"15min":1.095},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.6000000000000001,"5min":10.619,"15min":10.619},"last":0.59699999999999998}]},{"osd":5,"last update":"Sat Apr 20 01:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":1.099,"15min":1.169},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.673,"5min":10.895,"15min":10.895},"last":0.622},{"interface":"front","average":{"1min":0.64200000000000002,"5min":1.012,"15min":1.0740000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1910000000000001,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":0.498}]},{"osd":6,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":1.1890000000000001,"15min":1.2649999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1719999999999999,"5min":11.051,"15min":11.051},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":1.1279999999999999,"15min":1.2},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96199999999999997,"5min":10.589,"15min":10.589},"last":0.76600000000000001}]},{"osd":7,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":1.1950000000000001,"15min":1.2649999999999999},"min":{"1min":0.56000000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.571,"5min":10.957000000000001,"15min":10.957000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.79400000000000004,"5min":1.1799999999999999,"15min":1.244},"min":{"1min":0.47299999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3080000000000001,"5min":10.656000000000001,"15min":10.656000000000001},"last":0.66200000000000003}]},{"osd":8,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.79,"5min":3.79,"15min":3.79},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.0269999999999999}]},{"osd":9,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.385,"15min":1.385},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":10.929,"5min":10.929,"15min":10.929},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":10.555999999999999,"5min":10.555999999999999,"15min":10.555999999999999},"last":1.077}]},{"osd":10,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":1.149,"15min":1.204},"min":{"1min":0.57799999999999996,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.331,"5min":5.0229999999999997,"15min":5.0229999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":1.2729999999999999,"15min":1.351},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2330000000000001,"5min":10.69,"15min":10.69},"last":0.69899999999999995}]},{"osd":11,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.3600000000000001,"15min":1.4410000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.448,"5min":10.987,"15min":10.987},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":1.1879999999999999,"15min":1.244},"min":{"1min":0.63700000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.2290000000000001,"5min":4.9900000000000002,"15min":4.9900000000000002},"last":0.63500000000000001}]},{"osd":12,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.3240000000000001,"15min":1.4059999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.181,"5min":10.720000000000001,"15min":10.720000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.85999999999999999,"5min":1.589,"15min":1.7110000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3149999999999999,"5min":16.617000000000001,"15min":16.617000000000001},"last":0.70899999999999996}]},{"osd":13,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":3.8999999999999999,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":0.745},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":4.5019999999999998,"5min":4.5019999999999998,"15min":4.5019999999999998},"last":0.71799999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]},{"osd":5,"up_from":21,"seq":90194313247,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29044,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342796,"statfs":{"total":96636764160,"available":96607023104,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,8,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.85699999999999998,"15min":0.92100000000000004},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77600000000000002,"5min":10.423999999999999,"15min":10.423999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.92500000000000004,"15min":0.98399999999999999},"min":{"1min":0.374,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73499999999999999,"5min":10.465,"15min":10.465},"last":0.76900000000000002}]},{"osd":2,"last update":"Sat Apr 20 01:49:22 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.33600000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.42799999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.755},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.65900000000000003}]},{"osd":4,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.81999999999999995,"15min":0.86799999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71499999999999997,"5min":8.5269999999999992,"15min":8.5269999999999992},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60999999999999999,"15min":0.61699999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78100000000000003,"5min":2.79,"15min":2.79},"last":0.48699999999999999}]},{"osd":6,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84599999999999997,"5min":1.115,"15min":1.115},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.63100000000000001},"min":{"1min":0.38900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94599999999999995,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.51800000000000002}]},{"osd":7,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78800000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.626},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.49399999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.63800000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":10,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.91300000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.54500000000000004}]},{"osd":11,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71899999999999997,"15min":0.71199999999999997},"min":{"1min":0.53900000000000003,"5min":0.435,"15min":0.435},"max":{"1min":1.242,"5min":1.956,"15min":1.956},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70299999999999996,"15min":0.69199999999999995},"min":{"1min":0.55000000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.72599999999999998}]},{"osd":12,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":1.052,"15min":1.1080000000000001},"min":{"1min":0.55800000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90600000000000003,"5min":10.616,"15min":10.616},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.755,"5min":1.0900000000000001,"15min":1.1459999999999999},"min":{"1min":0.56999999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.98899999999999999,"5min":10.629,"15min":10.629},"last":0.82499999999999996}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.84699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T01:49:36.107 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-20T01:49:36.108 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-20T01:49:36.117 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-20T01:49:36.117 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T01:49:36.486 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:49:36.488 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T01:49:36.501 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-04-20T01:49:35.166444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":504836,"kb_used_data":33284,"kb_used_omap":38,"kb_used_meta":471193,"kb_avail":1509444604,"statfs":{"total":1546188226560,"available":1545671274496,"internally_reserved":0,"allocated":34082816,"data_stored":20267840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39290,"internal_metadata":482502278},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":128,"apply_latency_ms":128,"commit_latency_ns":128000000,"apply_latency_ns":128000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001507"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":129,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.387881+0000","last_change":"2024-04-20T01:48:41.550435+0000","last_active":"2024-04-20T01:49:30.387881+0000","last_peered":"2024-04-20T01:49:30.387881+0000","last_clean":"2024-04-20T01:49:30.387881+0000","last_became_active":"2024-04-20T01:47:20.071615+0000","last_became_peered":"2024-04-20T01:47:20.071615+0000","last_unstale":"2024-04-20T01:49:30.387881+0000","last_undegraded":"2024-04-20T01:49:30.387881+0000","last_fullsized":"2024-04-20T01:49:30.387881+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:41.550289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:41.550289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:05.389116+0000","scrub_duration":0.0015320100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":130,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.098794+0000","last_change":"2024-04-20T01:48:36.889025+0000","last_active":"2024-04-20T01:49:30.098794+0000","last_peered":"2024-04-20T01:49:30.098794+0000","last_clean":"2024-04-20T01:49:30.098794+0000","last_became_active":"2024-04-20T01:47:20.075314+0000","last_became_peered":"2024-04-20T01:47:20.075314+0000","last_unstale":"2024-04-20T01:49:30.098794+0000","last_undegraded":"2024-04-20T01:49:30.098794+0000","last_fullsized":"2024-04-20T01:49:30.098794+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:36.888923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:36.888923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:45.012481+0000","scrub_duration":0.0018745420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,12],"acting":[2,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":126,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.379548+0000","last_change":"2024-04-20T01:48:26.145238+0000","last_active":"2024-04-20T01:49:30.379548+0000","last_peered":"2024-04-20T01:49:30.379548+0000","last_clean":"2024-04-20T01:49:30.379548+0000","last_became_active":"2024-04-20T01:47:20.038972+0000","last_became_peered":"2024-04-20T01:47:20.038972+0000","last_unstale":"2024-04-20T01:49:30.379548+0000","last_undegraded":"2024-04-20T01:49:30.379548+0000","last_fullsized":"2024-04-20T01:49:30.379548+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:26.145095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:26.145095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:54.362058+0000","scrub_duration":0.0015014830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":140,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.098342+0000","last_change":"2024-04-20T01:49:28.041091+0000","last_active":"2024-04-20T01:49:30.098342+0000","last_peered":"2024-04-20T01:49:30.098342+0000","last_clean":"2024-04-20T01:49:30.098342+0000","last_became_active":"2024-04-20T01:47:20.038764+0000","last_became_peered":"2024-04-20T01:47:20.038764+0000","last_unstale":"2024-04-20T01:49:30.098342+0000","last_undegraded":"2024-04-20T01:49:30.098342+0000","last_fullsized":"2024-04-20T01:49:30.098342+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:49:28.040983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:49:28.040983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:54.519868+0000","scrub_duration":0.0021762000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"25'157","reported_seq":324,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.379550+0000","last_change":"2024-04-20T01:48:36.239045+0000","last_active":"2024-04-20T01:49:30.379550+0000","last_peered":"2024-04-20T01:49:30.379550+0000","last_clean":"2024-04-20T01:49:30.379550+0000","last_became_active":"2024-04-20T01:47:17.302105+0000","last_became_peered":"2024-04-20T01:47:17.302105+0000","last_unstale":"2024-04-20T01:49:30.379550+0000","last_undegraded":"2024-04-20T01:49:30.379550+0000","last_fullsized":"2024-04-20T01:49:30.379550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'157","last_scrub_stamp":"2024-04-20T01:48:36.238909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_clean_scrub_stamp":"2024-04-20T01:48:36.238909+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:57.382841+0000","scrub_duration":0.0025722610000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,15],"acting":[13,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":82,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.098791+0000","last_change":"2024-04-20T01:49:17.016142+0000","last_active":"2024-04-20T01:49:30.098791+0000","last_peered":"2024-04-20T01:49:30.098791+0000","last_clean":"2024-04-20T01:49:30.098791+0000","last_became_active":"2024-04-20T01:49:17.015873+0000","last_became_peered":"2024-04-20T01:49:17.015873+0000","last_unstale":"2024-04-20T01:49:30.098791+0000","last_undegraded":"2024-04-20T01:49:30.098791+0000","last_fullsized":"2024-04-20T01:49:30.098791+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:37.682441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:37.682441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:54.853315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":130,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.379496+0000","last_change":"2024-04-20T01:48:48.595681+0000","last_active":"2024-04-20T01:49:30.379496+0000","last_peered":"2024-04-20T01:49:30.379496+0000","last_clean":"2024-04-20T01:49:30.379496+0000","last_became_active":"2024-04-20T01:47:20.038107+0000","last_became_peered":"2024-04-20T01:47:20.038107+0000","last_unstale":"2024-04-20T01:49:30.379496+0000","last_undegraded":"2024-04-20T01:49:30.379496+0000","last_fullsized":"2024-04-20T01:49:30.379496+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:48.595563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:48.595563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:00.568790+0000","scrub_duration":0.002598885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":128,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.996918+0000","last_change":"2024-04-20T01:48:48.578126+0000","last_active":"2024-04-20T01:49:30.996918+0000","last_peered":"2024-04-20T01:49:30.996918+0000","last_clean":"2024-04-20T01:49:30.996918+0000","last_became_active":"2024-04-20T01:47:20.110260+0000","last_became_peered":"2024-04-20T01:47:20.110260+0000","last_unstale":"2024-04-20T01:49:30.996918+0000","last_undegraded":"2024-04-20T01:49:30.996918+0000","last_fullsized":"2024-04-20T01:49:30.996918+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:48.577995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:48.577995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:53.805538+0000","scrub_duration":0.002163692,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":128,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.993729+0000","last_change":"2024-04-20T01:48:37.432504+0000","last_active":"2024-04-20T01:49:30.993729+0000","last_peered":"2024-04-20T01:49:30.993729+0000","last_clean":"2024-04-20T01:49:30.993729+0000","last_became_active":"2024-04-20T01:47:20.036328+0000","last_became_peered":"2024-04-20T01:47:20.036328+0000","last_unstale":"2024-04-20T01:49:30.993729+0000","last_undegraded":"2024-04-20T01:49:30.993729+0000","last_fullsized":"2024-04-20T01:49:30.993729+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:37.432331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:37.432331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:49.945535+0000","scrub_duration":0.0020345559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,13],"acting":[6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280543,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33652,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338188,"statfs":{"total":96636764160,"available":96602304512,"internally_reserved":0,"allocated":2588672,"data_stored":1726664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[2,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.45100000000000001}]},{"osd":2,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.59799999999999998,"15min":0.61099999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.74299999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.64000000000000001,"15min":0.65100000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84699999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.43099999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.47699999999999998}]},{"osd":4,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":1.0760000000000001,"15min":1.155},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.747,"5min":9.0129999999999999,"15min":9.0129999999999999},"last":0.625},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.65900000000000003,"15min":0.67500000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.747,"5min":1.022,"15min":1.022},"last":0.60099999999999998}]},{"osd":5,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.59199999999999997,"15min":0.60899999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73999999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.56899999999999995,"15min":0.58299999999999996},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72299999999999998,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.379}]},{"osd":6,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.52800000000000002}]},{"osd":7,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.755,"15min":0.76800000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95999999999999996,"5min":1.143,"15min":1.143},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.78100000000000003,"15min":0.80100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97999999999999998,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.51600000000000001}]},{"osd":10,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.497},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.54100000000000004}]},{"osd":11,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.55400000000000005}]},{"osd":12,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.77800000000000002,"15min":0.80000000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.83699999999999997,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":1.141},{"interface":"front","average":{"1min":0.746,"5min":1.196,"15min":1.2709999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93999999999999995,"5min":9.1389999999999993,"15min":9.1389999999999993},"last":0.82199999999999995}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.60999999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.96099999999999997,"15min":0.98999999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.99399999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.88300000000000001,"15min":0.90600000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.94699999999999995,"5min":1.399,"15min":1.399},"last":0.66300000000000003}]}]},{"osd":15,"up_from":21,"seq":90194313247,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29564,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342276,"statfs":{"total":96636764160,"available":96606490624,"internally_reserved":0,"allocated":2592768,"data_stored":1727267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,3,4,5,6,7,8,9,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.34100000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.67300000000000004,"15min":0.70299999999999996},"min":{"1min":0.36499999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72399999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":0.46700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":2,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.68000000000000005}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42399999999999999,"15min":0.42599999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.76100000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.441}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.38800000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.55000000000000004}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84299999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.69099999999999995}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.627,"15min":0.63400000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86499999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63,"15min":0.63300000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.94399999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.71699999999999997}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.93000000000000005,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94899999999999995,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.61099999999999999}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45900000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96099999999999997,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.86199999999999999,"15min":0.90000000000000002},"min":{"1min":0.43099999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97599999999999998,"5min":4.5979999999999999,"15min":4.5979999999999999},"last":0.59699999999999998}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68400000000000005,"15min":0.68700000000000006},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93899999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.748,"15min":0.746},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.016,"5min":1.272,"15min":1.272},"last":0.81499999999999995}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.73699999999999999}]}]},{"osd":14,"up_from":102,"seq":438086664202,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31288,"kb_used_data":2024,"kb_used_omap":15,"kb_used_meta":29232,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":2072576,"data_stored":1207324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15377,"internal_metadata":29934575},"hb_peers":[0,1,2,3,4,5,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":12,"up_from":21,"seq":90194313247,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29040,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342800,"statfs":{"total":96636764160,"available":96607027200,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":1,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.58499999999999996,"15min":0.55100000000000005},"min":{"1min":0.65800000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66800000000000004,"15min":0.66200000000000003},"min":{"1min":0.504,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93799999999999994,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.47099999999999997}]},{"osd":2,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.48499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.73999999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88700000000000001,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.38100000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79800000000000004,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.49399999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.39500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.59999999999999998,"15min":0.61199999999999999},"min":{"1min":0.371,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82399999999999995,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.66200000000000003}]},{"osd":5,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.65500000000000003,"15min":0.63700000000000001},"min":{"1min":0.55200000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0940000000000001,"5min":1.115,"15min":1.115},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.47499999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.0800000000000001,"5min":1.726,"15min":1.726},"last":0.77600000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.66200000000000003,"15min":0.66600000000000004},"min":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.86699999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.74299999999999999,"15min":0.72899999999999998},"min":{"1min":0.63700000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0349999999999999,"5min":1.069,"15min":1.069},"last":0.871}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.76300000000000001,"15min":0.76500000000000001},"min":{"1min":0.54400000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0529999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.95399999999999996,"15min":0.98999999999999999},"min":{"1min":0.53800000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.012,"5min":6.6779999999999999,"15min":6.6779999999999999},"last":0.60499999999999998}]},{"osd":13,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.79800000000000004},"min":{"1min":0.59699999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0189999999999999,"5min":1.827,"15min":1.827},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.80900000000000005,"15min":0.79500000000000004},"min":{"1min":0.753,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1739999999999999,"5min":1.258,"15min":1.258},"last":0.91300000000000003}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":1.135},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.99299999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.79000000000000004},"min":{"1min":0.58199999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0109999999999999,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.82999999999999996,"15min":0.81699999999999995},"min":{"1min":0.753,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.94899999999999995}]}]},{"osd":11,"up_from":21,"seq":90194313246,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33140,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,2,3,4,8,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.58599999999999997,"15min":0.59799999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.83899999999999997,"5min":1.669,"15min":1.669},"last":4.4279999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.67300000000000004,"15min":0.68300000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0900000000000001,"5min":1.75,"15min":1.75},"last":10.49}]},{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.58099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.40300000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.28799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.49099999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.88,"15min":0.93500000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1220000000000001,"5min":8.5969999999999995,"15min":8.5969999999999995},"last":10.468999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.67000000000000004,"15min":0.67900000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.147,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":10.507}]},{"osd":8,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":4.4909999999999997}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.57499999999999996,"15min":0.56200000000000006},"min":{"1min":0.38600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.47299999999999998}]},{"osd":10,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47199999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.50900000000000001}]},{"osd":12,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85699999999999998,"5min":1.875,"15min":1.875},"last":10.619},{"interface":"front","average":{"1min":0.73799999999999999,"5min":1.262,"15min":1.349},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1879999999999999,"5min":12.385,"15min":12.385},"last":10.706}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.54600000000000004}]},{"osd":15,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.75,"15min":0.76100000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.96999999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.61399999999999999}]}]},{"osd":10,"up_from":21,"seq":90194313248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33140,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,4,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.42899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":7,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63300000000000001,"15min":0.626},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.77300000000000002}]},{"osd":8,"last update":"Sat Apr 20 01:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.70399999999999996}]},{"osd":9,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68999999999999995,"15min":0.68500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.76400000000000001}]},{"osd":11,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.42499999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82099999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.90200000000000002,"15min":0.92600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.292,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":0.79000000000000004}]},{"osd":12,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":1.208,"15min":1.29},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1120000000000001,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.997,"15min":1.054},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91000000000000003,"5min":9.7759999999999998,"15min":9.7759999999999998},"last":0.66000000000000003}]},{"osd":13,"last update":"Sat Apr 20 01:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89600000000000002,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.71699999999999997,"15min":0.72299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95799999999999996,"5min":1.05,"15min":1.05},"last":0.749}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.64200000000000002}]},{"osd":15,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.80100000000000005,"15min":0.79700000000000004},"min":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.53400000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.93700000000000006,"5min":1.157,"15min":1.157},"last":0.88400000000000001}]}]},{"osd":9,"up_from":21,"seq":90194313248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":29044,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342796,"statfs":{"total":96636764160,"available":96607023104,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,2,4,6,7,8,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47099999999999997,"15min":0.46300000000000002},"min":{"1min":0.32800000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.52500000000000002,"15min":0.51300000000000001},"min":{"1min":0.40300000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83599999999999997,"5min":0.998,"15min":0.998},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.59399999999999997}]},{"osd":6,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.872,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55700000000000005}]},{"osd":7,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.81000000000000005}]},{"osd":8,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.39600000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.67000000000000004}]},{"osd":10,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.93799999999999994,"5min":1.03,"15min":1.03},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58699999999999997,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76300000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.73899999999999999}]},{"osd":11,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88400000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.81999999999999995}]},{"osd":12,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.74099999999999999,"15min":0.73899999999999999},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97899999999999998,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.89900000000000002,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.85199999999999998}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.56000000000000005}]},{"osd":15,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.745,"15min":0.753},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69699999999999995,"15min":0.70499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88800000000000001,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.76800000000000002}]}]},{"osd":8,"up_from":21,"seq":90194313247,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":2076672,"data_stored":1207927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.34200000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89700000000000002,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.628,"15min":0.64100000000000001},"min":{"1min":0.39500000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.81699999999999995,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.58199999999999996}]},{"osd":3,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.45400000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85799999999999998,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.73599999999999999,"15min":0.753},"min":{"1min":0.44500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91600000000000004,"5min":1.821,"15min":1.821},"last":0.80200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":6,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70199999999999996,"15min":0.70999999999999996},"min":{"1min":0.46500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89900000000000002,"5min":1.784,"15min":1.784},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.44,"5min":0.437,"15min":0.437},"max":{"1min":0.79300000000000004,"5min":1.885,"15min":1.885},"last":0.54700000000000004}]},{"osd":7,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.72599999999999998,"15min":0.73499999999999999},"min":{"1min":0.50800000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93300000000000005,"5min":1.411,"15min":1.411},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.81999999999999995,"15min":0.84399999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95199999999999996,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.75700000000000001}]},{"osd":9,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.72199999999999998,"15min":0.73599999999999999},"min":{"1min":0.436,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91800000000000004,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.139,"5min":1.375,"15min":1.375},"last":0.78200000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":11,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.83899999999999997,"15min":0.84299999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.141,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.92400000000000004,"15min":0.93999999999999995},"min":{"1min":0.66100000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.071,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.65600000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":13,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.63},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.73099999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":15,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.88700000000000001,"15min":0.89100000000000001},"min":{"1min":0.621,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.095,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.95399999999999996,"15min":0.98399999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0229999999999999,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.85199999999999998}]}]},{"osd":7,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29032,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342808,"statfs":{"total":96636764160,"available":96607035392,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,3,4,6,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.38800000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.67000000000000004}]},{"osd":3,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.45900000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.379,"5min":0.372,"15min":0.372},"max":{"1min":0.92600000000000005,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.47199999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94399999999999995,"5min":4.1920000000000002,"15min":4.1920000000000002},"last":0.63800000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.751},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.68899999999999995}]},{"osd":8,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.78900000000000003,"15min":0.81499999999999995},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82599999999999996,"5min":5.7679999999999998,"15min":5.7679999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":1.0189999999999999,"15min":1.077},"min":{"1min":0.46300000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96599999999999997,"5min":9.952,"15min":9.952},"last":0.56799999999999995}]},{"osd":9,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.438,"5min":0.433,"15min":0.433},"max":{"1min":0.751,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.498,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88800000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.65400000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82899999999999996,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.88800000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.57999999999999996}]},{"osd":12,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.84399999999999997,"15min":0.86099999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.99199999999999999,"5min":4.2720000000000002,"15min":4.2720000000000002},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68700000000000006,"15min":0.68000000000000005},"min":{"1min":0.50900000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90300000000000002,"5min":0.995,"15min":0.995},"last":0.77000000000000002}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.79700000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":6,"up_from":21,"seq":90194313246,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":2076672,"data_stored":1207927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[3,4,5,7,8,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.5}]},{"osd":2,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.53700000000000003}]},{"osd":3,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.45600000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":5.141},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":8.0090000000000003}]},{"osd":5,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.38300000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.91700000000000004,"5min":1.083,"15min":1.083},"last":0.59999999999999998}]},{"osd":7,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59999999999999998,"15min":0.60399999999999998},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77600000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.626},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66100000000000003,"15min":0.66400000000000003},"min":{"1min":0.36499999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.89400000000000002,"5min":1.056,"15min":1.056},"last":0.56399999999999995}]},{"osd":8,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.755,"15min":0.77000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88300000000000001,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":6.1130000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.80100000000000005,"15min":0.81200000000000006},"min":{"1min":0.53800000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92000000000000004,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":6.0839999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":10,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67900000000000005,"15min":0.68700000000000006},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92500000000000004,"5min":1.129,"15min":1.129},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.67700000000000005,"15min":0.69099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88,"5min":1.272,"15min":1.272},"last":0.51400000000000001}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.92100000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.78100000000000003,"15min":0.79000000000000004},"min":{"1min":0.52200000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96599999999999997,"5min":1.167,"15min":1.167},"last":0.72099999999999997}]},{"osd":12,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.83499999999999996,"15min":0.84799999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.085,"5min":1.444,"15min":1.444},"last":5.351},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84899999999999998,"15min":0.84699999999999998},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.0089999999999999}]},{"osd":13,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.81699999999999995,"15min":0.82299999999999995},"min":{"1min":0.56899999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0329999999999999,"5min":1.482,"15min":1.482},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.753,"15min":0.76000000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94999999999999996,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.70199999999999996}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.65700000000000003}]},{"osd":15,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.85999999999999999,"15min":0.87},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0049999999999999,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97599999999999998,"5min":1.327,"15min":1.327},"last":0.65200000000000002}]}]},{"osd":1,"up_from":21,"seq":90194313247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33140,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,4,5,6,7,8,9,10,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.75600000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67400000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.41999999999999998}]},{"osd":2,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70199999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79600000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61799999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.76000000000000001}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67800000000000005,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72099999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.39000000000000001}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.497,"15min":0.5},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.499,"15min":0.50700000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.33300000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82599999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.58099999999999996}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66800000000000004,"15min":0.66300000000000003},"min":{"1min":0.48099999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65900000000000003,"15min":0.64900000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.59699999999999998}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.88300000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.72199999999999998,"15min":0.73999999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84899999999999998,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.48099999999999998}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.83499999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.435},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67800000000000005,"15min":0.68400000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92900000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.46200000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.70899999999999996}]}]},{"osd":0,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":2056192,"data_stored":1197753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64500000000000002,"15min":0.63400000000000001},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.76200000000000001}]},{"osd":2,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.90200000000000002,"15min":0.95599999999999996},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.89100000000000001,"5min":8.0619999999999994,"15min":8.0619999999999994},"last":0.626},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.1399999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.53300000000000003}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.80500000000000005,"5min":3.363,"15min":3.363},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.90000000000000002,"15min":0.93400000000000005},"min":{"1min":0.50800000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.998,"5min":7.9710000000000001,"15min":7.9710000000000001},"last":0.878}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66000000000000003,"15min":0.66500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97599999999999998,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90200000000000002,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":0.66700000000000004}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.94199999999999995,"15min":0.98699999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1779999999999999,"5min":8.0239999999999991,"15min":8.0239999999999991},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1739999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.90500000000000003}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.72999999999999998,"15min":0.748},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86299999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.98399999999999999,"15min":1.0229999999999999},"min":{"1min":0.42899999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1200000000000001,"5min":7.8630000000000004,"15min":7.8630000000000004},"last":0.91900000000000004}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.67800000000000005,"15min":0.69499999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.92100000000000004,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.95799999999999996,"15min":1.02},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.93400000000000005,"5min":8.1189999999999998,"15min":8.1189999999999998},"last":0.95099999999999996}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.81200000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.01,"5min":3.5699999999999998,"15min":3.5699999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.55300000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.044,"5min":1.927,"15min":1.927},"last":1.0209999999999999}]},{"osd":13,"last update":"Sat Apr 20 01:48:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.84899999999999998}]},{"osd":15,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.097,"5min":1.833,"15min":1.833},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.86899999999999999,"15min":0.88500000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96699999999999997,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":0.97099999999999997}]}]},{"osd":2,"up_from":21,"seq":90194313246,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29044,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342796,"statfs":{"total":96636764160,"available":96607023104,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.42799999999999999}]},{"osd":1,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54600000000000004,"15min":0.53900000000000003},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.72199999999999998}]},{"osd":3,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82199999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":7,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82799999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":0.60299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72299999999999998,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.499}]},{"osd":8,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65000000000000002,"15min":0.66100000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90100000000000002,"5min":2.512,"15min":2.512},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.68400000000000005,"15min":0.69599999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.77600000000000002,"5min":2.532,"15min":2.532},"last":0.63300000000000001}]},{"osd":9,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.435,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64800000000000002},"min":{"1min":0.43099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93000000000000005,"5min":1.107,"15min":1.107},"last":0.65500000000000003}]},{"osd":11,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0269999999999999,"5min":1.135,"15min":1.135},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68600000000000005,"15min":0.69399999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88900000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70299999999999996}]},{"osd":12,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.75800000000000001,"15min":0.78500000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.872,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.80200000000000005,"15min":0.82899999999999996},"min":{"1min":0.42499999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93899999999999995,"5min":3.492,"15min":3.492},"last":0.59099999999999997}]},{"osd":13,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.748}]},{"osd":15,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.60699999999999998}]}]},{"osd":3,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33140,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,4,5,6,7,8,9,10,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.374}]},{"osd":2,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.36199999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.90500000000000003,"5min":0.996,"15min":0.996},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.85599999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.49199999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.56499999999999995,"15min":0.56899999999999995},"min":{"1min":0.34000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69399999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.94599999999999995,"15min":1.014},"min":{"1min":0.39000000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83899999999999997,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":0.61599999999999999}]},{"osd":5,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.56299999999999994,"15min":0.57199999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66500000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.317},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55600000000000005,"15min":0.56000000000000005},"min":{"1min":0.36399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78700000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.44500000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.38600000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89300000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.432}]},{"osd":7,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66200000000000003,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.54800000000000004,"15min":0.56000000000000005},"min":{"1min":0.30099999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71299999999999997,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.41099999999999998}]},{"osd":8,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.41199999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85099999999999998,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.0780000000000001,"15min":1.1479999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81999999999999995,"5min":9.4700000000000006,"15min":9.4700000000000006},"last":0.54800000000000004}]},{"osd":9,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.622,"15min":0.625},"min":{"1min":0.45200000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84699999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.71199999999999997,"15min":0.72599999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74399999999999999,"5min":1.306,"15min":1.306},"last":0.60399999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":12,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":10.959,"5min":10.959,"15min":10.959},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.49399999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.63800000000000001}]}]},{"osd":4,"up_from":21,"seq":90194313248,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":2076672,"data_stored":1207927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,5,6,7,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.81000000000000005,"15min":0.87},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.93400000000000005,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.46700000000000003,"5min":1.1579999999999999,"15min":1.2729999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.008,"5min":16.27,"15min":16.27},"last":0.38800000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":1.012,"15min":1.081},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.92700000000000005,"5min":10.507999999999999,"15min":10.507999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":1.1559999999999999,"15min":1.23},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6319999999999999,"5min":11.019,"15min":11.019},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":3,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.997,"15min":1.0469999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4750000000000001,"5min":4.9180000000000001,"15min":4.9180000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":1.0449999999999999,"15min":1.095},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.6000000000000001,"5min":10.619,"15min":10.619},"last":0.59699999999999998}]},{"osd":5,"last update":"Sat Apr 20 01:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":1.099,"15min":1.169},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.673,"5min":10.895,"15min":10.895},"last":0.622},{"interface":"front","average":{"1min":0.64200000000000002,"5min":1.012,"15min":1.0740000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1910000000000001,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":0.498}]},{"osd":6,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":1.1890000000000001,"15min":1.2649999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1719999999999999,"5min":11.051,"15min":11.051},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":1.1279999999999999,"15min":1.2},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96199999999999997,"5min":10.589,"15min":10.589},"last":0.76600000000000001}]},{"osd":7,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":1.1950000000000001,"15min":1.2649999999999999},"min":{"1min":0.56000000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.571,"5min":10.957000000000001,"15min":10.957000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.79400000000000004,"5min":1.1799999999999999,"15min":1.244},"min":{"1min":0.47299999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3080000000000001,"5min":10.656000000000001,"15min":10.656000000000001},"last":0.66200000000000003}]},{"osd":8,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.79,"5min":3.79,"15min":3.79},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.0269999999999999}]},{"osd":9,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.385,"15min":1.385},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":10.929,"5min":10.929,"15min":10.929},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":10.555999999999999,"5min":10.555999999999999,"15min":10.555999999999999},"last":1.077}]},{"osd":10,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":1.149,"15min":1.204},"min":{"1min":0.57799999999999996,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.331,"5min":5.0229999999999997,"15min":5.0229999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":1.2729999999999999,"15min":1.351},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2330000000000001,"5min":10.69,"15min":10.69},"last":0.69899999999999995}]},{"osd":11,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.3600000000000001,"15min":1.4410000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.448,"5min":10.987,"15min":10.987},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":1.1879999999999999,"15min":1.244},"min":{"1min":0.63700000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.2290000000000001,"5min":4.9900000000000002,"15min":4.9900000000000002},"last":0.63500000000000001}]},{"osd":12,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.3240000000000001,"15min":1.4059999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.181,"5min":10.720000000000001,"15min":10.720000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.85999999999999999,"5min":1.589,"15min":1.7110000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3149999999999999,"5min":16.617000000000001,"15min":16.617000000000001},"last":0.70899999999999996}]},{"osd":13,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":3.8999999999999999,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":0.745},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":4.5019999999999998,"5min":4.5019999999999998,"15min":4.5019999999999998},"last":0.71799999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]},{"osd":5,"up_from":21,"seq":90194313247,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29044,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342796,"statfs":{"total":96636764160,"available":96607023104,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,8,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.85699999999999998,"15min":0.92100000000000004},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77600000000000002,"5min":10.423999999999999,"15min":10.423999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.92500000000000004,"15min":0.98399999999999999},"min":{"1min":0.374,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73499999999999999,"5min":10.465,"15min":10.465},"last":0.76900000000000002}]},{"osd":2,"last update":"Sat Apr 20 01:49:22 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.33600000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.42799999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.755},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.65900000000000003}]},{"osd":4,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.81999999999999995,"15min":0.86799999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71499999999999997,"5min":8.5269999999999992,"15min":8.5269999999999992},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60999999999999999,"15min":0.61699999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78100000000000003,"5min":2.79,"15min":2.79},"last":0.48699999999999999}]},{"osd":6,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84599999999999997,"5min":1.115,"15min":1.115},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.63100000000000001},"min":{"1min":0.38900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94599999999999995,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.51800000000000002}]},{"osd":7,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78800000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.626},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.49399999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.63800000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":10,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.91300000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.54500000000000004}]},{"osd":11,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71899999999999997,"15min":0.71199999999999997},"min":{"1min":0.53900000000000003,"5min":0.435,"15min":0.435},"max":{"1min":1.242,"5min":1.956,"15min":1.956},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70299999999999996,"15min":0.69199999999999995},"min":{"1min":0.55000000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.72599999999999998}]},{"osd":12,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":1.052,"15min":1.1080000000000001},"min":{"1min":0.55800000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90600000000000003,"5min":10.616,"15min":10.616},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.755,"5min":1.0900000000000001,"15min":1.1459999999999999},"min":{"1min":0.56999999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.98899999999999999,"5min":10.629,"15min":10.629},"last":0.82499999999999996}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.84699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T01:49:36.505 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T01:49:36.875 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:49:36.877 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T01:49:36.889 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-04-20T01:49:35.166444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":504836,"kb_used_data":33284,"kb_used_omap":38,"kb_used_meta":471193,"kb_avail":1509444604,"statfs":{"total":1546188226560,"available":1545671274496,"internally_reserved":0,"allocated":34082816,"data_stored":20267840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39290,"internal_metadata":482502278},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":128,"apply_latency_ms":128,"commit_latency_ns":128000000,"apply_latency_ns":128000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001507"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":129,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.387881+0000","last_change":"2024-04-20T01:48:41.550435+0000","last_active":"2024-04-20T01:49:30.387881+0000","last_peered":"2024-04-20T01:49:30.387881+0000","last_clean":"2024-04-20T01:49:30.387881+0000","last_became_active":"2024-04-20T01:47:20.071615+0000","last_became_peered":"2024-04-20T01:47:20.071615+0000","last_unstale":"2024-04-20T01:49:30.387881+0000","last_undegraded":"2024-04-20T01:49:30.387881+0000","last_fullsized":"2024-04-20T01:49:30.387881+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:41.550289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:41.550289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:05.389116+0000","scrub_duration":0.0015320100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":130,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.098794+0000","last_change":"2024-04-20T01:48:36.889025+0000","last_active":"2024-04-20T01:49:30.098794+0000","last_peered":"2024-04-20T01:49:30.098794+0000","last_clean":"2024-04-20T01:49:30.098794+0000","last_became_active":"2024-04-20T01:47:20.075314+0000","last_became_peered":"2024-04-20T01:47:20.075314+0000","last_unstale":"2024-04-20T01:49:30.098794+0000","last_undegraded":"2024-04-20T01:49:30.098794+0000","last_fullsized":"2024-04-20T01:49:30.098794+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:36.888923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:36.888923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:45.012481+0000","scrub_duration":0.0018745420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,12],"acting":[2,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":126,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.379548+0000","last_change":"2024-04-20T01:48:26.145238+0000","last_active":"2024-04-20T01:49:30.379548+0000","last_peered":"2024-04-20T01:49:30.379548+0000","last_clean":"2024-04-20T01:49:30.379548+0000","last_became_active":"2024-04-20T01:47:20.038972+0000","last_became_peered":"2024-04-20T01:47:20.038972+0000","last_unstale":"2024-04-20T01:49:30.379548+0000","last_undegraded":"2024-04-20T01:49:30.379548+0000","last_fullsized":"2024-04-20T01:49:30.379548+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:26.145095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:26.145095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:54.362058+0000","scrub_duration":0.0015014830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":140,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.098342+0000","last_change":"2024-04-20T01:49:28.041091+0000","last_active":"2024-04-20T01:49:30.098342+0000","last_peered":"2024-04-20T01:49:30.098342+0000","last_clean":"2024-04-20T01:49:30.098342+0000","last_became_active":"2024-04-20T01:47:20.038764+0000","last_became_peered":"2024-04-20T01:47:20.038764+0000","last_unstale":"2024-04-20T01:49:30.098342+0000","last_undegraded":"2024-04-20T01:49:30.098342+0000","last_fullsized":"2024-04-20T01:49:30.098342+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:49:28.040983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:49:28.040983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:54.519868+0000","scrub_duration":0.0021762000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"25'157","reported_seq":324,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.379550+0000","last_change":"2024-04-20T01:48:36.239045+0000","last_active":"2024-04-20T01:49:30.379550+0000","last_peered":"2024-04-20T01:49:30.379550+0000","last_clean":"2024-04-20T01:49:30.379550+0000","last_became_active":"2024-04-20T01:47:17.302105+0000","last_became_peered":"2024-04-20T01:47:17.302105+0000","last_unstale":"2024-04-20T01:49:30.379550+0000","last_undegraded":"2024-04-20T01:49:30.379550+0000","last_fullsized":"2024-04-20T01:49:30.379550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'157","last_scrub_stamp":"2024-04-20T01:48:36.238909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_clean_scrub_stamp":"2024-04-20T01:48:36.238909+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:57.382841+0000","scrub_duration":0.0025722610000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,15],"acting":[13,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":82,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.098791+0000","last_change":"2024-04-20T01:49:17.016142+0000","last_active":"2024-04-20T01:49:30.098791+0000","last_peered":"2024-04-20T01:49:30.098791+0000","last_clean":"2024-04-20T01:49:30.098791+0000","last_became_active":"2024-04-20T01:49:17.015873+0000","last_became_peered":"2024-04-20T01:49:17.015873+0000","last_unstale":"2024-04-20T01:49:30.098791+0000","last_undegraded":"2024-04-20T01:49:30.098791+0000","last_fullsized":"2024-04-20T01:49:30.098791+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:37.682441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:37.682441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:54.853315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":130,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.379496+0000","last_change":"2024-04-20T01:48:48.595681+0000","last_active":"2024-04-20T01:49:30.379496+0000","last_peered":"2024-04-20T01:49:30.379496+0000","last_clean":"2024-04-20T01:49:30.379496+0000","last_became_active":"2024-04-20T01:47:20.038107+0000","last_became_peered":"2024-04-20T01:47:20.038107+0000","last_unstale":"2024-04-20T01:49:30.379496+0000","last_undegraded":"2024-04-20T01:49:30.379496+0000","last_fullsized":"2024-04-20T01:49:30.379496+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:48.595563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:48.595563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:00.568790+0000","scrub_duration":0.002598885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":128,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.996918+0000","last_change":"2024-04-20T01:48:48.578126+0000","last_active":"2024-04-20T01:49:30.996918+0000","last_peered":"2024-04-20T01:49:30.996918+0000","last_clean":"2024-04-20T01:49:30.996918+0000","last_became_active":"2024-04-20T01:47:20.110260+0000","last_became_peered":"2024-04-20T01:47:20.110260+0000","last_unstale":"2024-04-20T01:49:30.996918+0000","last_undegraded":"2024-04-20T01:49:30.996918+0000","last_fullsized":"2024-04-20T01:49:30.996918+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:48.577995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:48.577995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:53.805538+0000","scrub_duration":0.002163692,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":128,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-20T01:49:30.993729+0000","last_change":"2024-04-20T01:48:37.432504+0000","last_active":"2024-04-20T01:49:30.993729+0000","last_peered":"2024-04-20T01:49:30.993729+0000","last_clean":"2024-04-20T01:49:30.993729+0000","last_became_active":"2024-04-20T01:47:20.036328+0000","last_became_peered":"2024-04-20T01:47:20.036328+0000","last_unstale":"2024-04-20T01:49:30.993729+0000","last_undegraded":"2024-04-20T01:49:30.993729+0000","last_fullsized":"2024-04-20T01:49:30.993729+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:37.432331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:37.432331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:49.945535+0000","scrub_duration":0.0020345559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,13],"acting":[6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280543,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33652,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338188,"statfs":{"total":96636764160,"available":96602304512,"internally_reserved":0,"allocated":2588672,"data_stored":1726664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[2,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.45100000000000001}]},{"osd":2,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.59799999999999998,"15min":0.61099999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.74299999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.64000000000000001,"15min":0.65100000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84699999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.43099999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.47699999999999998}]},{"osd":4,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":1.0760000000000001,"15min":1.155},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.747,"5min":9.0129999999999999,"15min":9.0129999999999999},"last":0.625},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.65900000000000003,"15min":0.67500000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.747,"5min":1.022,"15min":1.022},"last":0.60099999999999998}]},{"osd":5,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.59199999999999997,"15min":0.60899999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73999999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.56899999999999995,"15min":0.58299999999999996},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72299999999999998,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.379}]},{"osd":6,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.52800000000000002}]},{"osd":7,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.755,"15min":0.76800000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95999999999999996,"5min":1.143,"15min":1.143},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.78100000000000003,"15min":0.80100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97999999999999998,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.51600000000000001}]},{"osd":10,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.497},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.54100000000000004}]},{"osd":11,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.55400000000000005}]},{"osd":12,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.77800000000000002,"15min":0.80000000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.83699999999999997,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":1.141},{"interface":"front","average":{"1min":0.746,"5min":1.196,"15min":1.2709999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93999999999999995,"5min":9.1389999999999993,"15min":9.1389999999999993},"last":0.82199999999999995}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.60999999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.96099999999999997,"15min":0.98999999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.99399999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.88300000000000001,"15min":0.90600000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.94699999999999995,"5min":1.399,"15min":1.399},"last":0.66300000000000003}]}]},{"osd":15,"up_from":21,"seq":90194313247,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29564,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342276,"statfs":{"total":96636764160,"available":96606490624,"internally_reserved":0,"allocated":2592768,"data_stored":1727267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,3,4,5,6,7,8,9,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.34100000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.67300000000000004,"15min":0.70299999999999996},"min":{"1min":0.36499999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72399999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":0.46700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":2,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.68000000000000005}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42399999999999999,"15min":0.42599999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.76100000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.441}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.38800000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.55000000000000004}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84299999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.69099999999999995}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.627,"15min":0.63400000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86499999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63,"15min":0.63300000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.94399999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.71699999999999997}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.93000000000000005,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94899999999999995,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.61099999999999999}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45900000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96099999999999997,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.86199999999999999,"15min":0.90000000000000002},"min":{"1min":0.43099999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97599999999999998,"5min":4.5979999999999999,"15min":4.5979999999999999},"last":0.59699999999999998}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68400000000000005,"15min":0.68700000000000006},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93899999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.748,"15min":0.746},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.016,"5min":1.272,"15min":1.272},"last":0.81499999999999995}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.73699999999999999}]}]},{"osd":14,"up_from":102,"seq":438086664202,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31288,"kb_used_data":2024,"kb_used_omap":15,"kb_used_meta":29232,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":2072576,"data_stored":1207324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15377,"internal_metadata":29934575},"hb_peers":[0,1,2,3,4,5,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":12,"up_from":21,"seq":90194313247,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29040,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342800,"statfs":{"total":96636764160,"available":96607027200,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":1,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.58499999999999996,"15min":0.55100000000000005},"min":{"1min":0.65800000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66800000000000004,"15min":0.66200000000000003},"min":{"1min":0.504,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93799999999999994,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.47099999999999997}]},{"osd":2,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.48499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.73999999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88700000000000001,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.38100000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79800000000000004,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.49399999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.39500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.59999999999999998,"15min":0.61199999999999999},"min":{"1min":0.371,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82399999999999995,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.66200000000000003}]},{"osd":5,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.65500000000000003,"15min":0.63700000000000001},"min":{"1min":0.55200000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0940000000000001,"5min":1.115,"15min":1.115},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.47499999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.0800000000000001,"5min":1.726,"15min":1.726},"last":0.77600000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.66200000000000003,"15min":0.66600000000000004},"min":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.86699999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.74299999999999999,"15min":0.72899999999999998},"min":{"1min":0.63700000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0349999999999999,"5min":1.069,"15min":1.069},"last":0.871}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.76300000000000001,"15min":0.76500000000000001},"min":{"1min":0.54400000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0529999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.95399999999999996,"15min":0.98999999999999999},"min":{"1min":0.53800000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.012,"5min":6.6779999999999999,"15min":6.6779999999999999},"last":0.60499999999999998}]},{"osd":13,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.79800000000000004},"min":{"1min":0.59699999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0189999999999999,"5min":1.827,"15min":1.827},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.80900000000000005,"15min":0.79500000000000004},"min":{"1min":0.753,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1739999999999999,"5min":1.258,"15min":1.258},"last":0.91300000000000003}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":1.135},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.99299999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.79000000000000004},"min":{"1min":0.58199999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0109999999999999,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.82999999999999996,"15min":0.81699999999999995},"min":{"1min":0.753,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.94899999999999995}]}]},{"osd":11,"up_from":21,"seq":90194313246,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33140,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,2,3,4,8,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.58599999999999997,"15min":0.59799999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.83899999999999997,"5min":1.669,"15min":1.669},"last":4.4279999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.67300000000000004,"15min":0.68300000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0900000000000001,"5min":1.75,"15min":1.75},"last":10.49}]},{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.58099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.40300000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.28799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.49099999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.88,"15min":0.93500000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1220000000000001,"5min":8.5969999999999995,"15min":8.5969999999999995},"last":10.468999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.67000000000000004,"15min":0.67900000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.147,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":10.507}]},{"osd":8,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":4.4909999999999997}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.57499999999999996,"15min":0.56200000000000006},"min":{"1min":0.38600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.47299999999999998}]},{"osd":10,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47199999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.50900000000000001}]},{"osd":12,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85699999999999998,"5min":1.875,"15min":1.875},"last":10.619},{"interface":"front","average":{"1min":0.73799999999999999,"5min":1.262,"15min":1.349},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1879999999999999,"5min":12.385,"15min":12.385},"last":10.706}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.54600000000000004}]},{"osd":15,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.75,"15min":0.76100000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.96999999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.61399999999999999}]}]},{"osd":10,"up_from":21,"seq":90194313248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33140,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,4,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.42899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":7,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63300000000000001,"15min":0.626},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.77300000000000002}]},{"osd":8,"last update":"Sat Apr 20 01:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.70399999999999996}]},{"osd":9,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68999999999999995,"15min":0.68500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.76400000000000001}]},{"osd":11,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.42499999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82099999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.90200000000000002,"15min":0.92600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.292,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":0.79000000000000004}]},{"osd":12,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":1.208,"15min":1.29},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1120000000000001,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.997,"15min":1.054},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91000000000000003,"5min":9.7759999999999998,"15min":9.7759999999999998},"last":0.66000000000000003}]},{"osd":13,"last update":"Sat Apr 20 01:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89600000000000002,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.71699999999999997,"15min":0.72299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95799999999999996,"5min":1.05,"15min":1.05},"last":0.749}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.64200000000000002}]},{"osd":15,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.80100000000000005,"15min":0.79700000000000004},"min":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.53400000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.93700000000000006,"5min":1.157,"15min":1.157},"last":0.88400000000000001}]}]},{"osd":9,"up_from":21,"seq":90194313248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":29044,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342796,"statfs":{"total":96636764160,"available":96607023104,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,2,4,6,7,8,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47099999999999997,"15min":0.46300000000000002},"min":{"1min":0.32800000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.52500000000000002,"15min":0.51300000000000001},"min":{"1min":0.40300000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83599999999999997,"5min":0.998,"15min":0.998},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.59399999999999997}]},{"osd":6,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.872,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55700000000000005}]},{"osd":7,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.81000000000000005}]},{"osd":8,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.39600000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.67000000000000004}]},{"osd":10,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.93799999999999994,"5min":1.03,"15min":1.03},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58699999999999997,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76300000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.73899999999999999}]},{"osd":11,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88400000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.81999999999999995}]},{"osd":12,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.74099999999999999,"15min":0.73899999999999999},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97899999999999998,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.89900000000000002,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.85199999999999998}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.56000000000000005}]},{"osd":15,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.745,"15min":0.753},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69699999999999995,"15min":0.70499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88800000000000001,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.76800000000000002}]}]},{"osd":8,"up_from":21,"seq":90194313247,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":2076672,"data_stored":1207927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.34200000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89700000000000002,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.628,"15min":0.64100000000000001},"min":{"1min":0.39500000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.81699999999999995,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.58199999999999996}]},{"osd":3,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.45400000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85799999999999998,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.73599999999999999,"15min":0.753},"min":{"1min":0.44500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91600000000000004,"5min":1.821,"15min":1.821},"last":0.80200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":6,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70199999999999996,"15min":0.70999999999999996},"min":{"1min":0.46500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89900000000000002,"5min":1.784,"15min":1.784},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.44,"5min":0.437,"15min":0.437},"max":{"1min":0.79300000000000004,"5min":1.885,"15min":1.885},"last":0.54700000000000004}]},{"osd":7,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.72599999999999998,"15min":0.73499999999999999},"min":{"1min":0.50800000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93300000000000005,"5min":1.411,"15min":1.411},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.81999999999999995,"15min":0.84399999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95199999999999996,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.75700000000000001}]},{"osd":9,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.72199999999999998,"15min":0.73599999999999999},"min":{"1min":0.436,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91800000000000004,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.139,"5min":1.375,"15min":1.375},"last":0.78200000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":11,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.83899999999999997,"15min":0.84299999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.141,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.92400000000000004,"15min":0.93999999999999995},"min":{"1min":0.66100000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.071,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.65600000000000003}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":13,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.63},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.73099999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":15,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.88700000000000001,"15min":0.89100000000000001},"min":{"1min":0.621,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.095,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.95399999999999996,"15min":0.98399999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0229999999999999,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.85199999999999998}]}]},{"osd":7,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29032,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342808,"statfs":{"total":96636764160,"available":96607035392,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,3,4,6,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.38800000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.67000000000000004}]},{"osd":3,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.45900000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.379,"5min":0.372,"15min":0.372},"max":{"1min":0.92600000000000005,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.47199999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94399999999999995,"5min":4.1920000000000002,"15min":4.1920000000000002},"last":0.63800000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.751},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.68899999999999995}]},{"osd":8,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.78900000000000003,"15min":0.81499999999999995},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82599999999999996,"5min":5.7679999999999998,"15min":5.7679999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":1.0189999999999999,"15min":1.077},"min":{"1min":0.46300000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96599999999999997,"5min":9.952,"15min":9.952},"last":0.56799999999999995}]},{"osd":9,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.438,"5min":0.433,"15min":0.433},"max":{"1min":0.751,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.498,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88800000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.65400000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82899999999999996,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.88800000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.57999999999999996}]},{"osd":12,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.84399999999999997,"15min":0.86099999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.99199999999999999,"5min":4.2720000000000002,"15min":4.2720000000000002},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68700000000000006,"15min":0.68000000000000005},"min":{"1min":0.50900000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90300000000000002,"5min":0.995,"15min":0.995},"last":0.77000000000000002}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.79700000000000004}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":6,"up_from":21,"seq":90194313246,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":2076672,"data_stored":1207927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[3,4,5,7,8,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.5}]},{"osd":2,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.53700000000000003}]},{"osd":3,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.45600000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":5.141},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":8.0090000000000003}]},{"osd":5,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.38300000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.91700000000000004,"5min":1.083,"15min":1.083},"last":0.59999999999999998}]},{"osd":7,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59999999999999998,"15min":0.60399999999999998},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77600000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.626},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66100000000000003,"15min":0.66400000000000003},"min":{"1min":0.36499999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.89400000000000002,"5min":1.056,"15min":1.056},"last":0.56399999999999995}]},{"osd":8,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.755,"15min":0.77000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88300000000000001,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":6.1130000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.80100000000000005,"15min":0.81200000000000006},"min":{"1min":0.53800000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92000000000000004,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":6.0839999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":10,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67900000000000005,"15min":0.68700000000000006},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92500000000000004,"5min":1.129,"15min":1.129},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.67700000000000005,"15min":0.69099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88,"5min":1.272,"15min":1.272},"last":0.51400000000000001}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.92100000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.78100000000000003,"15min":0.79000000000000004},"min":{"1min":0.52200000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96599999999999997,"5min":1.167,"15min":1.167},"last":0.72099999999999997}]},{"osd":12,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.83499999999999996,"15min":0.84799999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.085,"5min":1.444,"15min":1.444},"last":5.351},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84899999999999998,"15min":0.84699999999999998},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.0089999999999999}]},{"osd":13,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.81699999999999995,"15min":0.82299999999999995},"min":{"1min":0.56899999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0329999999999999,"5min":1.482,"15min":1.482},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.753,"15min":0.76000000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94999999999999996,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.70199999999999996}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.65700000000000003}]},{"osd":15,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.85999999999999999,"15min":0.87},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0049999999999999,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97599999999999998,"5min":1.327,"15min":1.327},"last":0.65200000000000002}]}]},{"osd":1,"up_from":21,"seq":90194313247,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33140,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,4,5,6,7,8,9,10,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.75600000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67400000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.41999999999999998}]},{"osd":2,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70199999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79600000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61799999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.76000000000000001}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67800000000000005,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72099999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.39000000000000001}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.497,"15min":0.5},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.499,"15min":0.50700000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.33300000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82599999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.58099999999999996}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66800000000000004,"15min":0.66300000000000003},"min":{"1min":0.48099999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65900000000000003,"15min":0.64900000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.59699999999999998}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.88300000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.72199999999999998,"15min":0.73999999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84899999999999998,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.48099999999999998}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.83499999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.435},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67800000000000005,"15min":0.68400000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92900000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.46200000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.70899999999999996}]}]},{"osd":0,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":2056192,"data_stored":1197753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64500000000000002,"15min":0.63400000000000001},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.76200000000000001}]},{"osd":2,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.90200000000000002,"15min":0.95599999999999996},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.89100000000000001,"5min":8.0619999999999994,"15min":8.0619999999999994},"last":0.626},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.1399999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.53300000000000003}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.80500000000000005,"5min":3.363,"15min":3.363},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.90000000000000002,"15min":0.93400000000000005},"min":{"1min":0.50800000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.998,"5min":7.9710000000000001,"15min":7.9710000000000001},"last":0.878}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66000000000000003,"15min":0.66500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97599999999999998,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90200000000000002,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":0.66700000000000004}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.94199999999999995,"15min":0.98699999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1779999999999999,"5min":8.0239999999999991,"15min":8.0239999999999991},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1739999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.90500000000000003}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.72999999999999998,"15min":0.748},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86299999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.98399999999999999,"15min":1.0229999999999999},"min":{"1min":0.42899999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1200000000000001,"5min":7.8630000000000004,"15min":7.8630000000000004},"last":0.91900000000000004}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.67800000000000005,"15min":0.69499999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.92100000000000004,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.95799999999999996,"15min":1.02},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.93400000000000005,"5min":8.1189999999999998,"15min":8.1189999999999998},"last":0.95099999999999996}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.81200000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.01,"5min":3.5699999999999998,"15min":3.5699999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.55300000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.044,"5min":1.927,"15min":1.927},"last":1.0209999999999999}]},{"osd":13,"last update":"Sat Apr 20 01:48:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.84899999999999998}]},{"osd":15,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.097,"5min":1.833,"15min":1.833},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.86899999999999999,"15min":0.88500000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96699999999999997,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":0.97099999999999997}]}]},{"osd":2,"up_from":21,"seq":90194313246,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29044,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342796,"statfs":{"total":96636764160,"available":96607023104,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.42799999999999999}]},{"osd":1,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54600000000000004,"15min":0.53900000000000003},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.72199999999999998}]},{"osd":3,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82199999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":7,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82799999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":0.60299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72299999999999998,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.499}]},{"osd":8,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65000000000000002,"15min":0.66100000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90100000000000002,"5min":2.512,"15min":2.512},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.68400000000000005,"15min":0.69599999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.77600000000000002,"5min":2.532,"15min":2.532},"last":0.63300000000000001}]},{"osd":9,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.435,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64800000000000002},"min":{"1min":0.43099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93000000000000005,"5min":1.107,"15min":1.107},"last":0.65500000000000003}]},{"osd":11,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0269999999999999,"5min":1.135,"15min":1.135},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68600000000000005,"15min":0.69399999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88900000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70299999999999996}]},{"osd":12,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.75800000000000001,"15min":0.78500000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.872,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.80200000000000005,"15min":0.82899999999999996},"min":{"1min":0.42499999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93899999999999995,"5min":3.492,"15min":3.492},"last":0.59099999999999997}]},{"osd":13,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.748}]},{"osd":15,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.60699999999999998}]}]},{"osd":3,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33140,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,4,5,6,7,8,9,10,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.374}]},{"osd":2,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.36199999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.90500000000000003,"5min":0.996,"15min":0.996},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.85599999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.49199999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.56499999999999995,"15min":0.56899999999999995},"min":{"1min":0.34000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69399999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.94599999999999995,"15min":1.014},"min":{"1min":0.39000000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83899999999999997,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":0.61599999999999999}]},{"osd":5,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.56299999999999994,"15min":0.57199999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66500000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.317},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55600000000000005,"15min":0.56000000000000005},"min":{"1min":0.36399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78700000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.44500000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.38600000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89300000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.432}]},{"osd":7,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66200000000000003,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.54800000000000004,"15min":0.56000000000000005},"min":{"1min":0.30099999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71299999999999997,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.41099999999999998}]},{"osd":8,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.41199999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85099999999999998,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.0780000000000001,"15min":1.1479999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81999999999999995,"5min":9.4700000000000006,"15min":9.4700000000000006},"last":0.54800000000000004}]},{"osd":9,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.622,"15min":0.625},"min":{"1min":0.45200000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84699999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.71199999999999997,"15min":0.72599999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74399999999999999,"5min":1.306,"15min":1.306},"last":0.60399999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":12,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":10.959,"5min":10.959,"15min":10.959},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.49399999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.63800000000000001}]}]},{"osd":4,"up_from":21,"seq":90194313248,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":2076672,"data_stored":1207927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,5,6,7,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.81000000000000005,"15min":0.87},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.93400000000000005,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.46700000000000003,"5min":1.1579999999999999,"15min":1.2729999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.008,"5min":16.27,"15min":16.27},"last":0.38800000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":1.012,"15min":1.081},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.92700000000000005,"5min":10.507999999999999,"15min":10.507999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":1.1559999999999999,"15min":1.23},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6319999999999999,"5min":11.019,"15min":11.019},"last":0.56000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":3,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.997,"15min":1.0469999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4750000000000001,"5min":4.9180000000000001,"15min":4.9180000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":1.0449999999999999,"15min":1.095},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.6000000000000001,"5min":10.619,"15min":10.619},"last":0.59699999999999998}]},{"osd":5,"last update":"Sat Apr 20 01:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":1.099,"15min":1.169},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.673,"5min":10.895,"15min":10.895},"last":0.622},{"interface":"front","average":{"1min":0.64200000000000002,"5min":1.012,"15min":1.0740000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1910000000000001,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":0.498}]},{"osd":6,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":1.1890000000000001,"15min":1.2649999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1719999999999999,"5min":11.051,"15min":11.051},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":1.1279999999999999,"15min":1.2},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96199999999999997,"5min":10.589,"15min":10.589},"last":0.76600000000000001}]},{"osd":7,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":1.1950000000000001,"15min":1.2649999999999999},"min":{"1min":0.56000000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.571,"5min":10.957000000000001,"15min":10.957000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.79400000000000004,"5min":1.1799999999999999,"15min":1.244},"min":{"1min":0.47299999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3080000000000001,"5min":10.656000000000001,"15min":10.656000000000001},"last":0.66200000000000003}]},{"osd":8,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.79,"5min":3.79,"15min":3.79},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.0269999999999999}]},{"osd":9,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.385,"15min":1.385},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":10.929,"5min":10.929,"15min":10.929},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":10.555999999999999,"5min":10.555999999999999,"15min":10.555999999999999},"last":1.077}]},{"osd":10,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":1.149,"15min":1.204},"min":{"1min":0.57799999999999996,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.331,"5min":5.0229999999999997,"15min":5.0229999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":1.2729999999999999,"15min":1.351},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2330000000000001,"5min":10.69,"15min":10.69},"last":0.69899999999999995}]},{"osd":11,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.3600000000000001,"15min":1.4410000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.448,"5min":10.987,"15min":10.987},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":1.1879999999999999,"15min":1.244},"min":{"1min":0.63700000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.2290000000000001,"5min":4.9900000000000002,"15min":4.9900000000000002},"last":0.63500000000000001}]},{"osd":12,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.3240000000000001,"15min":1.4059999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.181,"5min":10.720000000000001,"15min":10.720000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.85999999999999999,"5min":1.589,"15min":1.7110000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3149999999999999,"5min":16.617000000000001,"15min":16.617000000000001},"last":0.70899999999999996}]},{"osd":13,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":3.8999999999999999,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":0.745},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":4.5019999999999998,"5min":4.5019999999999998,"15min":4.5019999999999998},"last":0.71799999999999997}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]},{"osd":5,"up_from":21,"seq":90194313247,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29044,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342796,"statfs":{"total":96636764160,"available":96607023104,"internally_reserved":0,"allocated":2060288,"data_stored":1198339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,8,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.85699999999999998,"15min":0.92100000000000004},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77600000000000002,"5min":10.423999999999999,"15min":10.423999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.92500000000000004,"15min":0.98399999999999999},"min":{"1min":0.374,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73499999999999999,"5min":10.465,"15min":10.465},"last":0.76900000000000002}]},{"osd":2,"last update":"Sat Apr 20 01:49:22 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.33600000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.42799999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.755},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.65900000000000003}]},{"osd":4,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.81999999999999995,"15min":0.86799999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71499999999999997,"5min":8.5269999999999992,"15min":8.5269999999999992},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60999999999999999,"15min":0.61699999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78100000000000003,"5min":2.79,"15min":2.79},"last":0.48699999999999999}]},{"osd":6,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84599999999999997,"5min":1.115,"15min":1.115},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.63100000000000001},"min":{"1min":0.38900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94599999999999995,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.51800000000000002}]},{"osd":7,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78800000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.626},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.49399999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.63800000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":10,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.91300000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.54500000000000004}]},{"osd":11,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71899999999999997,"15min":0.71199999999999997},"min":{"1min":0.53900000000000003,"5min":0.435,"15min":0.435},"max":{"1min":1.242,"5min":1.956,"15min":1.956},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70299999999999996,"15min":0.69199999999999995},"min":{"1min":0.55000000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.72599999999999998}]},{"osd":12,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":1.052,"15min":1.1080000000000001},"min":{"1min":0.55800000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90600000000000003,"5min":10.616,"15min":10.616},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.755,"5min":1.0900000000000001,"15min":1.1459999999999999},"min":{"1min":0.56999999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.98899999999999999,"5min":10.629,"15min":10.629},"last":0.82499999999999996}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.84699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T01:49:36.893 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-20T01:49:36.893 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T01:49:37.265 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:49:37.267 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T01:49:37.280 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-04-20T01:49:37.244590+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":505164,"kb_used_data":33716,"kb_used_omap":38,"kb_used_meta":471193,"kb_avail":1509444276,"statfs":{"total":1546188226560,"available":1545670938624,"internally_reserved":0,"allocated":34525184,"data_stored":20527466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39290,"internal_metadata":482502278},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.079124"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":130,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-20T01:49:31.216341+0000","last_change":"2024-04-20T01:48:41.550435+0000","last_active":"2024-04-20T01:49:31.216341+0000","last_peered":"2024-04-20T01:49:31.216341+0000","last_clean":"2024-04-20T01:49:31.216341+0000","last_became_active":"2024-04-20T01:47:20.071615+0000","last_became_peered":"2024-04-20T01:47:20.071615+0000","last_unstale":"2024-04-20T01:49:31.216341+0000","last_undegraded":"2024-04-20T01:49:31.216341+0000","last_fullsized":"2024-04-20T01:49:31.216341+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:41.550289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:41.550289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:05.389116+0000","scrub_duration":0.0015320100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":131,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-20T01:49:31.368178+0000","last_change":"2024-04-20T01:48:36.889025+0000","last_active":"2024-04-20T01:49:31.368178+0000","last_peered":"2024-04-20T01:49:31.368178+0000","last_clean":"2024-04-20T01:49:31.368178+0000","last_became_active":"2024-04-20T01:47:20.075314+0000","last_became_peered":"2024-04-20T01:47:20.075314+0000","last_unstale":"2024-04-20T01:49:31.368178+0000","last_undegraded":"2024-04-20T01:49:31.368178+0000","last_fullsized":"2024-04-20T01:49:31.368178+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:36.888923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:36.888923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:45.012481+0000","scrub_duration":0.0018745420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,12],"acting":[2,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":127,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-20T01:49:31.333563+0000","last_change":"2024-04-20T01:48:26.145238+0000","last_active":"2024-04-20T01:49:31.333563+0000","last_peered":"2024-04-20T01:49:31.333563+0000","last_clean":"2024-04-20T01:49:31.333563+0000","last_became_active":"2024-04-20T01:47:20.038972+0000","last_became_peered":"2024-04-20T01:47:20.038972+0000","last_unstale":"2024-04-20T01:49:31.333563+0000","last_undegraded":"2024-04-20T01:49:31.333563+0000","last_fullsized":"2024-04-20T01:49:31.333563+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:26.145095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:26.145095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:54.362058+0000","scrub_duration":0.0015014830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":141,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-20T01:49:31.333489+0000","last_change":"2024-04-20T01:49:28.041091+0000","last_active":"2024-04-20T01:49:31.333489+0000","last_peered":"2024-04-20T01:49:31.333489+0000","last_clean":"2024-04-20T01:49:31.333489+0000","last_became_active":"2024-04-20T01:47:20.038764+0000","last_became_peered":"2024-04-20T01:47:20.038764+0000","last_unstale":"2024-04-20T01:49:31.333489+0000","last_undegraded":"2024-04-20T01:49:31.333489+0000","last_fullsized":"2024-04-20T01:49:31.333489+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:49:28.040983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:49:28.040983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:54.519868+0000","scrub_duration":0.0021762000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"25'157","reported_seq":325,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-20T01:49:31.333567+0000","last_change":"2024-04-20T01:48:36.239045+0000","last_active":"2024-04-20T01:49:31.333567+0000","last_peered":"2024-04-20T01:49:31.333567+0000","last_clean":"2024-04-20T01:49:31.333567+0000","last_became_active":"2024-04-20T01:47:17.302105+0000","last_became_peered":"2024-04-20T01:47:17.302105+0000","last_unstale":"2024-04-20T01:49:31.333567+0000","last_undegraded":"2024-04-20T01:49:31.333567+0000","last_fullsized":"2024-04-20T01:49:31.333567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'157","last_scrub_stamp":"2024-04-20T01:48:36.238909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:15.986754+0000","last_clean_scrub_stamp":"2024-04-20T01:48:36.238909+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:57.382841+0000","scrub_duration":0.0025722610000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,15],"acting":[13,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":83,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-20T01:49:31.368224+0000","last_change":"2024-04-20T01:49:17.016142+0000","last_active":"2024-04-20T01:49:31.368224+0000","last_peered":"2024-04-20T01:49:31.368224+0000","last_clean":"2024-04-20T01:49:31.368224+0000","last_became_active":"2024-04-20T01:49:17.015873+0000","last_became_peered":"2024-04-20T01:49:17.015873+0000","last_unstale":"2024-04-20T01:49:31.368224+0000","last_undegraded":"2024-04-20T01:49:31.368224+0000","last_fullsized":"2024-04-20T01:49:31.368224+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:37.682441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:37.682441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:54.853315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":131,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-20T01:49:31.336075+0000","last_change":"2024-04-20T01:48:48.595681+0000","last_active":"2024-04-20T01:49:31.336075+0000","last_peered":"2024-04-20T01:49:31.336075+0000","last_clean":"2024-04-20T01:49:31.336075+0000","last_became_active":"2024-04-20T01:47:20.038107+0000","last_became_peered":"2024-04-20T01:47:20.038107+0000","last_unstale":"2024-04-20T01:49:31.336075+0000","last_undegraded":"2024-04-20T01:49:31.336075+0000","last_fullsized":"2024-04-20T01:49:31.336075+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:48.595563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:48.595563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:00.568790+0000","scrub_duration":0.002598885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":129,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-20T01:49:31.347469+0000","last_change":"2024-04-20T01:48:48.578126+0000","last_active":"2024-04-20T01:49:31.347469+0000","last_peered":"2024-04-20T01:49:31.347469+0000","last_clean":"2024-04-20T01:49:31.347469+0000","last_became_active":"2024-04-20T01:47:20.110260+0000","last_became_peered":"2024-04-20T01:47:20.110260+0000","last_unstale":"2024-04-20T01:49:31.347469+0000","last_undegraded":"2024-04-20T01:49:31.347469+0000","last_fullsized":"2024-04-20T01:49:31.347469+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:48.577995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:48.577995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:53.805538+0000","scrub_duration":0.002163692,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":129,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-20T01:49:31.168167+0000","last_change":"2024-04-20T01:48:37.432504+0000","last_active":"2024-04-20T01:49:31.168167+0000","last_peered":"2024-04-20T01:49:31.168167+0000","last_clean":"2024-04-20T01:49:31.168167+0000","last_became_active":"2024-04-20T01:47:20.036328+0000","last_became_peered":"2024-04-20T01:47:20.036328+0000","last_unstale":"2024-04-20T01:49:31.168167+0000","last_undegraded":"2024-04-20T01:49:31.168167+0000","last_fullsized":"2024-04-20T01:49:31.168167+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:37.432331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:37.432331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:49:49.945535+0000","scrub_duration":0.0020345559999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,13],"acting":[6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280544,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33676,"kb_used_data":2560,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338164,"statfs":{"total":96636764160,"available":96602279936,"internally_reserved":0,"allocated":2621440,"data_stored":1745890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[2,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.45100000000000001}]},{"osd":2,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.59799999999999998,"15min":0.61099999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.74299999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.64000000000000001,"15min":0.65100000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84699999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.84099999999999997}]},{"osd":3,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.47699999999999998}]},{"osd":4,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":1.0760000000000001,"15min":1.155},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.747,"5min":9.0129999999999999,"15min":9.0129999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.65900000000000003,"15min":0.67500000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.747,"5min":1.022,"15min":1.022},"last":0.86899999999999999}]},{"osd":5,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.59199999999999997,"15min":0.60899999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73999999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.56899999999999995,"15min":0.58299999999999996},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72299999999999998,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.40100000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.66100000000000003,"15min":0.67600000000000005},"min":{"1min":0.434,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77700000000000002,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.64700000000000002,"15min":0.66500000000000004},"min":{"1min":0.40100000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.66400000000000003,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.57099999999999995}]},{"osd":7,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.755,"15min":0.76800000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95999999999999996,"5min":1.143,"15min":1.143},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.78100000000000003,"15min":0.80100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97999999999999998,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.54500000000000004}]},{"osd":10,"last update":"Sat Apr 20 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.77300000000000002,"15min":0.79200000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86199999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.878,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.82299999999999995}]},{"osd":11,"last update":"Sat Apr 20 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.73699999999999999,"15min":0.752},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91900000000000004,"5min":1.109,"15min":1.109},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.84799999999999998,"15min":0.86799999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.92700000000000005,"5min":1.583,"15min":1.583},"last":0.58299999999999996}]},{"osd":12,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.77800000000000002,"15min":0.80000000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.83699999999999997,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.746,"5min":1.196,"15min":1.2709999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93999999999999995,"5min":9.1389999999999993,"15min":9.1389999999999993},"last":0.88300000000000001}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.627},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.72199999999999998}]},{"osd":15,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.96099999999999997,"15min":0.98999999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.99399999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.748},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.88300000000000001,"15min":0.90600000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.94699999999999995,"5min":1.399,"15min":1.399},"last":0.73599999999999999}]}]},{"osd":15,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29584,"kb_used_data":2564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342256,"statfs":{"total":96636764160,"available":96606470144,"internally_reserved":0,"allocated":2625536,"data_stored":1746493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,3,4,5,6,7,8,9,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.34100000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.67300000000000004,"15min":0.70299999999999996},"min":{"1min":0.36499999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72399999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":0.438}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":2,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.68000000000000005}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42399999999999999,"15min":0.42599999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.76100000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.36899999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.38800000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.38700000000000001}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84299999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.63100000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.627,"15min":0.63400000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86499999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63,"15min":0.63300000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.94399999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.60199999999999998}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.93000000000000005,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94899999999999995,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.53700000000000003}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45900000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96099999999999997,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.86199999999999999,"15min":0.90000000000000002},"min":{"1min":0.43099999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97599999999999998,"5min":4.5979999999999999,"15min":4.5979999999999999},"last":0.52300000000000002}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68400000000000005,"15min":0.68700000000000006},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93899999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.748,"15min":0.746},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.016,"5min":1.272,"15min":1.272},"last":0.68600000000000005}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.70999999999999996}]}]},{"osd":14,"up_from":102,"seq":438086664202,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31288,"kb_used_data":2024,"kb_used_omap":15,"kb_used_meta":29232,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":2072576,"data_stored":1207324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15377,"internal_metadata":29934575},"hb_peers":[0,1,2,3,4,5,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":12,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29064,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342776,"statfs":{"total":96636764160,"available":96607002624,"internally_reserved":0,"allocated":2093056,"data_stored":1217565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":1,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.58499999999999996,"15min":0.55100000000000005},"min":{"1min":0.65800000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66800000000000004,"15min":0.66200000000000003},"min":{"1min":0.504,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93799999999999994,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.69499999999999995}]},{"osd":2,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.48499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.441}]},{"osd":3,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88700000000000001,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.751},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.38100000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79800000000000004,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.878}]},{"osd":4,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.39500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.59999999999999998,"15min":0.61199999999999999},"min":{"1min":0.371,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82399999999999995,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.61599999999999999}]},{"osd":5,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.65500000000000003,"15min":0.63700000000000001},"min":{"1min":0.55200000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0940000000000001,"5min":1.115,"15min":1.115},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.47499999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.0800000000000001,"5min":1.726,"15min":1.726},"last":0.65500000000000003}]},{"osd":6,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.66200000000000003,"15min":0.66600000000000004},"min":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.86699999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.74299999999999999,"15min":0.72899999999999998},"min":{"1min":0.63700000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0349999999999999,"5min":1.069,"15min":1.069},"last":0.63400000000000001}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.76300000000000001,"15min":0.76500000000000001},"min":{"1min":0.54400000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0529999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.95399999999999996,"15min":0.98999999999999999},"min":{"1min":0.53800000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.012,"5min":6.6779999999999999,"15min":6.6779999999999999},"last":0.84499999999999997}]},{"osd":13,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.79800000000000004},"min":{"1min":0.59699999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0189999999999999,"5min":1.827,"15min":1.827},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.80900000000000005,"15min":0.79500000000000004},"min":{"1min":0.753,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1739999999999999,"5min":1.258,"15min":1.258},"last":0.58699999999999997}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":1.135},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.99299999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.79000000000000004},"min":{"1min":0.58199999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0109999999999999,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.82999999999999996,"15min":0.81699999999999995},"min":{"1min":0.753,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.67800000000000005}]}]},{"osd":11,"up_from":21,"seq":90194313247,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33160,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338680,"statfs":{"total":96636764160,"available":96602808320,"internally_reserved":0,"allocated":2093056,"data_stored":1217565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,2,3,4,8,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.58599999999999997,"15min":0.59799999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.83899999999999997,"5min":1.669,"15min":1.669},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.67300000000000004,"15min":0.68300000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0900000000000001,"5min":1.75,"15min":1.75},"last":3.0230000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.58099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.40300000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.31},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.28799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.33400000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.88,"15min":0.93500000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1220000000000001,"5min":8.5969999999999995,"15min":8.5969999999999995},"last":3.2709999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.67000000000000004,"15min":0.67900000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.147,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":3.302}]},{"osd":8,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":3.3559999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.68300000000000005}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.57499999999999996,"15min":0.56200000000000006},"min":{"1min":0.38600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.622}]},{"osd":10,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.745},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47199999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.72799999999999998}]},{"osd":12,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85699999999999998,"5min":1.875,"15min":1.875},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":1.262,"15min":1.349},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1879999999999999,"5min":12.385,"15min":12.385},"last":0.69299999999999995}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.54900000000000004}]},{"osd":15,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.75,"15min":0.76100000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.96999999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.47199999999999998}]}]},{"osd":10,"up_from":21,"seq":90194313249,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33160,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338680,"statfs":{"total":96636764160,"available":96602808320,"internally_reserved":0,"allocated":2093056,"data_stored":1217565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,4,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.29999999999999999}]},{"osd":1,"last update":"Sat Apr 20 01:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.62}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.41399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":7,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63300000000000001,"15min":0.626},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.49299999999999999}]},{"osd":8,"last update":"Sat Apr 20 01:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.38},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.59299999999999997}]},{"osd":9,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68999999999999995,"15min":0.68500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.67600000000000005}]},{"osd":11,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.42499999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82099999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.44},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.90200000000000002,"15min":0.92600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.292,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":0.51300000000000001}]},{"osd":12,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":1.208,"15min":1.29},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1120000000000001,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.997,"15min":1.054},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91000000000000003,"5min":9.7759999999999998,"15min":9.7759999999999998},"last":0.47999999999999998}]},{"osd":13,"last update":"Sat Apr 20 01:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89600000000000002,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.71699999999999997,"15min":0.72299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95799999999999996,"5min":1.05,"15min":1.05},"last":0.56799999999999995}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.64200000000000002}]},{"osd":15,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.80100000000000005,"15min":0.79700000000000004},"min":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.53400000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.93700000000000006,"5min":1.157,"15min":1.157},"last":0.54300000000000004}]}]},{"osd":9,"up_from":21,"seq":90194313249,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":29068,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342772,"statfs":{"total":96636764160,"available":96606998528,"internally_reserved":0,"allocated":2093056,"data_stored":1217565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,2,4,6,7,8,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47099999999999997,"15min":0.46300000000000002},"min":{"1min":0.32800000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.52500000000000002,"15min":0.51300000000000001},"min":{"1min":0.40300000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83599999999999997,"5min":0.998,"15min":0.998},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":4,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.64300000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.872,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.95899999999999996}]},{"osd":7,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.83999999999999997}]},{"osd":8,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.39600000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":5.5720000000000001}]},{"osd":10,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.93799999999999994,"5min":1.03,"15min":1.03},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58699999999999997,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76300000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.84099999999999997}]},{"osd":11,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88400000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.755}]},{"osd":12,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.74099999999999999,"15min":0.73899999999999999},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97899999999999998,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.89900000000000002,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.90500000000000003}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.56000000000000005}]},{"osd":15,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.745,"15min":0.753},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":1.075},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69699999999999995,"15min":0.70499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88800000000000001,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.80400000000000005}]}]},{"osd":8,"up_from":21,"seq":90194313248,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33160,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338680,"statfs":{"total":96636764160,"available":96602808320,"internally_reserved":0,"allocated":2093056,"data_stored":1217565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.34200000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89700000000000002,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.794},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.628,"15min":0.64100000000000001},"min":{"1min":0.39500000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.81699999999999995,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.97299999999999998}]},{"osd":3,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.45400000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85799999999999998,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":2.0139999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.73599999999999999,"15min":0.753},"min":{"1min":0.44500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91600000000000004,"5min":1.821,"15min":1.821},"last":1.022}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":6,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70199999999999996,"15min":0.70999999999999996},"min":{"1min":0.46500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89900000000000002,"5min":1.784,"15min":1.784},"last":2.4569999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.44,"5min":0.437,"15min":0.437},"max":{"1min":0.79300000000000004,"5min":1.885,"15min":1.885},"last":1.2350000000000001}]},{"osd":7,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.72599999999999998,"15min":0.73499999999999999},"min":{"1min":0.50800000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93300000000000005,"5min":1.411,"15min":1.411},"last":2.073},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.81999999999999995,"15min":0.84399999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95199999999999996,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":1.0529999999999999}]},{"osd":9,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.72199999999999998,"15min":0.73599999999999999},"min":{"1min":0.436,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91800000000000004,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.3340000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.139,"5min":1.375,"15min":1.375},"last":1.835}]},{"osd":10,"last update":"Sat Apr 20 01:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":1.274},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":1.8660000000000001}]},{"osd":11,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.83899999999999997,"15min":0.84299999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.141,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.899},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.92400000000000004,"15min":0.93999999999999995},"min":{"1min":0.66100000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.071,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":2.0430000000000001}]},{"osd":12,"last update":"Sat Apr 20 01:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":1.9319999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":2.415}]},{"osd":13,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":1.3069999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":1.988}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":15,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.88700000000000001,"15min":0.89100000000000001},"min":{"1min":0.621,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.095,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.96},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.95399999999999996,"15min":0.98399999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0229999999999999,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":1.095}]}]},{"osd":7,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29072,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342768,"statfs":{"total":96636764160,"available":96606994432,"internally_reserved":0,"allocated":2093056,"data_stored":1217565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,3,4,6,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.38800000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.59399999999999997}]},{"osd":3,"last update":"Sat Apr 20 01:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.435,"15min":0.438},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.58399999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.441,"5min":0.438,"15min":0.438},"min":{"1min":0.307,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.42099999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.379,"5min":0.372,"15min":0.372},"max":{"1min":0.92600000000000005,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.47199999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94399999999999995,"5min":4.1920000000000002,"15min":4.1920000000000002},"last":0.67900000000000005}]},{"osd":6,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.499},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.51000000000000001}]},{"osd":8,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.78900000000000003,"15min":0.81499999999999995},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82599999999999996,"5min":5.7679999999999998,"15min":5.7679999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":1.0189999999999999,"15min":1.077},"min":{"1min":0.46300000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96599999999999997,"5min":9.952,"15min":9.952},"last":0.44700000000000001}]},{"osd":9,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.438,"5min":0.433,"15min":0.433},"max":{"1min":0.751,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.498,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88800000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.63900000000000001}]},{"osd":10,"last update":"Sat Apr 20 01:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.629},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.66600000000000004}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82899999999999996,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.88800000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.70999999999999996}]},{"osd":12,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.84399999999999997,"15min":0.86099999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.99199999999999999,"5min":4.2720000000000002,"15min":4.2720000000000002},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68700000000000006,"15min":0.68000000000000005},"min":{"1min":0.50900000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90300000000000002,"5min":0.995,"15min":0.995},"last":0.65200000000000002}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.69699999999999995}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":6,"up_from":21,"seq":90194313247,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33164,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338676,"statfs":{"total":96636764160,"available":96602804224,"internally_reserved":0,"allocated":2093056,"data_stored":1217565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[3,4,5,7,8,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.5}]},{"osd":2,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.53700000000000003}]},{"osd":3,"last update":"Sat Apr 20 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.42199999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76900000000000002,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.76900000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.72099999999999997,"15min":0.68400000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":5.141,"5min":5.141,"15min":5.141},"last":4.532},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.67400000000000004,"15min":0.622},"min":{"1min":0.38500000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":8.0090000000000003,"5min":8.0090000000000003,"15min":8.0090000000000003},"last":2.9910000000000001}]},{"osd":5,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.433},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.38300000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.91700000000000004,"5min":1.083,"15min":1.083},"last":0.39200000000000002}]},{"osd":7,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59999999999999998,"15min":0.60399999999999998},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77600000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66100000000000003,"15min":0.66400000000000003},"min":{"1min":0.36499999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.89400000000000002,"5min":1.056,"15min":1.056},"last":0.69299999999999995}]},{"osd":8,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.755,"15min":0.77000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88300000000000001,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.80100000000000005,"15min":0.81200000000000006},"min":{"1min":0.53800000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92000000000000004,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.85599999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":10,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67900000000000005,"15min":0.68700000000000006},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92500000000000004,"5min":1.129,"15min":1.129},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.67700000000000005,"15min":0.69099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88,"5min":1.272,"15min":1.272},"last":0.51300000000000001}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.92100000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.78100000000000003,"15min":0.79000000000000004},"min":{"1min":0.52200000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96599999999999997,"5min":1.167,"15min":1.167},"last":0.89900000000000002}]},{"osd":12,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.83499999999999996,"15min":0.84799999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.085,"5min":1.444,"15min":1.444},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84899999999999998,"15min":0.84699999999999998},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.59399999999999997}]},{"osd":13,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.81699999999999995,"15min":0.82299999999999995},"min":{"1min":0.56899999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0329999999999999,"5min":1.482,"15min":1.482},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.753,"15min":0.76000000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94999999999999996,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.57499999999999996}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.65700000000000003}]},{"osd":15,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.85999999999999999,"15min":0.87},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0049999999999999,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97599999999999998,"5min":1.327,"15min":1.327},"last":0.79700000000000004}]}]},{"osd":1,"up_from":21,"seq":90194313248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33160,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338680,"statfs":{"total":96636764160,"available":96602808320,"internally_reserved":0,"allocated":2093056,"data_stored":1217565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,4,5,6,7,8,9,10,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.75600000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67400000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.47999999999999998}]},{"osd":2,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70199999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79600000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59999999999999998}]},{"osd":3,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.76000000000000001}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67800000000000005,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72099999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.443}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.497,"15min":0.5},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.499,"15min":0.50700000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.40100000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82599999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64300000000000002}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66800000000000004,"15min":0.66300000000000003},"min":{"1min":0.48099999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.754},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65900000000000003,"15min":0.64900000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.61599999999999999}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.88300000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.72199999999999998,"15min":0.73999999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84899999999999998,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.56699999999999995}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.83499999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67800000000000005,"15min":0.68400000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92900000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68100000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.79000000000000004}]}]},{"osd":0,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2088960,"data_stored":1216979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64500000000000002,"15min":0.63400000000000001},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.54800000000000004}]},{"osd":2,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.90200000000000002,"15min":0.95599999999999996},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.89100000000000001,"5min":8.0619999999999994,"15min":8.0619999999999994},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.1399999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.72899999999999998}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.80500000000000005,"5min":3.363,"15min":3.363},"last":0.748},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.90000000000000002,"15min":0.93400000000000005},"min":{"1min":0.50800000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.998,"5min":7.9710000000000001,"15min":7.9710000000000001},"last":0.89900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66000000000000003,"15min":0.66500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97599999999999998,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90200000000000002,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":0.68700000000000006}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.94199999999999995,"15min":0.98699999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1779999999999999,"5min":8.0239999999999991,"15min":8.0239999999999991},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1739999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.65000000000000002}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.72999999999999998,"15min":0.748},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86299999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.98399999999999999,"15min":1.0229999999999999},"min":{"1min":0.42899999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1200000000000001,"5min":7.8630000000000004,"15min":7.8630000000000004},"last":0.84199999999999997}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.67800000000000005,"15min":0.69499999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.92100000000000004,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.95799999999999996,"15min":1.02},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.93400000000000005,"5min":8.1189999999999998,"15min":8.1189999999999998},"last":0.70899999999999996}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.81200000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.01,"5min":3.5699999999999998,"15min":3.5699999999999998},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.55300000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.044,"5min":1.927,"15min":1.927},"last":0.66800000000000004}]},{"osd":13,"last update":"Sat Apr 20 01:48:34 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.80400000000000005}]},{"osd":15,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.097,"5min":1.833,"15min":1.833},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.86899999999999999,"15min":0.88500000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96699999999999997,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":0.88200000000000001}]}]},{"osd":2,"up_from":21,"seq":90194313247,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29064,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342776,"statfs":{"total":96636764160,"available":96607002624,"internally_reserved":0,"allocated":2093056,"data_stored":1217565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.46600000000000003,"15min":0.45600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.47099999999999997,"15min":0.46300000000000002},"min":{"1min":0.311,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":3.4569999999999999}]},{"osd":1,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54600000000000004,"15min":0.53900000000000003},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.35599999999999998}]},{"osd":3,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82199999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.623}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":7,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82799999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":0.60299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72299999999999998,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60799999999999998}]},{"osd":8,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65000000000000002,"15min":0.66100000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90100000000000002,"5min":2.512,"15min":2.512},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.68400000000000005,"15min":0.69599999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.77600000000000002,"5min":2.532,"15min":2.532},"last":0.82399999999999995}]},{"osd":9,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.435,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64800000000000002},"min":{"1min":0.43099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93000000000000005,"5min":1.107,"15min":1.107},"last":0.39600000000000002}]},{"osd":11,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0269999999999999,"5min":1.135,"15min":1.135},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68600000000000005,"15min":0.69399999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88900000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.59299999999999997}]},{"osd":12,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.75800000000000001,"15min":0.78500000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.872,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.80200000000000005,"15min":0.82899999999999996},"min":{"1min":0.42499999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93899999999999995,"5min":3.492,"15min":3.492},"last":0.93799999999999994}]},{"osd":13,"last update":"Sat Apr 20 01:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.67800000000000005,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.499}]},{"osd":15,"last update":"Sat Apr 20 01:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.66600000000000004,"15min":0.66000000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68000000000000005,"15min":0.67500000000000004},"min":{"1min":0.53600000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64900000000000002}]}]},{"osd":3,"up_from":21,"seq":90194313250,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33160,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338680,"statfs":{"total":96636764160,"available":96602808320,"internally_reserved":0,"allocated":2093056,"data_stored":1217565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,4,5,6,7,8,9,10,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.435},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.53900000000000003}]},{"osd":2,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.36199999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.90500000000000003,"5min":0.996,"15min":0.996},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.85599999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.73299999999999998}]},{"osd":4,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.56499999999999995,"15min":0.56899999999999995},"min":{"1min":0.34000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69399999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.5},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.94599999999999995,"15min":1.014},"min":{"1min":0.39000000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83899999999999997,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":0.48499999999999999}]},{"osd":5,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.56299999999999994,"15min":0.57199999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66500000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55600000000000005,"15min":0.56000000000000005},"min":{"1min":0.36399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78700000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.67700000000000005}]},{"osd":6,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.38600000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89300000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.65000000000000002}]},{"osd":7,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66200000000000003,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.54800000000000004,"15min":0.56000000000000005},"min":{"1min":0.30099999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71299999999999997,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.39800000000000002}]},{"osd":8,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.41199999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85099999999999998,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.0780000000000001,"15min":1.1479999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81999999999999995,"5min":9.4700000000000006,"15min":9.4700000000000006},"last":0.56399999999999995}]},{"osd":9,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.622,"15min":0.625},"min":{"1min":0.45200000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84699999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.71199999999999997,"15min":0.72599999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74399999999999999,"5min":1.306,"15min":1.306},"last":0.622}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":12,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":10.959,"5min":10.959,"15min":10.959},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.49399999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.66000000000000003}]}]},{"osd":4,"up_from":21,"seq":90194313249,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33164,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338676,"statfs":{"total":96636764160,"available":96602804224,"internally_reserved":0,"allocated":2093056,"data_stored":1217565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,5,6,7,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.81000000000000005,"15min":0.87},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.93400000000000005,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":1.1579999999999999,"15min":1.2729999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.008,"5min":16.27,"15min":16.27},"last":0.42999999999999999}]},{"osd":1,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":1.012,"15min":1.081},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.92700000000000005,"5min":10.507999999999999,"15min":10.507999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":1.1559999999999999,"15min":1.23},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6319999999999999,"5min":11.019,"15min":11.019},"last":0.51400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":3,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.997,"15min":1.0469999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4750000000000001,"5min":4.9180000000000001,"15min":4.9180000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":1.0449999999999999,"15min":1.095},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.6000000000000001,"5min":10.619,"15min":10.619},"last":0.57199999999999995}]},{"osd":5,"last update":"Sat Apr 20 01:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":1.099,"15min":1.169},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.673,"5min":10.895,"15min":10.895},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":1.012,"15min":1.0740000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1910000000000001,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":0.53800000000000003}]},{"osd":6,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":1.1890000000000001,"15min":1.2649999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1719999999999999,"5min":11.051,"15min":11.051},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":1.1279999999999999,"15min":1.2},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96199999999999997,"5min":10.589,"15min":10.589},"last":0.60599999999999998}]},{"osd":7,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":1.1950000000000001,"15min":1.2649999999999999},"min":{"1min":0.56000000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.571,"5min":10.957000000000001,"15min":10.957000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":1.1799999999999999,"15min":1.244},"min":{"1min":0.47299999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3080000000000001,"5min":10.656000000000001,"15min":10.656000000000001},"last":0.68200000000000005}]},{"osd":8,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.79,"5min":3.79,"15min":3.79},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.0269999999999999}]},{"osd":9,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.385,"15min":1.385},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":10.929,"5min":10.929,"15min":10.929},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":10.555999999999999,"5min":10.555999999999999,"15min":10.555999999999999},"last":1.077}]},{"osd":10,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":1.149,"15min":1.204},"min":{"1min":0.57799999999999996,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.331,"5min":5.0229999999999997,"15min":5.0229999999999997},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":1.2729999999999999,"15min":1.351},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2330000000000001,"5min":10.69,"15min":10.69},"last":0.63600000000000001}]},{"osd":11,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.3600000000000001,"15min":1.4410000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.448,"5min":10.987,"15min":10.987},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.84799999999999998,"5min":1.1879999999999999,"15min":1.244},"min":{"1min":0.63700000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.2290000000000001,"5min":4.9900000000000002,"15min":4.9900000000000002},"last":0.76300000000000001}]},{"osd":12,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.3240000000000001,"15min":1.4059999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.181,"5min":10.720000000000001,"15min":10.720000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.85999999999999999,"5min":1.589,"15min":1.7110000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3149999999999999,"5min":16.617000000000001,"15min":16.617000000000001},"last":0.71899999999999997}]},{"osd":13,"last update":"Sat Apr 20 01:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.98499999999999999,"15min":1.0089999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.732,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.0469999999999999,"15min":1.075},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.546,"5min":4.5019999999999998,"15min":4.5019999999999998},"last":0.70399999999999996}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]},{"osd":5,"up_from":21,"seq":90194313248,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29064,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342776,"statfs":{"total":96636764160,"available":96607002624,"internally_reserved":0,"allocated":2093056,"data_stored":1217565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,8,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.85699999999999998,"15min":0.92100000000000004},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77600000000000002,"5min":10.423999999999999,"15min":10.423999999999999},"last":0.437},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.92500000000000004,"15min":0.98399999999999999},"min":{"1min":0.374,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73499999999999999,"5min":10.465,"15min":10.465},"last":0.52500000000000002}]},{"osd":2,"last update":"Sat Apr 20 01:49:22 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.33600000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.59599999999999997}]},{"osd":3,"last update":"Sat Apr 20 01:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.46999999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.443,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.83399999999999996}]},{"osd":4,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.81999999999999995,"15min":0.86799999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71499999999999997,"5min":8.5269999999999992,"15min":8.5269999999999992},"last":0.504},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60999999999999999,"15min":0.61699999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78100000000000003,"5min":2.79,"15min":2.79},"last":0.72699999999999998}]},{"osd":6,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84599999999999997,"5min":1.115,"15min":1.115},"last":0.62},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.63100000000000001},"min":{"1min":0.38900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94599999999999995,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.68799999999999994}]},{"osd":7,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78800000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.49399999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.71299999999999997}]},{"osd":8,"last update":"Sat Apr 20 01:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.63500000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":10,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.91300000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.753},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.67000000000000004}]},{"osd":11,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71899999999999997,"15min":0.71199999999999997},"min":{"1min":0.53900000000000003,"5min":0.435,"15min":0.435},"max":{"1min":1.242,"5min":1.956,"15min":1.956},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70299999999999996,"15min":0.69199999999999995},"min":{"1min":0.55000000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.78000000000000003}]},{"osd":12,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":1.052,"15min":1.1080000000000001},"min":{"1min":0.55800000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90600000000000003,"5min":10.616,"15min":10.616},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.755,"5min":1.0900000000000001,"15min":1.1459999999999999},"min":{"1min":0.56999999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.98899999999999999,"5min":10.629,"15min":10.629},"last":0.85699999999999998}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.84699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T01:49:37.284 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T01:49:37.651 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:49:37.652 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":138,"fsid":"9c2f64c1-aa35-4337-9f5c-081b9e3a2938","created":"2024-04-20T01:47:02.934241+0000","modified":"2024-04-20T01:49:37.237017+0000","last_up_change":"2024-04-20T01:48:54.636657+0000","last_in_change":"2024-04-20T01:47:07.894719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T01:47:15.195181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-20T01:47:18.448004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3.9900000095367432,"score_stable":3.9900000095367432,"optimal_score":0.68999999761581421,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3903b88f-ff2b-4252-9480-92b701b1e59b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":43518},{"type":"v1","addr":"172.21.15.44:6825","nonce":43518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":43518},{"type":"v1","addr":"172.21.15.44:6827","nonce":43518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":43518},{"type":"v1","addr":"172.21.15.44:6831","nonce":43518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":43518},{"type":"v1","addr":"172.21.15.44:6829","nonce":43518}]},"public_addr":"172.21.15.44:6825/43518","cluster_addr":"172.21.15.44:6827/43518","heartbeat_back_addr":"172.21.15.44:6831/43518","heartbeat_front_addr":"172.21.15.44:6829/43518","state":["exists","up"]},{"osd":1,"uuid":"c5ff0175-b4f0-40a5-86c7-7a0a351bf070","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":74361},{"type":"v1","addr":"172.21.15.121:6817","nonce":74361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":74361},{"type":"v1","addr":"172.21.15.121:6819","nonce":74361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":74361},{"type":"v1","addr":"172.21.15.121:6823","nonce":74361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":74361},{"type":"v1","addr":"172.21.15.121:6821","nonce":74361}]},"public_addr":"172.21.15.121:6817/74361","cluster_addr":"172.21.15.121:6819/74361","heartbeat_back_addr":"172.21.15.121:6823/74361","heartbeat_front_addr":"172.21.15.121:6821/74361","state":["exists","up"]},{"osd":2,"uuid":"7af325c1-36b5-420c-a2a9-44c00a002977","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6809","nonce":43108},{"type":"v1","addr":"172.21.15.123:6811","nonce":43108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":43108},{"type":"v1","addr":"172.21.15.123:6816","nonce":43108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":43108},{"type":"v1","addr":"172.21.15.123:6823","nonce":43108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6819","nonce":43108},{"type":"v1","addr":"172.21.15.123:6821","nonce":43108}]},"public_addr":"172.21.15.123:6811/43108","cluster_addr":"172.21.15.123:6816/43108","heartbeat_back_addr":"172.21.15.123:6823/43108","heartbeat_front_addr":"172.21.15.123:6821/43108","state":["exists","up"]},{"osd":3,"uuid":"99334f43-83ea-4c3f-b91d-ae72c8361b8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":74545},{"type":"v1","addr":"172.21.15.165:6825","nonce":74545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":74545},{"type":"v1","addr":"172.21.15.165:6827","nonce":74545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":74545},{"type":"v1","addr":"172.21.15.165:6831","nonce":74545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":74545},{"type":"v1","addr":"172.21.15.165:6829","nonce":74545}]},"public_addr":"172.21.15.165:6825/74545","cluster_addr":"172.21.15.165:6827/74545","heartbeat_back_addr":"172.21.15.165:6831/74545","heartbeat_front_addr":"172.21.15.165:6829/74545","state":["exists","up"]},{"osd":4,"uuid":"8764d799-af67-4e6c-889f-9fdd91201cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6809","nonce":43517},{"type":"v1","addr":"172.21.15.44:6811","nonce":43517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6813","nonce":43517},{"type":"v1","addr":"172.21.15.44:6815","nonce":43517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6821","nonce":43517},{"type":"v1","addr":"172.21.15.44:6823","nonce":43517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6817","nonce":43517},{"type":"v1","addr":"172.21.15.44:6819","nonce":43517}]},"public_addr":"172.21.15.44:6811/43517","cluster_addr":"172.21.15.44:6815/43517","heartbeat_back_addr":"172.21.15.44:6823/43517","heartbeat_front_addr":"172.21.15.44:6819/43517","state":["exists","up"]},{"osd":5,"uuid":"2556b2fa-4dcc-479c-a379-6f9fe96e9c77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":74354},{"type":"v1","addr":"172.21.15.121:6801","nonce":74354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":74354},{"type":"v1","addr":"172.21.15.121:6803","nonce":74354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":74354},{"type":"v1","addr":"172.21.15.121:6807","nonce":74354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":74354},{"type":"v1","addr":"172.21.15.121:6805","nonce":74354}]},"public_addr":"172.21.15.121:6801/74354","cluster_addr":"172.21.15.121:6803/74354","heartbeat_back_addr":"172.21.15.121:6807/74354","heartbeat_front_addr":"172.21.15.121:6805/74354","state":["exists","up"]},{"osd":6,"uuid":"baeaae3a-fff4-4ccc-a08a-ff32db4531ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":43107},{"type":"v1","addr":"172.21.15.123:6801","nonce":43107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":43107},{"type":"v1","addr":"172.21.15.123:6803","nonce":43107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":43107},{"type":"v1","addr":"172.21.15.123:6807","nonce":43107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":43107},{"type":"v1","addr":"172.21.15.123:6805","nonce":43107}]},"public_addr":"172.21.15.123:6801/43107","cluster_addr":"172.21.15.123:6803/43107","heartbeat_back_addr":"172.21.15.123:6807/43107","heartbeat_front_addr":"172.21.15.123:6805/43107","state":["exists","up"]},{"osd":7,"uuid":"0019f005-3e62-4d9d-b334-f1d347a484ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":74543},{"type":"v1","addr":"172.21.15.165:6801","nonce":74543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":74543},{"type":"v1","addr":"172.21.15.165:6803","nonce":74543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":74543},{"type":"v1","addr":"172.21.15.165:6807","nonce":74543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":74543},{"type":"v1","addr":"172.21.15.165:6805","nonce":74543}]},"public_addr":"172.21.15.165:6801/74543","cluster_addr":"172.21.15.165:6803/74543","heartbeat_back_addr":"172.21.15.165:6807/74543","heartbeat_front_addr":"172.21.15.165:6805/74543","state":["exists","up"]},{"osd":8,"uuid":"6f5200a3-8980-4241-a6fc-ef604fa612b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":43515},{"type":"v1","addr":"172.21.15.44:6810","nonce":43515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":43515},{"type":"v1","addr":"172.21.15.44:6814","nonce":43515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":43515},{"type":"v1","addr":"172.21.15.44:6822","nonce":43515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":43515},{"type":"v1","addr":"172.21.15.44:6818","nonce":43515}]},"public_addr":"172.21.15.44:6810/43515","cluster_addr":"172.21.15.44:6814/43515","heartbeat_back_addr":"172.21.15.44:6822/43515","heartbeat_front_addr":"172.21.15.44:6818/43515","state":["exists","up"]},{"osd":9,"uuid":"29cf9aaa-3a99-4605-92f4-64f1f6a1728d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":74379},{"type":"v1","addr":"172.21.15.121:6825","nonce":74379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":74379},{"type":"v1","addr":"172.21.15.121:6827","nonce":74379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":74379},{"type":"v1","addr":"172.21.15.121:6831","nonce":74379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":74379},{"type":"v1","addr":"172.21.15.121:6829","nonce":74379}]},"public_addr":"172.21.15.121:6825/74379","cluster_addr":"172.21.15.121:6827/74379","heartbeat_back_addr":"172.21.15.121:6831/74379","heartbeat_front_addr":"172.21.15.121:6829/74379","state":["exists","up"]},{"osd":10,"uuid":"705906c4-1514-4754-ac40-45226a837180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":43109},{"type":"v1","addr":"172.21.15.123:6810","nonce":43109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":43109},{"type":"v1","addr":"172.21.15.123:6813","nonce":43109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":43109},{"type":"v1","addr":"172.21.15.123:6820","nonce":43109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6815","nonce":43109},{"type":"v1","addr":"172.21.15.123:6817","nonce":43109}]},"public_addr":"172.21.15.123:6810/43109","cluster_addr":"172.21.15.123:6813/43109","heartbeat_back_addr":"172.21.15.123:6820/43109","heartbeat_front_addr":"172.21.15.123:6817/43109","state":["exists","up"]},{"osd":11,"uuid":"b45f2645-72d1-4d72-89e9-2212f5aa074c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":74544},{"type":"v1","addr":"172.21.15.165:6809","nonce":74544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":74544},{"type":"v1","addr":"172.21.15.165:6811","nonce":74544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":74544},{"type":"v1","addr":"172.21.15.165:6815","nonce":74544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":74544},{"type":"v1","addr":"172.21.15.165:6813","nonce":74544}]},"public_addr":"172.21.15.165:6809/74544","cluster_addr":"172.21.15.165:6811/74544","heartbeat_back_addr":"172.21.15.165:6815/74544","heartbeat_front_addr":"172.21.15.165:6813/74544","state":["exists","up"]},{"osd":12,"uuid":"3ea410b9-a368-467b-a179-2e3aba562d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":43516},{"type":"v1","addr":"172.21.15.44:6801","nonce":43516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":43516},{"type":"v1","addr":"172.21.15.44:6803","nonce":43516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":43516},{"type":"v1","addr":"172.21.15.44:6807","nonce":43516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":43516},{"type":"v1","addr":"172.21.15.44:6805","nonce":43516}]},"public_addr":"172.21.15.44:6801/43516","cluster_addr":"172.21.15.44:6803/43516","heartbeat_back_addr":"172.21.15.44:6807/43516","heartbeat_front_addr":"172.21.15.44:6805/43516","state":["exists","up"]},{"osd":13,"uuid":"15a5154b-e7cb-4b57-8468-dcb048d93f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":74355},{"type":"v1","addr":"172.21.15.121:6809","nonce":74355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":74355},{"type":"v1","addr":"172.21.15.121:6811","nonce":74355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":74355},{"type":"v1","addr":"172.21.15.121:6815","nonce":74355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":74355},{"type":"v1","addr":"172.21.15.121:6813","nonce":74355}]},"public_addr":"172.21.15.121:6809/74355","cluster_addr":"172.21.15.121:6811/74355","heartbeat_back_addr":"172.21.15.121:6815/74355","heartbeat_front_addr":"172.21.15.121:6813/74355","state":["exists","up"]},{"osd":14,"uuid":"23b53da2-f848-4e76-9632-4d9effd5b430","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":21,"last_clean_end":88,"up_from":102,"up_thru":30,"down_at":90,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":53453},{"type":"v1","addr":"172.21.15.123:6825","nonce":53453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":53453},{"type":"v1","addr":"172.21.15.123:6827","nonce":53453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":53453},{"type":"v1","addr":"172.21.15.123:6831","nonce":53453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":53453},{"type":"v1","addr":"172.21.15.123:6829","nonce":53453}]},"public_addr":"172.21.15.123:6825/53453","cluster_addr":"172.21.15.123:6827/53453","heartbeat_back_addr":"172.21.15.123:6831/53453","heartbeat_front_addr":"172.21.15.123:6829/53453","state":["exists","up"]},{"osd":15,"uuid":"546efc0f-82a3-47d2-aeec-2d849eb1faa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":74562},{"type":"v1","addr":"172.21.15.165:6817","nonce":74562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":74562},{"type":"v1","addr":"172.21.15.165:6819","nonce":74562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":74562},{"type":"v1","addr":"172.21.15.165:6823","nonce":74562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":74562},{"type":"v1","addr":"172.21.15.165:6821","nonce":74562}]},"public_addr":"172.21.15.165:6817/74562","cluster_addr":"172.21.15.165:6819/74562","heartbeat_back_addr":"172.21.15.165:6823/74562","heartbeat_front_addr":"172.21.15.165:6821/74562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:32.980703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:49:37.155276+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:41.034618+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:49:32.937643+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:25.217793+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:25.078524+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:35.034578+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:49:32.611236+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:28.281278+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:23.704764+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:49:30.690926+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:49:22.391036+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:28.932481+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:29.119303+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:52.833382+0000","dead_epoch":90},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T01:48:17.320410+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":40,"length":1},{"begin":54,"length":1},{"begin":56,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T01:49:38.668 INFO:tasks.ceph:Scrubbing osd.0 2024-04-20T01:49:38.668 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-20T01:49:38.844 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:49:38.844 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T01:49:38.845 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:49:38.858 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-20T01:49:39.228 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 0 to deep-scrub 2024-04-20T01:49:39.242 INFO:tasks.ceph:Scrubbing osd.4 2024-04-20T01:49:39.242 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-20T01:49:39.407 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:49:39.407 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T01:49:39.408 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:49:39.421 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-20T01:49:39.789 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 4 to deep-scrub 2024-04-20T01:49:39.802 INFO:tasks.ceph:Scrubbing osd.8 2024-04-20T01:49:39.802 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 config set osd_debug_deep_scrub_sleep 0 2024-04-20T01:49:39.967 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:49:39.967 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T01:49:39.967 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:49:39.980 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 8 2024-04-20T01:49:40.351 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 8 to deep-scrub 2024-04-20T01:49:40.364 INFO:tasks.ceph:Scrubbing osd.12 2024-04-20T01:49:40.364 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 config set osd_debug_deep_scrub_sleep 0 2024-04-20T01:49:40.529 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:49:40.530 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T01:49:40.530 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:49:40.543 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 12 2024-04-20T01:49:40.917 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 12 to deep-scrub 2024-04-20T01:49:40.930 INFO:tasks.ceph:Scrubbing osd.1 2024-04-20T01:49:40.930 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-20T01:49:41.096 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:49:41.096 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T01:49:41.096 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:49:41.109 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-20T01:49:41.476 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 1 to deep-scrub 2024-04-20T01:49:41.489 INFO:tasks.ceph:Scrubbing osd.5 2024-04-20T01:49:41.489 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-20T01:49:41.654 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:49:41.654 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T01:49:41.655 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:49:41.668 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-20T01:49:42.039 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 5 to deep-scrub 2024-04-20T01:49:42.052 INFO:tasks.ceph:Scrubbing osd.9 2024-04-20T01:49:42.052 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 config set osd_debug_deep_scrub_sleep 0 2024-04-20T01:49:42.217 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:49:42.217 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T01:49:42.217 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:49:42.231 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 9 2024-04-20T01:49:42.597 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 9 to deep-scrub 2024-04-20T01:49:42.610 INFO:tasks.ceph:Scrubbing osd.13 2024-04-20T01:49:42.611 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 config set osd_debug_deep_scrub_sleep 0 2024-04-20T01:49:42.777 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:49:42.777 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T01:49:42.777 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:49:42.791 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 13 2024-04-20T01:49:43.156 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 13 to deep-scrub 2024-04-20T01:49:43.168 INFO:tasks.ceph:Scrubbing osd.2 2024-04-20T01:49:43.169 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-20T01:49:43.336 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:49:43.336 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T01:49:43.336 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:49:43.349 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-20T01:49:43.718 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 2 to deep-scrub 2024-04-20T01:49:43.730 INFO:tasks.ceph:Scrubbing osd.6 2024-04-20T01:49:43.730 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-20T01:49:43.896 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:49:43.896 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T01:49:43.896 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:49:43.909 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-20T01:49:44.281 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 6 to deep-scrub 2024-04-20T01:49:44.294 INFO:tasks.ceph:Scrubbing osd.10 2024-04-20T01:49:44.294 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 config set osd_debug_deep_scrub_sleep 0 2024-04-20T01:49:44.460 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:49:44.460 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T01:49:44.460 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:49:44.474 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 10 2024-04-20T01:49:44.845 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 10 to deep-scrub 2024-04-20T01:49:44.858 INFO:tasks.ceph:Scrubbing osd.14 2024-04-20T01:49:44.859 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 config set osd_debug_deep_scrub_sleep 0 2024-04-20T01:49:45.024 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:49:45.024 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T01:49:45.024 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:49:45.037 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 14 2024-04-20T01:49:45.406 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 14 to deep-scrub 2024-04-20T01:49:45.419 INFO:tasks.ceph:Scrubbing osd.3 2024-04-20T01:49:45.419 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-20T01:49:45.583 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:49:45.584 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T01:49:45.584 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:49:45.597 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-20T01:49:45.964 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 3 to deep-scrub 2024-04-20T01:49:45.977 INFO:tasks.ceph:Scrubbing osd.7 2024-04-20T01:49:45.977 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-20T01:49:46.143 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:49:46.143 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T01:49:46.143 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:49:46.157 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-20T01:49:46.530 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 7 to deep-scrub 2024-04-20T01:49:46.543 INFO:tasks.ceph:Scrubbing osd.11 2024-04-20T01:49:46.543 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 config set osd_debug_deep_scrub_sleep 0 2024-04-20T01:49:46.708 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:49:46.709 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T01:49:46.709 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:49:46.722 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 11 2024-04-20T01:49:47.092 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 11 to deep-scrub 2024-04-20T01:49:47.105 INFO:tasks.ceph:Scrubbing osd.15 2024-04-20T01:49:47.105 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 config set osd_debug_deep_scrub_sleep 0 2024-04-20T01:49:47.272 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-20T01:49:47.272 INFO:teuthology.orchestra.run.smithi044.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T01:49:47.272 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-20T01:49:47.286 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 15 2024-04-20T01:49:47.657 INFO:teuthology.orchestra.run.smithi044.stderr:instructed osd(s) 15 to deep-scrub 2024-04-20T01:49:47.670 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T01:49:48.036 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:49:48.037 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T01:49:48.050 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-04-20T01:49:47.284222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":506156,"kb_used_data":34724,"kb_used_omap":38,"kb_used_meta":471193,"kb_avail":1509443284,"statfs":{"total":1546188226560,"available":1545669922816,"internally_reserved":0,"allocated":35557376,"data_stored":21132360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39290,"internal_metadata":482502278},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.116049"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":146,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T01:49:44.220443+0000","last_change":"2024-04-20T01:49:44.220443+0000","last_active":"2024-04-20T01:49:44.220443+0000","last_peered":"2024-04-20T01:49:44.220443+0000","last_clean":"2024-04-20T01:49:44.220443+0000","last_became_active":"2024-04-20T01:47:20.071615+0000","last_became_peered":"2024-04-20T01:47:20.071615+0000","last_unstale":"2024-04-20T01:49:44.220443+0000","last_undegraded":"2024-04-20T01:49:44.220443+0000","last_fullsized":"2024-04-20T01:49:44.220443+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:49:44.220341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:49:44.220341+0000","last_clean_scrub_stamp":"2024-04-20T01:49:44.220341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:45.850655+0000","scrub_duration":0.0017132390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":147,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-20T01:49:44.739731+0000","last_change":"2024-04-20T01:49:44.739858+0000","last_active":"2024-04-20T01:49:44.739731+0000","last_peered":"2024-04-20T01:49:44.739731+0000","last_clean":"2024-04-20T01:49:44.739731+0000","last_became_active":"2024-04-20T01:47:20.075314+0000","last_became_peered":"2024-04-20T01:47:20.075314+0000","last_unstale":"2024-04-20T01:49:44.739731+0000","last_undegraded":"2024-04-20T01:49:44.739731+0000","last_fullsized":"2024-04-20T01:49:44.739731+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:49:44.739698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:49:44.739698+0000","last_clean_scrub_stamp":"2024-04-20T01:49:44.739698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:59.443474+0000","scrub_duration":0.0018436769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,12],"acting":[2,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":144,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-20T01:49:45.349840+0000","last_change":"2024-04-20T01:49:45.349840+0000","last_active":"2024-04-20T01:49:45.349840+0000","last_peered":"2024-04-20T01:49:45.349840+0000","last_clean":"2024-04-20T01:49:45.349840+0000","last_became_active":"2024-04-20T01:47:20.038972+0000","last_became_peered":"2024-04-20T01:47:20.038972+0000","last_unstale":"2024-04-20T01:49:45.349840+0000","last_undegraded":"2024-04-20T01:49:45.349840+0000","last_fullsized":"2024-04-20T01:49:45.349840+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:49:45.349688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:49:45.349688+0000","last_clean_scrub_stamp":"2024-04-20T01:49:45.349688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:51:14.791741+0000","scrub_duration":0.0016176809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":146,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T01:49:42.052291+0000","last_change":"2024-04-20T01:49:28.041091+0000","last_active":"2024-04-20T01:49:42.052291+0000","last_peered":"2024-04-20T01:49:42.052291+0000","last_clean":"2024-04-20T01:49:42.052291+0000","last_became_active":"2024-04-20T01:47:20.038764+0000","last_became_peered":"2024-04-20T01:47:20.038764+0000","last_unstale":"2024-04-20T01:49:42.052291+0000","last_undegraded":"2024-04-20T01:49:42.052291+0000","last_fullsized":"2024-04-20T01:49:42.052291+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:49:28.040983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:49:28.040983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for deep scrub","scrub_duration":0.0021762000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"141'159","reported_seq":344,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-20T01:49:44.338113+0000","last_change":"2024-04-20T01:49:44.338113+0000","last_active":"2024-04-20T01:49:44.338113+0000","last_peered":"2024-04-20T01:49:44.338113+0000","last_clean":"2024-04-20T01:49:44.338113+0000","last_became_active":"2024-04-20T01:47:17.302105+0000","last_became_peered":"2024-04-20T01:47:17.302105+0000","last_unstale":"2024-04-20T01:49:44.338113+0000","last_undegraded":"2024-04-20T01:49:44.338113+0000","last_fullsized":"2024-04-20T01:49:44.338113+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"141'159","last_scrub_stamp":"2024-04-20T01:49:44.337957+0000","last_deep_scrub":"141'159","last_deep_scrub_stamp":"2024-04-20T01:49:44.337957+0000","last_clean_scrub_stamp":"2024-04-20T01:49:44.337957+0000","objects_scrubbed":2,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:51:00.179024+0000","scrub_duration":0.0086315999999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,15],"acting":[13,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":99,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-20T01:49:45.753540+0000","last_change":"2024-04-20T01:49:45.753657+0000","last_active":"2024-04-20T01:49:45.753540+0000","last_peered":"2024-04-20T01:49:45.753540+0000","last_clean":"2024-04-20T01:49:45.753540+0000","last_became_active":"2024-04-20T01:49:17.015873+0000","last_became_peered":"2024-04-20T01:49:17.015873+0000","last_unstale":"2024-04-20T01:49:45.753540+0000","last_undegraded":"2024-04-20T01:49:45.753540+0000","last_fullsized":"2024-04-20T01:49:45.753540+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:49:45.753517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:49:45.753517+0000","last_clean_scrub_stamp":"2024-04-20T01:49:45.753517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:51:12.916083+0000","scrub_duration":0.0024870169999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":135,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T01:49:40.449469+0000","last_change":"2024-04-20T01:48:48.595681+0000","last_active":"2024-04-20T01:49:40.449469+0000","last_peered":"2024-04-20T01:49:40.449469+0000","last_clean":"2024-04-20T01:49:40.449469+0000","last_became_active":"2024-04-20T01:47:20.038107+0000","last_became_peered":"2024-04-20T01:47:20.038107+0000","last_unstale":"2024-04-20T01:49:40.449469+0000","last_undegraded":"2024-04-20T01:49:40.449469+0000","last_fullsized":"2024-04-20T01:49:40.449469+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:48:48.595563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:47:19.015601+0000","last_clean_scrub_stamp":"2024-04-20T01:48:48.595563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:00.568790+0000","scrub_duration":0.002598885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":145,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-04-20T01:49:40.403305+0000","last_change":"2024-04-20T01:49:40.403305+0000","last_active":"2024-04-20T01:49:40.403305+0000","last_peered":"2024-04-20T01:49:40.403305+0000","last_clean":"2024-04-20T01:49:40.403305+0000","last_became_active":"2024-04-20T01:47:20.110260+0000","last_became_peered":"2024-04-20T01:47:20.110260+0000","last_unstale":"2024-04-20T01:49:40.403305+0000","last_undegraded":"2024-04-20T01:49:40.403305+0000","last_fullsized":"2024-04-20T01:49:40.403305+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:49:40.403196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:49:40.403196+0000","last_clean_scrub_stamp":"2024-04-20T01:49:40.403196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:58.620548+0000","scrub_duration":0.0036137309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":145,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-20T01:49:45.364440+0000","last_change":"2024-04-20T01:49:45.364543+0000","last_active":"2024-04-20T01:49:45.364440+0000","last_peered":"2024-04-20T01:49:45.364440+0000","last_clean":"2024-04-20T01:49:45.364440+0000","last_became_active":"2024-04-20T01:47:20.036328+0000","last_became_peered":"2024-04-20T01:47:20.036328+0000","last_unstale":"2024-04-20T01:49:45.364440+0000","last_undegraded":"2024-04-20T01:49:45.364440+0000","last_fullsized":"2024-04-20T01:49:45.364440+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:49:45.364416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:49:45.364416+0000","last_clean_scrub_stamp":"2024-04-20T01:49:45.364416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:53.686381+0000","scrub_duration":0.0019838540000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,13],"acting":[6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280546,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33740,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338100,"statfs":{"total":96636764160,"available":96602214400,"internally_reserved":0,"allocated":2686976,"data_stored":1784292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[2,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.45100000000000001}]},{"osd":2,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.59799999999999998,"15min":0.61099999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.74299999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.64000000000000001,"15min":0.65100000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84699999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.67400000000000004}]},{"osd":3,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.47699999999999998}]},{"osd":4,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":1.0760000000000001,"15min":1.155},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.747,"5min":9.0129999999999999,"15min":9.0129999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.65900000000000003,"15min":0.67500000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.747,"5min":1.022,"15min":1.022},"last":0.41399999999999998}]},{"osd":5,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.59199999999999997,"15min":0.60899999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73999999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.56899999999999995,"15min":0.58299999999999996},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72299999999999998,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.50600000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.66100000000000003,"15min":0.67600000000000005},"min":{"1min":0.434,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77700000000000002,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.64700000000000002,"15min":0.66500000000000004},"min":{"1min":0.40100000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.66400000000000003,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.68400000000000005}]},{"osd":7,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.755,"15min":0.76800000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95999999999999996,"5min":1.143,"15min":1.143},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.78100000000000003,"15min":0.80100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97999999999999998,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.71399999999999997}]},{"osd":10,"last update":"Sat Apr 20 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.77300000000000002,"15min":0.79200000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86199999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.878,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.61399999999999999}]},{"osd":11,"last update":"Sat Apr 20 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.73699999999999999,"15min":0.752},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91900000000000004,"5min":1.109,"15min":1.109},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.84799999999999998,"15min":0.86799999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.92700000000000005,"5min":1.583,"15min":1.583},"last":0.79100000000000004}]},{"osd":12,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.77800000000000002,"15min":0.80000000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.83699999999999997,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.746,"5min":1.196,"15min":1.2709999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93999999999999995,"5min":9.1389999999999993,"15min":9.1389999999999993},"last":0.53300000000000003}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.77000000000000002}]},{"osd":15,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.96099999999999997,"15min":0.98999999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.99399999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.88300000000000001,"15min":0.90600000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.94699999999999995,"5min":1.399,"15min":1.399},"last":0.747}]}]},{"osd":15,"up_from":21,"seq":90194313250,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29648,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342192,"statfs":{"total":96636764160,"available":96606404608,"internally_reserved":0,"allocated":2691072,"data_stored":1784895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,3,4,5,6,7,8,9,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.34100000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.67300000000000004,"15min":0.70299999999999996},"min":{"1min":0.36499999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72399999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":0.40400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":2,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.68000000000000005}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42399999999999999,"15min":0.42599999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.76100000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.36299999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.38800000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.48499999999999999}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84299999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.63100000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.627,"15min":0.63400000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86499999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63,"15min":0.63300000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.94399999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.60699999999999998}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.93000000000000005,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94899999999999995,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.67400000000000004}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45900000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96099999999999997,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.86199999999999999,"15min":0.90000000000000002},"min":{"1min":0.43099999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97599999999999998,"5min":4.5979999999999999,"15min":4.5979999999999999},"last":0.51700000000000002}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68400000000000005,"15min":0.68700000000000006},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93899999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.748,"15min":0.746},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.016,"5min":1.272,"15min":1.272},"last":0.74399999999999999}]},{"osd":13,"last update":"Sat Apr 20 01:49:41 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.76400000000000001}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.64800000000000002}]}]},{"osd":14,"up_from":102,"seq":438086664204,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31352,"kb_used_data":2088,"kb_used_omap":15,"kb_used_meta":29232,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":2138112,"data_stored":1245776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15377,"internal_metadata":29934575},"hb_peers":[0,1,2,3,4,5,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":12,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":13,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]}]},{"osd":12,"up_from":21,"seq":90194313250,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29128,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342712,"statfs":{"total":96636764160,"available":96606937088,"internally_reserved":0,"allocated":2158592,"data_stored":1255967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":18.143000000000001,"5min":18.143000000000001,"15min":18.143000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":18.170000000000002,"5min":18.170000000000002,"15min":18.170000000000002},"last":0.53000000000000003}]},{"osd":1,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.58499999999999996,"15min":0.55100000000000005},"min":{"1min":0.65800000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66800000000000004,"15min":0.66200000000000003},"min":{"1min":0.504,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93799999999999994,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.497}]},{"osd":2,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.48499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.57599999999999996}]},{"osd":3,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88700000000000001,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.38100000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79800000000000004,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.40799999999999997}]},{"osd":4,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.39500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.59999999999999998,"15min":0.61199999999999999},"min":{"1min":0.371,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82399999999999995,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.39400000000000002}]},{"osd":5,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.65500000000000003,"15min":0.63700000000000001},"min":{"1min":0.55200000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0940000000000001,"5min":1.115,"15min":1.115},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.47499999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.0800000000000001,"5min":1.726,"15min":1.726},"last":0.54500000000000004}]},{"osd":6,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.66200000000000003,"15min":0.66600000000000004},"min":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.86699999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.74299999999999999,"15min":0.72899999999999998},"min":{"1min":0.63700000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0349999999999999,"5min":1.069,"15min":1.069},"last":0.69599999999999995}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.76300000000000001,"15min":0.76500000000000001},"min":{"1min":0.54400000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0529999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.623},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.95399999999999996,"15min":0.98999999999999999},"min":{"1min":0.53800000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.012,"5min":6.6779999999999999,"15min":6.6779999999999999},"last":0.68100000000000005}]},{"osd":13,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.79800000000000004},"min":{"1min":0.59699999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0189999999999999,"5min":1.827,"15min":1.827},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.80900000000000005,"15min":0.79500000000000004},"min":{"1min":0.753,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1739999999999999,"5min":1.258,"15min":1.258},"last":0.65200000000000002}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":1.135},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.99299999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.79000000000000004},"min":{"1min":0.58199999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0109999999999999,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.82999999999999996,"15min":0.81699999999999995},"min":{"1min":0.753,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.67200000000000004}]}]},{"osd":11,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33224,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338616,"statfs":{"total":96636764160,"available":96602742784,"internally_reserved":0,"allocated":2158592,"data_stored":1255967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,2,3,4,8,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.58599999999999997,"15min":0.59799999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.83899999999999997,"5min":1.669,"15min":1.669},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.67300000000000004,"15min":0.68300000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0900000000000001,"5min":1.75,"15min":1.75},"last":0.436}]},{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.58099999999999996}]},{"osd":2,"last update":"Sat Apr 20 01:49:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.75},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.78200000000000003}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.40300000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.28799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.621}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.88,"15min":0.93500000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1220000000000001,"5min":8.5969999999999995,"15min":8.5969999999999995},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.67000000000000004,"15min":0.67900000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.147,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.65200000000000002}]},{"osd":8,"last update":"Sat Apr 20 01:49:37 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.60199999999999998,"15min":0.58099999999999996},"min":{"1min":0.39200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":3.3559999999999999,"5min":3.3559999999999999,"15min":3.3559999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.874,"5min":0.68600000000000005,"15min":0.65500000000000003},"min":{"1min":0.442,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.4909999999999997,"5min":4.4909999999999997,"15min":4.4909999999999997},"last":0.72299999999999998}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.57499999999999996,"15min":0.56200000000000006},"min":{"1min":0.38600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.73199999999999998}]},{"osd":10,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47199999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.86699999999999999}]},{"osd":12,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85699999999999998,"5min":1.875,"15min":1.875},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":1.262,"15min":1.349},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1879999999999999,"5min":12.385,"15min":12.385},"last":0.85299999999999998}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.79700000000000004}]},{"osd":15,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.75,"15min":0.76100000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.96999999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.88},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.76100000000000001}]}]},{"osd":10,"up_from":21,"seq":90194313251,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33224,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338616,"statfs":{"total":96636764160,"available":96602742784,"internally_reserved":0,"allocated":2158592,"data_stored":1255967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,4,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.495,"15min":0.48899999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.505,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.74199999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.64200000000000002}]},{"osd":1,"last update":"Sat Apr 20 01:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.61399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.437},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.48799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":7,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63300000000000001,"15min":0.626},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.79700000000000004}]},{"osd":8,"last update":"Sat Apr 20 01:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.032,"5min":1.915,"15min":1.915},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68000000000000005,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1299999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.66100000000000003}]},{"osd":9,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68999999999999995,"15min":0.68500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.70199999999999996}]},{"osd":11,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.42499999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82099999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.90200000000000002,"15min":0.92600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.292,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":0.86799999999999999}]},{"osd":12,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":1.208,"15min":1.29},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1120000000000001,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.997,"15min":1.054},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91000000000000003,"5min":9.7759999999999998,"15min":9.7759999999999998},"last":0.53300000000000003}]},{"osd":13,"last update":"Sat Apr 20 01:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89600000000000002,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.71699999999999997,"15min":0.72299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95799999999999996,"5min":1.05,"15min":1.05},"last":0.76600000000000001}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.64200000000000002}]},{"osd":15,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.80100000000000005,"15min":0.79700000000000004},"min":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.53400000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.93700000000000006,"5min":1.157,"15min":1.157},"last":0.72399999999999998}]}]},{"osd":9,"up_from":21,"seq":90194313251,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":29128,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342712,"statfs":{"total":96636764160,"available":96606937088,"internally_reserved":0,"allocated":2158592,"data_stored":1255967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,2,4,6,7,8,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47099999999999997,"15min":0.46300000000000002},"min":{"1min":0.32800000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.52500000000000002,"15min":0.51300000000000001},"min":{"1min":0.40300000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83599999999999997,"5min":0.998,"15min":0.998},"last":0.33100000000000002}]},{"osd":2,"last update":"Sat Apr 20 01:49:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.48699999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46800000000000003}]},{"osd":6,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.872,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.76900000000000002}]},{"osd":7,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57599999999999996}]},{"osd":8,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.504},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.39600000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.59199999999999997}]},{"osd":10,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.93799999999999994,"5min":1.03,"15min":1.03},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58699999999999997,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76300000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.73499999999999999}]},{"osd":11,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88400000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.752}]},{"osd":12,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.74099999999999999,"15min":0.73899999999999999},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97899999999999998,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.89900000000000002,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.92700000000000005}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.56000000000000005}]},{"osd":15,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.745,"15min":0.753},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69699999999999995,"15min":0.70499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88800000000000001,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.61699999999999999}]}]},{"osd":8,"up_from":21,"seq":90194313250,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33224,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338616,"statfs":{"total":96636764160,"available":96602742784,"internally_reserved":0,"allocated":2158592,"data_stored":1255967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.34200000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89700000000000002,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.628,"15min":0.64100000000000001},"min":{"1min":0.39500000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.81699999999999995,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.53000000000000003}]},{"osd":3,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.45400000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85799999999999998,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.73599999999999999,"15min":0.753},"min":{"1min":0.44500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91600000000000004,"5min":1.821,"15min":1.821},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":6,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70199999999999996,"15min":0.70999999999999996},"min":{"1min":0.46500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89900000000000002,"5min":1.784,"15min":1.784},"last":0.443},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.44,"5min":0.437,"15min":0.437},"max":{"1min":0.79300000000000004,"5min":1.885,"15min":1.885},"last":0.432}]},{"osd":7,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.72599999999999998,"15min":0.73499999999999999},"min":{"1min":0.50800000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93300000000000005,"5min":1.411,"15min":1.411},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.81999999999999995,"15min":0.84399999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95199999999999996,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.57399999999999995}]},{"osd":9,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.72199999999999998,"15min":0.73599999999999999},"min":{"1min":0.436,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91800000000000004,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.63},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.139,"5min":1.375,"15min":1.375},"last":0.46300000000000002}]},{"osd":10,"last update":"Sat Apr 20 01:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.50800000000000001}]},{"osd":11,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.83899999999999997,"15min":0.84299999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.141,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.92400000000000004,"15min":0.93999999999999995},"min":{"1min":0.66100000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.071,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.56100000000000005}]},{"osd":12,"last update":"Sat Apr 20 01:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.38800000000000001}]},{"osd":13,"last update":"Sat Apr 20 01:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.83499999999999996,"15min":0.82499999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.60199999999999998}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":15,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.88700000000000001,"15min":0.89100000000000001},"min":{"1min":0.621,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.095,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.95399999999999996,"15min":0.98399999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0229999999999999,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.64500000000000002}]}]},{"osd":7,"up_from":21,"seq":90194313251,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29128,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342712,"statfs":{"total":96636764160,"available":96606937088,"internally_reserved":0,"allocated":2158592,"data_stored":1255967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,3,4,6,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.38800000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.45700000000000002}]},{"osd":3,"last update":"Sat Apr 20 01:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.435,"15min":0.438},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.58399999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.438,"15min":0.438},"min":{"1min":0.307,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.629}]},{"osd":4,"last update":"Sat Apr 20 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.379,"5min":0.372,"15min":0.372},"max":{"1min":0.92600000000000005,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.47199999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94399999999999995,"5min":4.1920000000000002,"15min":4.1920000000000002},"last":0.65600000000000003}]},{"osd":6,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60299999999999998}]},{"osd":8,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.78900000000000003,"15min":0.81499999999999995},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82599999999999996,"5min":5.7679999999999998,"15min":5.7679999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":1.0189999999999999,"15min":1.077},"min":{"1min":0.46300000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96599999999999997,"5min":9.952,"15min":9.952},"last":0.53200000000000003}]},{"osd":9,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.438,"5min":0.433,"15min":0.433},"max":{"1min":0.751,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.498,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88800000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68700000000000006}]},{"osd":10,"last update":"Sat Apr 20 01:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.72199999999999998}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82899999999999996,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.88800000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73399999999999999}]},{"osd":12,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.84399999999999997,"15min":0.86099999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.99199999999999999,"5min":4.2720000000000002,"15min":4.2720000000000002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68700000000000006,"15min":0.68000000000000005},"min":{"1min":0.50900000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90300000000000002,"5min":0.995,"15min":0.995},"last":0.74299999999999999}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.76700000000000002}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":6,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33224,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338616,"statfs":{"total":96636764160,"available":96602742784,"internally_reserved":0,"allocated":2158592,"data_stored":1255967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[3,4,5,7,8,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.5}]},{"osd":2,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.53700000000000003}]},{"osd":3,"last update":"Sat Apr 20 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.42199999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76900000000000002,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.57899999999999996}]},{"osd":4,"last update":"Sat Apr 20 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.72099999999999997,"15min":0.68400000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":5.141,"5min":5.141,"15min":5.141},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.67400000000000004,"15min":0.622},"min":{"1min":0.38500000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":8.0090000000000003,"5min":8.0090000000000003,"15min":8.0090000000000003},"last":0.69999999999999996}]},{"osd":5,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.38300000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.91700000000000004,"5min":1.083,"15min":1.083},"last":0.65900000000000003}]},{"osd":7,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59999999999999998,"15min":0.60399999999999998},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77600000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66100000000000003,"15min":0.66400000000000003},"min":{"1min":0.36499999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.89400000000000002,"5min":1.056,"15min":1.056},"last":0.72999999999999998}]},{"osd":8,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.755,"15min":0.77000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88300000000000001,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.80100000000000005,"15min":0.81200000000000006},"min":{"1min":0.53800000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92000000000000004,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.0549999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":10,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67900000000000005,"15min":0.68700000000000006},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92500000000000004,"5min":1.129,"15min":1.129},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.67700000000000005,"15min":0.69099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88,"5min":1.272,"15min":1.272},"last":0.81299999999999994}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.92100000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.78100000000000003,"15min":0.79000000000000004},"min":{"1min":0.52200000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96599999999999997,"5min":1.167,"15min":1.167},"last":0.96399999999999997}]},{"osd":12,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.83499999999999996,"15min":0.84799999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.085,"5min":1.444,"15min":1.444},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84899999999999998,"15min":0.84699999999999998},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.88300000000000001}]},{"osd":13,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.81699999999999995,"15min":0.82299999999999995},"min":{"1min":0.56899999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0329999999999999,"5min":1.482,"15min":1.482},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.753,"15min":0.76000000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94999999999999996,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.86499999999999999}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.65700000000000003}]},{"osd":15,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.85999999999999999,"15min":0.87},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0049999999999999,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":1.02},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97599999999999998,"5min":1.327,"15min":1.327},"last":0.98199999999999998}]}]},{"osd":1,"up_from":21,"seq":90194313250,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33212,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2142208,"data_stored":1246379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,4,5,6,7,8,9,10,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.75600000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67400000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.437}]},{"osd":2,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70199999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79600000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.64100000000000001}]},{"osd":3,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.76000000000000001}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67800000000000005,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72099999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.46200000000000002}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.497,"15min":0.5},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.499,"15min":0.50700000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.495}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82599999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.59999999999999998}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66800000000000004,"15min":0.66300000000000003},"min":{"1min":0.48099999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65900000000000003,"15min":0.64900000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.65600000000000003}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.88300000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.72199999999999998,"15min":0.73999999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84899999999999998,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.626}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.83499999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67800000000000005,"15min":0.68400000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92900000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.58799999999999997}]},{"osd":10,"last update":"Sat Apr 20 01:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.69899999999999995}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.78700000000000003}]}]},{"osd":0,"up_from":21,"seq":90194313251,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33220,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338620,"statfs":{"total":96636764160,"available":96602746880,"internally_reserved":0,"allocated":2154496,"data_stored":1255381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.625},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64500000000000002,"15min":0.63400000000000001},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.81000000000000005}]},{"osd":2,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.90200000000000002,"15min":0.95599999999999996},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.89100000000000001,"5min":8.0619999999999994,"15min":8.0619999999999994},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.1399999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.52000000000000002}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.80500000000000005,"5min":3.363,"15min":3.363},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.90000000000000002,"15min":0.93400000000000005},"min":{"1min":0.50800000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.998,"5min":7.9710000000000001,"15min":7.9710000000000001},"last":0.442}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66000000000000003,"15min":0.66500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97599999999999998,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90200000000000002,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":0.77400000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.94199999999999995,"15min":0.98699999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1779999999999999,"5min":8.0239999999999991,"15min":8.0239999999999991},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1739999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.54800000000000004}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.72999999999999998,"15min":0.748},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86299999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.98399999999999999,"15min":1.0229999999999999},"min":{"1min":0.42899999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1200000000000001,"5min":7.8630000000000004,"15min":7.8630000000000004},"last":0.746}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.67800000000000005,"15min":0.69499999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.92100000000000004,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.95799999999999996,"15min":1.02},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.93400000000000005,"5min":8.1189999999999998,"15min":8.1189999999999998},"last":0.505}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.81200000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.01,"5min":3.5699999999999998,"15min":3.5699999999999998},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.55300000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.044,"5min":1.927,"15min":1.927},"last":0.72999999999999998}]},{"osd":13,"last update":"Sat Apr 20 01:49:37 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.3300000000000001,"5min":0.91800000000000004,"15min":0.84899999999999998},"min":{"1min":0.64200000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":9.4339999999999993,"5min":9.4339999999999993,"15min":9.4339999999999993},"last":0.85799999999999998}]},{"osd":15,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.097,"5min":1.833,"15min":1.833},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.86899999999999999,"15min":0.88500000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96699999999999997,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":0.82699999999999996}]}]},{"osd":2,"up_from":21,"seq":90194313249,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29128,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342712,"statfs":{"total":96636764160,"available":96606937088,"internally_reserved":0,"allocated":2158592,"data_stored":1255967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.46600000000000003,"15min":0.45600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.47099999999999997,"15min":0.46300000000000002},"min":{"1min":0.311,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.47499999999999998}]},{"osd":1,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54600000000000004,"15min":0.53900000000000003},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.58499999999999996}]},{"osd":3,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82199999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.79300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":7,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82799999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.627},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":0.60299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72299999999999998,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.66000000000000003}]},{"osd":8,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65000000000000002,"15min":0.66100000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90100000000000002,"5min":2.512,"15min":2.512},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.68400000000000005,"15min":0.69599999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.77600000000000002,"5min":2.532,"15min":2.532},"last":0.55900000000000005}]},{"osd":9,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.435,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.752},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64800000000000002},"min":{"1min":0.43099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93000000000000005,"5min":1.107,"15min":1.107},"last":0.81699999999999995}]},{"osd":11,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0269999999999999,"5min":1.135,"15min":1.135},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68600000000000005,"15min":0.69399999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88900000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.83799999999999997}]},{"osd":12,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.75800000000000001,"15min":0.78500000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.872,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.80200000000000005,"15min":0.82899999999999996},"min":{"1min":0.42499999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93899999999999995,"5min":3.492,"15min":3.492},"last":0.85799999999999998}]},{"osd":13,"last update":"Sat Apr 20 01:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.67800000000000005,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76100000000000001}]},{"osd":15,"last update":"Sat Apr 20 01:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.66600000000000004,"15min":0.66000000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68000000000000005,"15min":0.67500000000000004},"min":{"1min":0.53600000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.80400000000000005}]}]},{"osd":3,"up_from":21,"seq":90194313252,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33224,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338616,"statfs":{"total":96636764160,"available":96602742784,"internally_reserved":0,"allocated":2158592,"data_stored":1255967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,4,5,6,7,8,9,10,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:49:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.48199999999999998,"15min":0.47699999999999998},"min":{"1min":0.312,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.48499999999999999,"15min":0.47899999999999998},"min":{"1min":0.33800000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.68100000000000005}]},{"osd":2,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.36199999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.90500000000000003,"5min":0.996,"15min":0.996},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.85599999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.42899999999999999}]},{"osd":4,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.56499999999999995,"15min":0.56899999999999995},"min":{"1min":0.34000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69399999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.94599999999999995,"15min":1.014},"min":{"1min":0.39000000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83899999999999997,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":0.34499999999999997}]},{"osd":5,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.56299999999999994,"15min":0.57199999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66500000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55600000000000005,"15min":0.56000000000000005},"min":{"1min":0.36399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78700000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.627}]},{"osd":6,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.38600000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89300000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.48799999999999999}]},{"osd":7,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66200000000000003,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.54800000000000004,"15min":0.56000000000000005},"min":{"1min":0.30099999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71299999999999997,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.47199999999999998}]},{"osd":8,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.41199999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85099999999999998,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.442},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.0780000000000001,"15min":1.1479999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81999999999999995,"5min":9.4700000000000006,"15min":9.4700000000000006},"last":0.52900000000000003}]},{"osd":9,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.622,"15min":0.625},"min":{"1min":0.45200000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84699999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.71199999999999997,"15min":0.72599999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74399999999999999,"5min":1.306,"15min":1.306},"last":0.60099999999999998}]},{"osd":10,"last update":"Sat Apr 20 01:49:37 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.51200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":12,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":10.959,"5min":10.959,"15min":10.959},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.49399999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:49:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56399999999999995}]}]},{"osd":4,"up_from":21,"seq":90194313251,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33224,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338616,"statfs":{"total":96636764160,"available":96602742784,"internally_reserved":0,"allocated":2158592,"data_stored":1255967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,5,6,7,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.81000000000000005,"15min":0.87},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.93400000000000005,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":1.1579999999999999,"15min":1.2729999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.008,"5min":16.27,"15min":16.27},"last":0.312}]},{"osd":1,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":1.012,"15min":1.081},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.92700000000000005,"5min":10.507999999999999,"15min":10.507999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":1.1559999999999999,"15min":1.23},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6319999999999999,"5min":11.019,"15min":11.019},"last":0.50700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":3,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.997,"15min":1.0469999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4750000000000001,"5min":4.9180000000000001,"15min":4.9180000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":1.0449999999999999,"15min":1.095},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.6000000000000001,"5min":10.619,"15min":10.619},"last":0.51600000000000001}]},{"osd":5,"last update":"Sat Apr 20 01:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":1.099,"15min":1.169},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.673,"5min":10.895,"15min":10.895},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":1.012,"15min":1.0740000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1910000000000001,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":0.45400000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":1.1890000000000001,"15min":1.2649999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1719999999999999,"5min":11.051,"15min":11.051},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":1.1279999999999999,"15min":1.2},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96199999999999997,"5min":10.589,"15min":10.589},"last":0.38500000000000001}]},{"osd":7,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":1.1950000000000001,"15min":1.2649999999999999},"min":{"1min":0.56000000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.571,"5min":10.957000000000001,"15min":10.957000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":1.1799999999999999,"15min":1.244},"min":{"1min":0.47299999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3080000000000001,"5min":10.656000000000001,"15min":10.656000000000001},"last":0.58599999999999997}]},{"osd":8,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.79,"5min":3.79,"15min":3.79},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.0269999999999999}]},{"osd":9,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.385,"15min":1.385},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":10.929,"5min":10.929,"15min":10.929},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":10.555999999999999,"5min":10.555999999999999,"15min":10.555999999999999},"last":1.077}]},{"osd":10,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":1.149,"15min":1.204},"min":{"1min":0.57799999999999996,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.331,"5min":5.0229999999999997,"15min":5.0229999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":1.2729999999999999,"15min":1.351},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2330000000000001,"5min":10.69,"15min":10.69},"last":0.42599999999999999}]},{"osd":11,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.3600000000000001,"15min":1.4410000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.448,"5min":10.987,"15min":10.987},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":1.1879999999999999,"15min":1.244},"min":{"1min":0.63700000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.2290000000000001,"5min":4.9900000000000002,"15min":4.9900000000000002},"last":0.67200000000000004}]},{"osd":12,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.3240000000000001,"15min":1.4059999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.181,"5min":10.720000000000001,"15min":10.720000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.85999999999999999,"5min":1.589,"15min":1.7110000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3149999999999999,"5min":16.617000000000001,"15min":16.617000000000001},"last":0.57699999999999996}]},{"osd":13,"last update":"Sat Apr 20 01:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.98499999999999999,"15min":1.0089999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.732,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.0469999999999999,"15min":1.075},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.546,"5min":4.5019999999999998,"15min":4.5019999999999998},"last":0.73999999999999999}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]},{"osd":5,"up_from":21,"seq":90194313250,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29128,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342712,"statfs":{"total":96636764160,"available":96606937088,"internally_reserved":0,"allocated":2158592,"data_stored":1255967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,8,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.85699999999999998,"15min":0.92100000000000004},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77600000000000002,"5min":10.423999999999999,"15min":10.423999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.92500000000000004,"15min":0.98399999999999999},"min":{"1min":0.374,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73499999999999999,"5min":10.465,"15min":10.465},"last":0.60199999999999998}]},{"osd":2,"last update":"Sat Apr 20 01:49:22 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.33600000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.58299999999999996}]},{"osd":3,"last update":"Sat Apr 20 01:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.46999999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.443,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.70899999999999996}]},{"osd":4,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.81999999999999995,"15min":0.86799999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71499999999999997,"5min":8.5269999999999992,"15min":8.5269999999999992},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60999999999999999,"15min":0.61699999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78100000000000003,"5min":2.79,"15min":2.79},"last":0.65500000000000003}]},{"osd":6,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84599999999999997,"5min":1.115,"15min":1.115},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.63100000000000001},"min":{"1min":0.38900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94599999999999995,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.69899999999999995}]},{"osd":7,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78800000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.49399999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.90500000000000003}]},{"osd":8,"last update":"Sat Apr 20 01:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.76900000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":10,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.91300000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.68300000000000005}]},{"osd":11,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71899999999999997,"15min":0.71199999999999997},"min":{"1min":0.53900000000000003,"5min":0.435,"15min":0.435},"max":{"1min":1.242,"5min":1.956,"15min":1.956},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70299999999999996,"15min":0.69199999999999995},"min":{"1min":0.55000000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.84999999999999998}]},{"osd":12,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":1.052,"15min":1.1080000000000001},"min":{"1min":0.55800000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90600000000000003,"5min":10.616,"15min":10.616},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.755,"5min":1.0900000000000001,"15min":1.1459999999999999},"min":{"1min":0.56999999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.98899999999999999,"5min":10.629,"15min":10.629},"last":0.81599999999999995}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.84699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T01:49:48.056 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-20T01:49:28.040983+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=1, tm_min=49, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=1, tm_min=49, tm_sec=37, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T01:49:48.057 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-20T01:48:48.595563+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=1, tm_min=48, tm_sec=48, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=1, tm_min=49, tm_sec=37, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T01:49:48.057 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-20T01:50:08.057 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T01:50:08.435 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-20T01:50:08.436 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-20T01:50:08.449 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-04-20T01:50:07.174077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":507960,"kb_used_data":36516,"kb_used_omap":38,"kb_used_meta":471193,"kb_avail":1509441480,"statfs":{"total":1546188226560,"available":1545668075520,"internally_reserved":0,"allocated":37392384,"data_stored":22206216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":39290,"internal_metadata":482502278},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001684"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":152,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-20T01:50:00.540774+0000","last_change":"2024-04-20T01:49:44.220443+0000","last_active":"2024-04-20T01:50:00.540774+0000","last_peered":"2024-04-20T01:50:00.540774+0000","last_clean":"2024-04-20T01:50:00.540774+0000","last_became_active":"2024-04-20T01:47:20.071615+0000","last_became_peered":"2024-04-20T01:47:20.071615+0000","last_unstale":"2024-04-20T01:50:00.540774+0000","last_undegraded":"2024-04-20T01:50:00.540774+0000","last_fullsized":"2024-04-20T01:50:00.540774+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:49:44.220341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:49:44.220341+0000","last_clean_scrub_stamp":"2024-04-20T01:49:44.220341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:45.850655+0000","scrub_duration":0.0017132390000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":153,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-20T01:50:00.835226+0000","last_change":"2024-04-20T01:49:44.739858+0000","last_active":"2024-04-20T01:50:00.835226+0000","last_peered":"2024-04-20T01:50:00.835226+0000","last_clean":"2024-04-20T01:50:00.835226+0000","last_became_active":"2024-04-20T01:47:20.075314+0000","last_became_peered":"2024-04-20T01:47:20.075314+0000","last_unstale":"2024-04-20T01:50:00.835226+0000","last_undegraded":"2024-04-20T01:50:00.835226+0000","last_fullsized":"2024-04-20T01:50:00.835226+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:49:44.739698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:49:44.739698+0000","last_clean_scrub_stamp":"2024-04-20T01:49:44.739698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:59.443474+0000","scrub_duration":0.0018436769999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,12],"acting":[2,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":150,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-20T01:50:00.489277+0000","last_change":"2024-04-20T01:49:45.349840+0000","last_active":"2024-04-20T01:50:00.489277+0000","last_peered":"2024-04-20T01:50:00.489277+0000","last_clean":"2024-04-20T01:50:00.489277+0000","last_became_active":"2024-04-20T01:47:20.038972+0000","last_became_peered":"2024-04-20T01:47:20.038972+0000","last_unstale":"2024-04-20T01:50:00.489277+0000","last_undegraded":"2024-04-20T01:50:00.489277+0000","last_fullsized":"2024-04-20T01:50:00.489277+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:49:45.349688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:49:45.349688+0000","last_clean_scrub_stamp":"2024-04-20T01:49:45.349688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:51:14.791741+0000","scrub_duration":0.0016176809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":163,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-20T01:50:00.940301+0000","last_change":"2024-04-20T01:49:46.037768+0000","last_active":"2024-04-20T01:50:00.940301+0000","last_peered":"2024-04-20T01:50:00.940301+0000","last_clean":"2024-04-20T01:50:00.940301+0000","last_became_active":"2024-04-20T01:47:20.038764+0000","last_became_peered":"2024-04-20T01:47:20.038764+0000","last_unstale":"2024-04-20T01:50:00.940301+0000","last_undegraded":"2024-04-20T01:50:00.940301+0000","last_fullsized":"2024-04-20T01:50:00.940301+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:49:46.037648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:49:46.037648+0000","last_clean_scrub_stamp":"2024-04-20T01:49:46.037648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:51:03.045971+0000","scrub_duration":0.001933817,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"141'159","reported_seq":350,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-20T01:50:00.489306+0000","last_change":"2024-04-20T01:49:44.338113+0000","last_active":"2024-04-20T01:50:00.489306+0000","last_peered":"2024-04-20T01:50:00.489306+0000","last_clean":"2024-04-20T01:50:00.489306+0000","last_became_active":"2024-04-20T01:47:17.302105+0000","last_became_peered":"2024-04-20T01:47:17.302105+0000","last_unstale":"2024-04-20T01:50:00.489306+0000","last_undegraded":"2024-04-20T01:50:00.489306+0000","last_fullsized":"2024-04-20T01:50:00.489306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"141'159","last_scrub_stamp":"2024-04-20T01:49:44.337957+0000","last_deep_scrub":"141'159","last_deep_scrub_stamp":"2024-04-20T01:49:44.337957+0000","last_clean_scrub_stamp":"2024-04-20T01:49:44.337957+0000","objects_scrubbed":2,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:51:00.179024+0000","scrub_duration":0.0086315999999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,15],"acting":[13,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":105,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-20T01:50:00.835214+0000","last_change":"2024-04-20T01:49:45.753657+0000","last_active":"2024-04-20T01:50:00.835214+0000","last_peered":"2024-04-20T01:50:00.835214+0000","last_clean":"2024-04-20T01:50:00.835214+0000","last_became_active":"2024-04-20T01:49:17.015873+0000","last_became_peered":"2024-04-20T01:49:17.015873+0000","last_unstale":"2024-04-20T01:50:00.835214+0000","last_undegraded":"2024-04-20T01:50:00.835214+0000","last_fullsized":"2024-04-20T01:50:00.835214+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:49:45.753517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:49:45.753517+0000","last_clean_scrub_stamp":"2024-04-20T01:49:45.753517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:51:12.916083+0000","scrub_duration":0.0024870169999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":153,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-20T01:50:00.253159+0000","last_change":"2024-04-20T01:49:52.131986+0000","last_active":"2024-04-20T01:50:00.253159+0000","last_peered":"2024-04-20T01:50:00.253159+0000","last_clean":"2024-04-20T01:50:00.253159+0000","last_became_active":"2024-04-20T01:47:20.038107+0000","last_became_peered":"2024-04-20T01:47:20.038107+0000","last_unstale":"2024-04-20T01:50:00.253159+0000","last_undegraded":"2024-04-20T01:50:00.253159+0000","last_fullsized":"2024-04-20T01:50:00.253159+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:49:52.131819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:49:52.131819+0000","last_clean_scrub_stamp":"2024-04-20T01:49:52.131819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:51:16.769550+0000","scrub_duration":0.0025876860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":152,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-20T01:49:59.328831+0000","last_change":"2024-04-20T01:49:40.403305+0000","last_active":"2024-04-20T01:49:59.328831+0000","last_peered":"2024-04-20T01:49:59.328831+0000","last_clean":"2024-04-20T01:49:59.328831+0000","last_became_active":"2024-04-20T01:47:20.110260+0000","last_became_peered":"2024-04-20T01:47:20.110260+0000","last_unstale":"2024-04-20T01:49:59.328831+0000","last_undegraded":"2024-04-20T01:49:59.328831+0000","last_fullsized":"2024-04-20T01:49:59.328831+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:49:40.403196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:49:40.403196+0000","last_clean_scrub_stamp":"2024-04-20T01:49:40.403196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:58.620548+0000","scrub_duration":0.0036137309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11],"acting":[8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":152,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-04-20T01:49:59.333948+0000","last_change":"2024-04-20T01:49:45.364543+0000","last_active":"2024-04-20T01:49:59.333948+0000","last_peered":"2024-04-20T01:49:59.333948+0000","last_clean":"2024-04-20T01:49:59.333948+0000","last_became_active":"2024-04-20T01:47:20.036328+0000","last_became_peered":"2024-04-20T01:47:20.036328+0000","last_unstale":"2024-04-20T01:49:59.333948+0000","last_undegraded":"2024-04-20T01:49:59.333948+0000","last_fullsized":"2024-04-20T01:49:59.333948+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T01:49:45.364416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T01:49:45.364416+0000","last_clean_scrub_stamp":"2024-04-20T01:49:45.364416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-20T01:50:53.686381+0000","scrub_duration":0.0019838540000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,13],"acting":[6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":11}],"osd_stats":[{"osd":13,"up_from":22,"seq":94489280550,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33852,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337988,"statfs":{"total":96636764160,"available":96602099712,"internally_reserved":0,"allocated":2801664,"data_stored":1851408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[2,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.45100000000000001}]},{"osd":2,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.59799999999999998,"15min":0.61099999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.74299999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.64000000000000001,"15min":0.65100000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84699999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.41099999999999998}]},{"osd":3,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.47699999999999998}]},{"osd":4,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":1.0760000000000001,"15min":1.155},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.747,"5min":9.0129999999999999,"15min":9.0129999999999999},"last":0.499},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.65900000000000003,"15min":0.67500000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.747,"5min":1.022,"15min":1.022},"last":0.48599999999999999}]},{"osd":5,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.59199999999999997,"15min":0.60899999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73999999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.316},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.56899999999999995,"15min":0.58299999999999996},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72299999999999998,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.34699999999999998}]},{"osd":6,"last update":"Sat Apr 20 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.66100000000000003,"15min":0.67600000000000005},"min":{"1min":0.434,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77700000000000002,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.64700000000000002,"15min":0.66500000000000004},"min":{"1min":0.40100000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.66400000000000003,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.44700000000000001}]},{"osd":7,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.755,"15min":0.76800000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95999999999999996,"5min":1.143,"15min":1.143},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.78100000000000003,"15min":0.80100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97999999999999998,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.628}]},{"osd":10,"last update":"Sat Apr 20 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.77300000000000002,"15min":0.79200000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86199999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.878,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.51500000000000001}]},{"osd":11,"last update":"Sat Apr 20 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.73699999999999999,"15min":0.752},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91900000000000004,"5min":1.109,"15min":1.109},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.84799999999999998,"15min":0.86799999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.92700000000000005,"5min":1.583,"15min":1.583},"last":0.61699999999999999}]},{"osd":12,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.77800000000000002,"15min":0.80000000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.83699999999999997,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.746,"5min":1.196,"15min":1.2709999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93999999999999995,"5min":9.1389999999999993,"15min":9.1389999999999993},"last":0.58299999999999996}]},{"osd":14,"last update":"Sat Apr 20 01:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.59999999999999998}]},{"osd":15,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.96099999999999997,"15min":0.98999999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.99399999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.88300000000000001,"15min":0.90600000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.94699999999999995,"5min":1.399,"15min":1.399},"last":0.66000000000000003}]}]},{"osd":15,"up_from":21,"seq":90194313254,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29764,"kb_used_data":2740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342076,"statfs":{"total":96636764160,"available":96606285824,"internally_reserved":0,"allocated":2805760,"data_stored":1852011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,3,4,5,6,7,8,9,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.34100000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.496,"5min":0.67300000000000004,"15min":0.70299999999999996},"min":{"1min":0.36499999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72399999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":0.54900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":2,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.68000000000000005}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42399999999999999,"15min":0.42599999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.76100000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.32600000000000001}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.38800000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.443,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.75900000000000001}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84299999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.57099999999999995}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.627,"15min":0.63400000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86499999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63,"15min":0.63300000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.94399999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.53400000000000003}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58599999999999997,"15min":0.59399999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.93000000000000005,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94899999999999995,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.44600000000000001}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45900000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96099999999999997,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.86199999999999999,"15min":0.90000000000000002},"min":{"1min":0.43099999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97599999999999998,"5min":4.5979999999999999,"15min":4.5979999999999999},"last":0.79000000000000004}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68400000000000005,"15min":0.68700000000000006},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.93899999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.748,"15min":0.746},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.016,"5min":1.272,"15min":1.272},"last":0.628}]},{"osd":13,"last update":"Sat Apr 20 01:49:41 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.749}]},{"osd":14,"last update":"Sat Apr 20 01:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.81100000000000005}]}]},{"osd":14,"up_from":102,"seq":438086664208,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31448,"kb_used_data":2184,"kb_used_omap":15,"kb_used_meta":29232,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":2236416,"data_stored":1303304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15377,"internal_metadata":29934575},"hb_peers":[0,1,2,3,4,5,9,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":9.4670000000000005,"5min":9.4670000000000005,"15min":9.4670000000000005},"last":0.53400000000000003}]},{"osd":1,"last update":"Sat Apr 20 01:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.379}]},{"osd":2,"last update":"Sat Apr 20 01:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.35699999999999998}]},{"osd":3,"last update":"Sat Apr 20 01:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.47299999999999998}]},{"osd":4,"last update":"Sat Apr 20 01:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.65800000000000003}]},{"osd":5,"last update":"Sat Apr 20 01:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.50700000000000001}]},{"osd":9,"last update":"Sat Apr 20 01:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.67500000000000004}]},{"osd":12,"last update":"Sat Apr 20 01:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.61599999999999999}]},{"osd":13,"last update":"Sat Apr 20 01:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.54700000000000004}]},{"osd":15,"last update":"Sat Apr 20 01:49:58 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.72299999999999998}]}]},{"osd":12,"up_from":21,"seq":90194313254,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29240,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342600,"statfs":{"total":96636764160,"available":96606822400,"internally_reserved":0,"allocated":2273280,"data_stored":1323083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:39 2024","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":18.143000000000001,"5min":18.143000000000001,"15min":18.143000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":18.170000000000002,"5min":18.170000000000002,"15min":18.170000000000002},"last":0.65200000000000002}]},{"osd":1,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.58499999999999996,"15min":0.55100000000000005},"min":{"1min":0.65800000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66800000000000004,"15min":0.66200000000000003},"min":{"1min":0.504,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93799999999999994,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.67600000000000005}]},{"osd":2,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52800000000000002,"15min":0.52600000000000002},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.48499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.86899999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88700000000000001,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.746},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.38100000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79800000000000004,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.78100000000000003}]},{"osd":4,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.39500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.59999999999999998,"15min":0.61199999999999999},"min":{"1min":0.371,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82399999999999995,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.56299999999999994}]},{"osd":5,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.65500000000000003,"15min":0.63700000000000001},"min":{"1min":0.55200000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0940000000000001,"5min":1.115,"15min":1.115},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.47499999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.0800000000000001,"5min":1.726,"15min":1.726},"last":0.79700000000000004}]},{"osd":6,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.66200000000000003,"15min":0.66600000000000004},"min":{"1min":0.45700000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.86699999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.74299999999999999,"15min":0.72899999999999998},"min":{"1min":0.63700000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0349999999999999,"5min":1.069,"15min":1.069},"last":0.98899999999999999}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.76300000000000001,"15min":0.76500000000000001},"min":{"1min":0.54400000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0529999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.95399999999999996,"15min":0.98999999999999999},"min":{"1min":0.53800000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.012,"5min":6.6779999999999999,"15min":6.6779999999999999},"last":0.84699999999999998}]},{"osd":13,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.79800000000000004},"min":{"1min":0.59699999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0189999999999999,"5min":1.827,"15min":1.827},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.80900000000000005,"15min":0.79500000000000004},"min":{"1min":0.753,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1739999999999999,"5min":1.258,"15min":1.258},"last":1.016}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":1.135},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.99299999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.79000000000000004},"min":{"1min":0.58199999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0109999999999999,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.82999999999999996,"15min":0.81699999999999995},"min":{"1min":0.753,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":1.0660000000000001}]}]},{"osd":11,"up_from":21,"seq":90194313253,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33340,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338500,"statfs":{"total":96636764160,"available":96602624000,"internally_reserved":0,"allocated":2273280,"data_stored":1323083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,2,3,4,8,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.58599999999999997,"15min":0.59799999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.83899999999999997,"5min":1.669,"15min":1.669},"last":0.624},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.67300000000000004,"15min":0.68300000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0900000000000001,"5min":1.75,"15min":1.75},"last":0.83299999999999996}]},{"osd":1,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.58099999999999996}]},{"osd":2,"last update":"Sat Apr 20 01:49:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.57299999999999995}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.40300000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.28799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.78600000000000003}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.88,"15min":0.93500000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1220000000000001,"5min":8.5969999999999995,"15min":8.5969999999999995},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.67000000000000004,"15min":0.67900000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.147,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.75900000000000001}]},{"osd":8,"last update":"Sat Apr 20 01:49:37 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.60199999999999998,"15min":0.58099999999999996},"min":{"1min":0.39200000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":3.3559999999999999,"5min":3.3559999999999999,"15min":3.3559999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.68600000000000005,"15min":0.65500000000000003},"min":{"1min":0.442,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.4909999999999997,"5min":4.4909999999999997,"15min":4.4909999999999997},"last":0.873}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.57499999999999996,"15min":0.56200000000000006},"min":{"1min":0.38600000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.81699999999999995}]},{"osd":10,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47199999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.68899999999999995}]},{"osd":12,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85699999999999998,"5min":1.875,"15min":1.875},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":1.262,"15min":1.349},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1879999999999999,"5min":12.385,"15min":12.385},"last":0.93600000000000005}]},{"osd":14,"last update":"Sat Apr 20 01:49:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.85599999999999998}]},{"osd":15,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.75,"15min":0.76100000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.96999999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.84399999999999997}]}]},{"osd":10,"up_from":21,"seq":90194313255,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33336,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338504,"statfs":{"total":96636764160,"available":96602628096,"internally_reserved":0,"allocated":2273280,"data_stored":1323083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,4,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.495,"15min":0.48899999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.437},{"interface":"front","average":{"1min":0.501,"5min":0.505,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.74199999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.40300000000000002}]},{"osd":1,"last update":"Sat Apr 20 01:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":4,"last update":"Sat Apr 20 01:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.38},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.36099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":7,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63300000000000001,"15min":0.626},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.376,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.52900000000000003}]},{"osd":8,"last update":"Sat Apr 20 01:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.032,"5min":1.915,"15min":1.915},"last":0.498},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68000000000000005,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1299999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.48099999999999998}]},{"osd":9,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68999999999999995,"15min":0.68500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.67500000000000004}]},{"osd":11,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.42499999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82099999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.90200000000000002,"15min":0.92600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.292,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":0.69299999999999995}]},{"osd":12,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":1.208,"15min":1.29},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1120000000000001,"5min":9.7509999999999994,"15min":9.7509999999999994},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.997,"15min":1.054},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91000000000000003,"5min":9.7759999999999998,"15min":9.7759999999999998},"last":0.55500000000000005}]},{"osd":13,"last update":"Sat Apr 20 01:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89600000000000002,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.71699999999999997,"15min":0.72299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95799999999999996,"5min":1.05,"15min":1.05},"last":0.61399999999999999}]},{"osd":14,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.64200000000000002}]},{"osd":15,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.80100000000000005,"15min":0.79700000000000004},"min":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.53400000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.93700000000000006,"5min":1.157,"15min":1.157},"last":0.626}]}]},{"osd":9,"up_from":21,"seq":90194313255,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":29240,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342600,"statfs":{"total":96636764160,"available":96606822400,"internally_reserved":0,"allocated":2273280,"data_stored":1323083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,2,4,6,7,8,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47099999999999997,"15min":0.46300000000000002},"min":{"1min":0.32800000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.52500000000000002,"15min":0.51300000000000001},"min":{"1min":0.40300000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83599999999999997,"5min":0.998,"15min":0.998},"last":0.67300000000000004}]},{"osd":2,"last update":"Sat Apr 20 01:49:39 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.50800000000000001}]},{"osd":4,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.65000000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.55400000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.872,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.52300000000000002}]},{"osd":7,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.625}]},{"osd":8,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.39600000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.54300000000000004}]},{"osd":10,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.93799999999999994,"5min":1.03,"15min":1.03},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58699999999999997,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76300000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56200000000000006}]},{"osd":11,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88400000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.72899999999999998}]},{"osd":12,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.74099999999999999,"15min":0.73899999999999999},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97899999999999998,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.89900000000000002,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.71199999999999997}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.56000000000000005}]},{"osd":15,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.745,"15min":0.753},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69699999999999995,"15min":0.70499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88800000000000001,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68500000000000005}]}]},{"osd":8,"up_from":21,"seq":90194313254,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33336,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338504,"statfs":{"total":96636764160,"available":96602628096,"internally_reserved":0,"allocated":2273280,"data_stored":1323083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,3,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":1,"last update":"Sat Apr 20 01:49:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.34200000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89700000000000002,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.628,"15min":0.64100000000000001},"min":{"1min":0.39500000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.81699999999999995,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.49199999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.45400000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85799999999999998,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.73599999999999999,"15min":0.753},"min":{"1min":0.44500000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91600000000000004,"5min":1.821,"15min":1.821},"last":0.68200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":6,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70199999999999996,"15min":0.70999999999999996},"min":{"1min":0.46500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89900000000000002,"5min":1.784,"15min":1.784},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.72199999999999998,"15min":0.746},"min":{"1min":0.44,"5min":0.437,"15min":0.437},"max":{"1min":0.79300000000000004,"5min":1.885,"15min":1.885},"last":0.47499999999999998}]},{"osd":7,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.72599999999999998,"15min":0.73499999999999999},"min":{"1min":0.50800000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93300000000000005,"5min":1.411,"15min":1.411},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.81999999999999995,"15min":0.84399999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95199999999999996,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.55000000000000004}]},{"osd":9,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.72199999999999998,"15min":0.73599999999999999},"min":{"1min":0.436,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91800000000000004,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.139,"5min":1.375,"15min":1.375},"last":0.77000000000000002}]},{"osd":10,"last update":"Sat Apr 20 01:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.72999999999999998}]},{"osd":11,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.83899999999999997,"15min":0.84299999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.141,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.92400000000000004,"15min":0.93999999999999995},"min":{"1min":0.66100000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.071,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.82099999999999995}]},{"osd":12,"last update":"Sat Apr 20 01:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.434},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.41799999999999998}]},{"osd":13,"last update":"Sat Apr 20 01:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.83499999999999996,"15min":0.82499999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.873}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":15,"last update":"Sat Apr 20 01:49:15 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.88700000000000001,"15min":0.89100000000000001},"min":{"1min":0.621,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.095,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.95399999999999996,"15min":0.98399999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0229999999999999,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":0.71499999999999997}]}]},{"osd":7,"up_from":21,"seq":90194313255,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29240,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342600,"statfs":{"total":96636764160,"available":96606822400,"internally_reserved":0,"allocated":2273280,"data_stored":1323083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,3,4,6,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:48:33 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.38800000000000001}]},{"osd":1,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.5},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.54300000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.53300000000000003}]},{"osd":3,"last update":"Sat Apr 20 01:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.435,"15min":0.438},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.58399999999999996,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.441,"5min":0.438,"15min":0.438},"min":{"1min":0.307,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.63900000000000001}]},{"osd":4,"last update":"Sat Apr 20 01:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.73899999999999999,"15min":0.751},"min":{"1min":0.379,"5min":0.372,"15min":0.372},"max":{"1min":0.92600000000000005,"5min":4.1630000000000003,"15min":4.1630000000000003},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.47199999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94399999999999995,"5min":4.1920000000000002,"15min":4.1920000000000002},"last":0.84199999999999997}]},{"osd":6,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.78700000000000003}]},{"osd":8,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.78900000000000003,"15min":0.81499999999999995},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82599999999999996,"5min":5.7679999999999998,"15min":5.7679999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":1.0189999999999999,"15min":1.077},"min":{"1min":0.46300000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96599999999999997,"5min":9.952,"15min":9.952},"last":0.82599999999999996}]},{"osd":9,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61299999999999999,"15min":0.61499999999999999},"min":{"1min":0.438,"5min":0.433,"15min":0.433},"max":{"1min":0.751,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.498,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88800000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.621}]},{"osd":10,"last update":"Sat Apr 20 01:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.81499999999999995}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82899999999999996,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.58699999999999997},"min":{"1min":0.41099999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.88800000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.72199999999999998}]},{"osd":12,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.84399999999999997,"15min":0.86099999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.99199999999999999,"5min":4.2720000000000002,"15min":4.2720000000000002},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68700000000000006,"15min":0.68000000000000005},"min":{"1min":0.50900000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.90300000000000002,"5min":0.995,"15min":0.995},"last":0.85299999999999998}]},{"osd":14,"last update":"Sat Apr 20 01:49:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.875}]},{"osd":15,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":6,"up_from":21,"seq":90194313253,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33336,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338504,"statfs":{"total":96636764160,"available":96602628096,"internally_reserved":0,"allocated":2273280,"data_stored":1323083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[3,4,5,7,8,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.5}]},{"osd":2,"last update":"Sat Apr 20 01:48:32 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.53700000000000003}]},{"osd":3,"last update":"Sat Apr 20 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.42199999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76900000000000002,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.59699999999999998}]},{"osd":4,"last update":"Sat Apr 20 01:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.72099999999999997,"15min":0.68400000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":5.141,"5min":5.141,"15min":5.141},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.67400000000000004,"15min":0.622},"min":{"1min":0.38500000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":8.0090000000000003,"5min":8.0090000000000003,"15min":8.0090000000000003},"last":0.49399999999999999}]},{"osd":5,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.38300000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.91700000000000004,"5min":1.083,"15min":1.083},"last":0.57499999999999996}]},{"osd":7,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59999999999999998,"15min":0.60399999999999998},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77600000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66100000000000003,"15min":0.66400000000000003},"min":{"1min":0.36499999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.89400000000000002,"5min":1.056,"15min":1.056},"last":0.70599999999999996}]},{"osd":8,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.755,"15min":0.77000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88300000000000001,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.80100000000000005,"15min":0.81200000000000006},"min":{"1min":0.53800000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92000000000000004,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.54000000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":10,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.67900000000000005,"15min":0.68700000000000006},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92500000000000004,"5min":1.129,"15min":1.129},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.67700000000000005,"15min":0.69099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88,"5min":1.272,"15min":1.272},"last":0.52400000000000002}]},{"osd":11,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.92100000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.78100000000000003,"15min":0.79000000000000004},"min":{"1min":0.52200000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96599999999999997,"5min":1.167,"15min":1.167},"last":0.80800000000000005}]},{"osd":12,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.83499999999999996,"15min":0.84799999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.085,"5min":1.444,"15min":1.444},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84899999999999998,"15min":0.84699999999999998},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.58699999999999997}]},{"osd":13,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.81699999999999995,"15min":0.82299999999999995},"min":{"1min":0.56899999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0329999999999999,"5min":1.482,"15min":1.482},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.753,"15min":0.76000000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.94999999999999996,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.68000000000000005}]},{"osd":14,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.65700000000000003}]},{"osd":15,"last update":"Sat Apr 20 01:49:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.85999999999999999,"15min":0.87},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0049999999999999,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77900000000000003,"15min":0.79200000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97599999999999998,"5min":1.327,"15min":1.327},"last":0.72199999999999998}]}]},{"osd":1,"up_from":21,"seq":90194313254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33336,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338504,"statfs":{"total":96636764160,"available":96602628096,"internally_reserved":0,"allocated":2273280,"data_stored":1323083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,4,5,6,7,8,9,10,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.75600000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.38},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67400000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.42299999999999999}]},{"osd":2,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70199999999999996,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79600000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.55300000000000005}]},{"osd":3,"last update":"Sat Apr 20 01:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.76000000000000001}]},{"osd":4,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67800000000000005,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72099999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.40200000000000002}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.497,"15min":0.5},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.499,"15min":0.50700000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.67700000000000005}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61899999999999999,"15min":0.623},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82599999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.60199999999999998}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66800000000000004,"15min":0.66300000000000003},"min":{"1min":0.48099999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65900000000000003,"15min":0.64900000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.77700000000000002}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.68999999999999995,"15min":0.69799999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.88300000000000001,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.72199999999999998,"15min":0.73999999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84899999999999998,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.73599999999999999}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.83499999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67800000000000005,"15min":0.68400000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92900000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.81100000000000005}]},{"osd":10,"last update":"Sat Apr 20 01:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.64600000000000002}]},{"osd":14,"last update":"Sat Apr 20 01:49:55 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.72099999999999997}]}]},{"osd":0,"up_from":21,"seq":90194313255,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33332,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338508,"statfs":{"total":96636764160,"available":96602632192,"internally_reserved":0,"allocated":2269184,"data_stored":1322497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.64500000000000002,"15min":0.63400000000000001},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.89800000000000002}]},{"osd":2,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.90200000000000002,"15min":0.95599999999999996},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.89100000000000001,"5min":8.0619999999999994,"15min":8.0619999999999994},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.1399999999999999,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.53600000000000003}]},{"osd":3,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.68300000000000005,"15min":0.69099999999999995},"min":{"1min":0.37,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.80500000000000005,"5min":3.363,"15min":3.363},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.90000000000000002,"15min":0.93400000000000005},"min":{"1min":0.50800000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.998,"5min":7.9710000000000001,"15min":7.9710000000000001},"last":0.67400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66000000000000003,"15min":0.66500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.97599999999999998,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90200000000000002,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":0.75700000000000001}]},{"osd":6,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.94199999999999995,"15min":0.98699999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1779999999999999,"5min":8.0239999999999991,"15min":8.0239999999999991},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1739999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.52400000000000002}]},{"osd":7,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.72999999999999998,"15min":0.748},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86299999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.98399999999999999,"15min":1.0229999999999999},"min":{"1min":0.42899999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1200000000000001,"5min":7.8630000000000004,"15min":7.8630000000000004},"last":0.57799999999999996}]},{"osd":8,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.67800000000000005,"15min":0.69499999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.92100000000000004,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.95799999999999996,"15min":1.02},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.93400000000000005,"5min":8.1189999999999998,"15min":8.1189999999999998},"last":0.39200000000000002}]},{"osd":9,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.79800000000000004,"15min":0.81200000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.01,"5min":3.5699999999999998,"15min":3.5699999999999998},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.55300000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.044,"5min":1.927,"15min":1.927},"last":0.59599999999999997}]},{"osd":13,"last update":"Sat Apr 20 01:49:37 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.3300000000000001,"5min":0.91800000000000004,"15min":0.84899999999999998},"min":{"1min":0.64200000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":9.4339999999999993,"5min":9.4339999999999993,"15min":9.4339999999999993},"last":0.95299999999999996}]},{"osd":15,"last update":"Sat Apr 20 01:49:19 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.097,"5min":1.833,"15min":1.833},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.86899999999999999,"15min":0.88500000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96699999999999997,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":0.88600000000000001}]}]},{"osd":2,"up_from":21,"seq":90194313253,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29244,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342596,"statfs":{"total":96636764160,"available":96606818304,"internally_reserved":0,"allocated":2273280,"data_stored":1323083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.46600000000000003,"15min":0.45600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.47099999999999997,"15min":0.46300000000000002},"min":{"1min":0.311,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.56799999999999995}]},{"osd":1,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55500000000000005,"15min":0.54500000000000004},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54600000000000004,"15min":0.53900000000000003},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.46100000000000002}]},{"osd":3,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82199999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":7,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.60099999999999998,"15min":0.60299999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82799999999999996,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":0.60299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.72299999999999998,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.67100000000000004}]},{"osd":8,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65000000000000002,"15min":0.66100000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90100000000000002,"5min":2.512,"15min":2.512},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.68400000000000005,"15min":0.69599999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.77600000000000002,"5min":2.532,"15min":2.532},"last":0.69699999999999995}]},{"osd":9,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.435,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64700000000000002,"15min":0.64800000000000002},"min":{"1min":0.43099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93000000000000005,"5min":1.107,"15min":1.107},"last":0.60099999999999998}]},{"osd":11,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0269999999999999,"5min":1.135,"15min":1.135},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68600000000000005,"15min":0.69399999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88900000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.74299999999999999}]},{"osd":12,"last update":"Sat Apr 20 01:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.75800000000000001,"15min":0.78500000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.872,"5min":3.1190000000000002,"15min":3.1190000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.80200000000000005,"15min":0.82899999999999996},"min":{"1min":0.42499999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93899999999999995,"5min":3.492,"15min":3.492},"last":0.66200000000000003}]},{"osd":13,"last update":"Sat Apr 20 01:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.67800000000000005,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.68500000000000005}]},{"osd":15,"last update":"Sat Apr 20 01:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.66600000000000004,"15min":0.66000000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68000000000000005,"15min":0.67500000000000004},"min":{"1min":0.53600000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.78800000000000003}]}]},{"osd":3,"up_from":21,"seq":90194313256,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33336,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338504,"statfs":{"total":96636764160,"available":96602628096,"internally_reserved":0,"allocated":2273280,"data_stored":1323083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,4,5,6,7,8,9,10,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Apr 20 01:49:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.48199999999999998,"15min":0.47699999999999998},"min":{"1min":0.312,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.48499999999999999,"15min":0.47899999999999998},"min":{"1min":0.33800000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.58899999999999997}]},{"osd":2,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.36199999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.90500000000000003,"5min":0.996,"15min":0.996},"last":0.63},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58999999999999997,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.85599999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.68000000000000005}]},{"osd":4,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.56499999999999995,"15min":0.56899999999999995},"min":{"1min":0.34000000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69399999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.94599999999999995,"15min":1.014},"min":{"1min":0.39000000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83899999999999997,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":0.54200000000000004}]},{"osd":5,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.56299999999999994,"15min":0.57199999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66500000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.55600000000000005,"15min":0.56000000000000005},"min":{"1min":0.36399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78700000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.79400000000000004}]},{"osd":6,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.38600000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.89300000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.76700000000000002}]},{"osd":7,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66200000000000003,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.54800000000000004,"15min":0.56000000000000005},"min":{"1min":0.30099999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71299999999999997,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.66900000000000004}]},{"osd":8,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.41199999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85099999999999998,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":1.0780000000000001,"15min":1.1479999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81999999999999995,"5min":9.4700000000000006,"15min":9.4700000000000006},"last":0.71499999999999997}]},{"osd":9,"last update":"Sat Apr 20 01:49:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.622,"15min":0.625},"min":{"1min":0.45200000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84699999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.71199999999999997,"15min":0.72599999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74399999999999999,"5min":1.306,"15min":1.306},"last":0.88400000000000001}]},{"osd":10,"last update":"Sat Apr 20 01:49:37 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.80300000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":12,"last update":"Sat Apr 20 01:48:16 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":10.959,"5min":10.959,"15min":10.959},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.49399999999999999}]},{"osd":15,"last update":"Sat Apr 20 01:49:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.753},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.74199999999999999}]}]},{"osd":4,"up_from":21,"seq":90194313255,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33336,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338504,"statfs":{"total":96636764160,"available":96602628096,"internally_reserved":0,"allocated":2273280,"data_stored":1323083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,5,6,7,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.81000000000000005,"15min":0.87},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.93400000000000005,"5min":5.0679999999999996,"15min":5.0679999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":1.1579999999999999,"15min":1.2729999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.008,"5min":16.27,"15min":16.27},"last":0.48799999999999999}]},{"osd":1,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":1.012,"15min":1.081},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.92700000000000005,"5min":10.507999999999999,"15min":10.507999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":1.1559999999999999,"15min":1.23},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6319999999999999,"5min":11.019,"15min":11.019},"last":0.73699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":3,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.997,"15min":1.0469999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4750000000000001,"5min":4.9180000000000001,"15min":4.9180000000000001},"last":0.75},{"interface":"front","average":{"1min":0.74099999999999999,"5min":1.0449999999999999,"15min":1.095},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.6000000000000001,"5min":10.619,"15min":10.619},"last":0.66800000000000004}]},{"osd":5,"last update":"Sat Apr 20 01:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":1.099,"15min":1.169},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.673,"5min":10.895,"15min":10.895},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":1.012,"15min":1.0740000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1910000000000001,"5min":4.9589999999999996,"15min":4.9589999999999996},"last":0.68400000000000005}]},{"osd":6,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":1.1890000000000001,"15min":1.2649999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1719999999999999,"5min":11.051,"15min":11.051},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":1.1279999999999999,"15min":1.2},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96199999999999997,"5min":10.589,"15min":10.589},"last":0.79100000000000004}]},{"osd":7,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":1.1950000000000001,"15min":1.2649999999999999},"min":{"1min":0.56000000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.571,"5min":10.957000000000001,"15min":10.957000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":1.1799999999999999,"15min":1.244},"min":{"1min":0.47299999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3080000000000001,"5min":10.656000000000001,"15min":10.656000000000001},"last":0.71999999999999997}]},{"osd":8,"last update":"Sat Apr 20 01:48:35 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.79,"5min":3.79,"15min":3.79},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.0269999999999999}]},{"osd":9,"last update":"Sat Apr 20 01:48:19 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.385,"15min":1.385},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":10.929,"5min":10.929,"15min":10.929},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":10.555999999999999,"5min":10.555999999999999,"15min":10.555999999999999},"last":1.077}]},{"osd":10,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":1.149,"15min":1.204},"min":{"1min":0.57799999999999996,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.331,"5min":5.0229999999999997,"15min":5.0229999999999997},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":1.2729999999999999,"15min":1.351},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2330000000000001,"5min":10.69,"15min":10.69},"last":0.874}]},{"osd":11,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.3600000000000001,"15min":1.4410000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.448,"5min":10.987,"15min":10.987},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.84799999999999998,"5min":1.1879999999999999,"15min":1.244},"min":{"1min":0.63700000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.2290000000000001,"5min":4.9900000000000002,"15min":4.9900000000000002},"last":0.93600000000000005}]},{"osd":12,"last update":"Sat Apr 20 01:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.3240000000000001,"15min":1.4059999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.181,"5min":10.720000000000001,"15min":10.720000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.85999999999999999,"5min":1.589,"15min":1.7110000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3149999999999999,"5min":16.617000000000001,"15min":16.617000000000001},"last":0.70399999999999996}]},{"osd":13,"last update":"Sat Apr 20 01:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.98499999999999999,"15min":1.0089999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.732,"5min":3.8999999999999999,"15min":3.8999999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.0469999999999999,"15min":1.075},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.546,"5min":4.5019999999999998,"15min":4.5019999999999998},"last":0.89700000000000002}]},{"osd":14,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]},{"osd":5,"up_from":21,"seq":90194313254,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29244,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342596,"statfs":{"total":96636764160,"available":96606818304,"internally_reserved":0,"allocated":2273280,"data_stored":1323083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,8,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.85699999999999998,"15min":0.92100000000000004},"min":{"1min":0.30099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77600000000000002,"5min":10.423999999999999,"15min":10.423999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.92500000000000004,"15min":0.98399999999999999},"min":{"1min":0.374,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73499999999999999,"5min":10.465,"15min":10.465},"last":0.62}]},{"osd":2,"last update":"Sat Apr 20 01:49:22 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.33600000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.74099999999999999}]},{"osd":3,"last update":"Sat Apr 20 01:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.46999999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.443,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.69099999999999995}]},{"osd":4,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.81999999999999995,"15min":0.86799999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71499999999999997,"5min":8.5269999999999992,"15min":8.5269999999999992},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60999999999999999,"15min":0.61699999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78100000000000003,"5min":2.79,"15min":2.79},"last":0.64200000000000002}]},{"osd":6,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84599999999999997,"5min":1.115,"15min":1.115},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.63100000000000001},"min":{"1min":0.38900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94599999999999995,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.85299999999999998}]},{"osd":7,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78800000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.49399999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.88500000000000001}]},{"osd":8,"last update":"Sat Apr 20 01:49:35 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.77400000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":10,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.91300000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45600000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.80800000000000005}]},{"osd":11,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71899999999999997,"15min":0.71199999999999997},"min":{"1min":0.53900000000000003,"5min":0.435,"15min":0.435},"max":{"1min":1.242,"5min":1.956,"15min":1.956},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70299999999999996,"15min":0.69199999999999995},"min":{"1min":0.55000000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.86299999999999999}]},{"osd":12,"last update":"Sat Apr 20 01:49:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":1.052,"15min":1.1080000000000001},"min":{"1min":0.55800000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90600000000000003,"5min":10.616,"15min":10.616},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.755,"5min":1.0900000000000001,"15min":1.1459999999999999},"min":{"1min":0.56999999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.98899999999999999,"5min":10.629,"15min":10.629},"last":0.874}]},{"osd":14,"last update":"Sat Apr 20 01:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.84699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T01:50:08.454 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-20T01:50:08.837 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-20T01:50:08.837 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-20T01:50:08.837 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-20T01:50:08.838 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:08.890 INFO:tasks.ceph.osd.0:Stopped 2024-04-20T01:50:08.891 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-20T01:50:08.891 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:08.935 INFO:tasks.ceph.osd.4:Stopped 2024-04-20T01:50:08.935 DEBUG:tasks.ceph.osd.8:waiting for process to exit 2024-04-20T01:50:08.936 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:08.977 INFO:tasks.ceph.osd.8:Stopped 2024-04-20T01:50:08.977 DEBUG:tasks.ceph.osd.12:waiting for process to exit 2024-04-20T01:50:08.977 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:09.011 INFO:tasks.ceph.osd.12:Stopped 2024-04-20T01:50:09.011 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-20T01:50:09.011 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:09.053 INFO:tasks.ceph.osd.1:Stopped 2024-04-20T01:50:09.053 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-20T01:50:09.053 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:09.088 INFO:tasks.ceph.osd.5:Stopped 2024-04-20T01:50:09.088 DEBUG:tasks.ceph.osd.9:waiting for process to exit 2024-04-20T01:50:09.088 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:09.162 INFO:tasks.ceph.osd.9:Stopped 2024-04-20T01:50:09.162 DEBUG:tasks.ceph.osd.13:waiting for process to exit 2024-04-20T01:50:09.162 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:09.234 INFO:tasks.ceph.osd.13:Stopped 2024-04-20T01:50:09.234 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-20T01:50:09.234 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:09.274 INFO:tasks.ceph.osd.2:Stopped 2024-04-20T01:50:09.275 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-20T01:50:09.275 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:09.339 INFO:tasks.ceph.osd.6:Stopped 2024-04-20T01:50:09.340 DEBUG:tasks.ceph.osd.10:waiting for process to exit 2024-04-20T01:50:09.340 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:09.416 INFO:tasks.ceph.osd.10:Stopped 2024-04-20T01:50:09.417 DEBUG:tasks.ceph.osd.14:waiting for process to exit 2024-04-20T01:50:09.417 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:09.484 INFO:tasks.ceph.osd.14:Stopped 2024-04-20T01:50:09.484 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-20T01:50:09.484 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:09.516 INFO:tasks.ceph.osd.3:Stopped 2024-04-20T01:50:09.516 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-20T01:50:09.516 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:09.586 INFO:tasks.ceph.osd.7:Stopped 2024-04-20T01:50:09.586 DEBUG:tasks.ceph.osd.11:waiting for process to exit 2024-04-20T01:50:09.586 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:09.663 INFO:tasks.ceph.osd.11:Stopped 2024-04-20T01:50:09.663 DEBUG:tasks.ceph.osd.15:waiting for process to exit 2024-04-20T01:50:09.663 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:09.740 INFO:tasks.ceph.osd.15:Stopped 2024-04-20T01:50:09.741 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-20T01:50:09.741 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-20T01:50:09.741 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:09.768 INFO:tasks.ceph.mgr.y:Stopped 2024-04-20T01:50:09.768 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-20T01:50:09.768 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:09.810 INFO:tasks.ceph.mgr.x:Stopped 2024-04-20T01:50:09.810 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-20T01:50:09.810 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-20T01:50:09.811 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:09.824 INFO:tasks.ceph.mon.a:Stopped 2024-04-20T01:50:09.825 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-20T01:50:09.825 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:09.838 INFO:tasks.ceph.mon.b:Stopped 2024-04-20T01:50:09.838 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-20T01:50:09.838 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T01:50:09.851 INFO:tasks.ceph.mon.c:Stopped 2024-04-20T01:50:09.851 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-20T01:50:09.851 DEBUG:teuthology.orchestra.run.smithi044:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | head -n 1 2024-04-20T01:50:09.906 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-20T01:50:09.907 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-20T01:50:10.187 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-20T01:50:10.187 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-20T01:50:10.283 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-8 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-20T01:50:10.283 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-8 2024-04-20T01:50:10.352 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-12 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-20T01:50:10.352 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-12 2024-04-20T01:50:10.427 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-20T01:50:10.427 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-20T01:50:10.626 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-20T01:50:10.626 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-20T01:50:10.703 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-9 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-20T01:50:10.703 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-9 2024-04-20T01:50:10.786 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-13 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-20T01:50:10.786 DEBUG:teuthology.orchestra.run.smithi121:> sync && sudo umount -f /var/lib/ceph/osd/ceph-13 2024-04-20T01:50:10.849 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-20T01:50:10.849 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-20T01:50:11.045 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-20T01:50:11.045 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-20T01:50:11.133 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-10 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-20T01:50:11.133 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-10 2024-04-20T01:50:11.226 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-14 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-20T01:50:11.227 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-14 2024-04-20T01:50:11.334 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-20T01:50:11.334 DEBUG:teuthology.orchestra.run.smithi165:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-20T01:50:11.499 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-20T01:50:11.499 DEBUG:teuthology.orchestra.run.smithi165:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-20T01:50:11.573 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-11 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-20T01:50:11.573 DEBUG:teuthology.orchestra.run.smithi165:> sync && sudo umount -f /var/lib/ceph/osd/ceph-11 2024-04-20T01:50:11.642 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-15 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-20T01:50:11.642 DEBUG:teuthology.orchestra.run.smithi165:> sync && sudo umount -f /var/lib/ceph/osd/ceph-15 2024-04-20T01:50:11.756 INFO:tasks.ceph:Archiving mon data... 2024-04-20T01:50:11.757 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102/data/mon.a.tgz 2024-04-20T01:50:11.758 DEBUG:teuthology.orchestra.run.smithi044:> mktemp 2024-04-20T01:50:11.780 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/tmp.PNouvSnewX 2024-04-20T01:50:11.780 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.PNouvSnewX 2024-04-20T01:50:11.973 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0666 /tmp/tmp.PNouvSnewX 2024-04-20T01:50:12.067 DEBUG:teuthology.orchestra.remote:smithi044:/tmp/tmp.PNouvSnewX is 414KB 2024-04-20T01:50:12.141 DEBUG:teuthology.orchestra.run.smithi044:> rm -fr /tmp/tmp.PNouvSnewX 2024-04-20T01:50:12.162 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102/data/mon.b.tgz 2024-04-20T01:50:12.163 DEBUG:teuthology.orchestra.run.smithi121:> mktemp 2024-04-20T01:50:12.183 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.rxgqLS5T0i 2024-04-20T01:50:12.184 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.rxgqLS5T0i 2024-04-20T01:50:12.385 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0666 /tmp/tmp.rxgqLS5T0i 2024-04-20T01:50:12.472 DEBUG:teuthology.orchestra.remote:smithi121:/tmp/tmp.rxgqLS5T0i is 592KB 2024-04-20T01:50:12.571 DEBUG:teuthology.orchestra.run.smithi121:> rm -fr /tmp/tmp.rxgqLS5T0i 2024-04-20T01:50:12.587 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102/data/mon.c.tgz 2024-04-20T01:50:12.587 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2024-04-20T01:50:12.609 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.cxMud9xhQT 2024-04-20T01:50:12.610 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.cxMud9xhQT 2024-04-20T01:50:12.817 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.cxMud9xhQT 2024-04-20T01:50:12.912 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.cxMud9xhQT is 592KB 2024-04-20T01:50:13.010 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.cxMud9xhQT 2024-04-20T01:50:13.027 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-20T01:50:13.027 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-20T01:50:13.030 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-20T01:50:13.032 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-20T01:50:13.069 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-20T01:50:13.101 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-04-20T01:50:13.101 DEBUG:teuthology.orchestra.run.smithi044:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-04-20T01:50:13.104 DEBUG:teuthology.orchestra.run.smithi121:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-04-20T01:50:13.106 DEBUG:teuthology.orchestra.run.smithi123:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-04-20T01:50:13.132 INFO:teuthology.orchestra.run.smithi121.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-04-20T01:50:13.136 DEBUG:teuthology.orchestra.run.smithi165:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-04-20T01:50:13.137 INFO:teuthology.orchestra.run.smithi044.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-04-20T01:50:13.160 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-04-20T01:50:13.173 INFO:teuthology.orchestra.run.smithi165.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-04-20T01:50:13.180 INFO:tasks.ceph:Archiving crash dumps... 2024-04-20T01:50:13.181 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102/remote/smithi044/crash 2024-04-20T01:50:13.182 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-20T01:50:13.219 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102/remote/smithi121/crash 2024-04-20T01:50:13.221 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-20T01:50:13.249 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102/remote/smithi123/crash 2024-04-20T01:50:13.250 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-20T01:50:13.279 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102/remote/smithi165/crash 2024-04-20T01:50:13.280 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-20T01:50:13.322 INFO:tasks.ceph:Compressing logs... 2024-04-20T01:50:13.322 DEBUG:teuthology.orchestra.run.smithi044:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T01:50:13.325 DEBUG:teuthology.orchestra.run.smithi121:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T01:50:13.327 DEBUG:teuthology.orchestra.run.smithi123:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T01:50:13.330 DEBUG:teuthology.orchestra.run.smithi165:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T01:50:16.259 INFO:tasks.ceph:Archiving logs... 2024-04-20T01:50:16.259 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102/remote/smithi044/log 2024-04-20T01:50:16.261 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T01:50:16.849 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102/remote/smithi121/log 2024-04-20T01:50:16.850 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T01:50:17.183 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102/remote/smithi123/log 2024-04-20T01:50:17.184 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T01:50:17.574 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102/remote/smithi165/log 2024-04-20T01:50:17.575 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T01:50:17.942 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-20T01:50:17.954 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-20T01:50:17.954 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T01:50:17.959 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T01:50:17.962 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T01:50:17.964 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T01:50:18.088 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-20T01:50:18.098 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-20T01:50:18.099 DEBUG:teuthology.orchestra.run.smithi044:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T01:50:18.102 DEBUG:teuthology.orchestra.run.smithi121:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T01:50:18.104 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T01:50:18.107 DEBUG:teuthology.orchestra.run.smithi165:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T01:50:18.120 INFO:teuthology.orchestra.run.smithi044.stderr:bash: ntpq: command not found 2024-04-20T01:50:18.121 INFO:teuthology.orchestra.run.smithi121.stderr:bash: ntpq: command not found 2024-04-20T01:50:18.124 INFO:teuthology.orchestra.run.smithi123.stderr:bash: ntpq: command not found 2024-04-20T01:50:18.125 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T01:50:18.126 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-04-20T01:50:18.126 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 37 -371us[ -371us] +/- 95ms 2024-04-20T01:50:18.126 INFO:teuthology.orchestra.run.smithi044.stdout:^* hv02.front.sepia.ceph.com 3 6 377 37 -1296us[-1285us] +/- 54ms 2024-04-20T01:50:18.126 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +2344us[+2344us] +/- 83ms 2024-04-20T01:50:18.126 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T01:50:18.127 INFO:teuthology.orchestra.run.smithi165.stderr:bash: ntpq: command not found 2024-04-20T01:50:18.129 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T01:50:18.129 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2024-04-20T01:50:18.130 INFO:teuthology.orchestra.run.smithi121.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 36 -367us[ -367us] +/- 95ms 2024-04-20T01:50:18.130 INFO:teuthology.orchestra.run.smithi121.stdout:^* hv02.front.sepia.ceph.com 3 6 377 36 -1287us[-1274us] +/- 54ms 2024-04-20T01:50:18.130 INFO:teuthology.orchestra.run.smithi121.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 +2349us[+2362us] +/- 83ms 2024-04-20T01:50:18.130 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T01:50:18.132 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T01:50:18.132 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-04-20T01:50:18.132 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 37 -364us[ -353us] +/- 95ms 2024-04-20T01:50:18.132 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv02.front.sepia.ceph.com 3 6 377 37 -1299us[-1288us] +/- 54ms 2024-04-20T01:50:18.132 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +2335us[+2335us] +/- 84ms 2024-04-20T01:50:18.132 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T01:50:18.134 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T01:50:18.134 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-04-20T01:50:18.134 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 34 -362us[ -362us] +/- 95ms 2024-04-20T01:50:18.134 INFO:teuthology.orchestra.run.smithi165.stdout:^* hv02.front.sepia.ceph.com 3 6 377 35 -1289us[-1273us] +/- 54ms 2024-04-20T01:50:18.134 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 +2347us[+2363us] +/- 83ms 2024-04-20T01:50:18.134 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T01:50:18.135 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-20T01:50:18.146 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-20T01:50:18.147 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-20T01:50:18.162 DEBUG:teuthology.orchestra.run.smithi044:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T01:50:18.256 DEBUG:teuthology.orchestra.run.smithi121:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T01:50:18.331 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T01:50:18.405 DEBUG:teuthology.orchestra.run.smithi165:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T01:50:18.479 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-20T01:50:18.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T01:50:18.518 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-20T01:50:18.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T01:50:18.554 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-20T01:50:18.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T01:50:18.590 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-20T01:50:18.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T01:50:18.623 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-20T01:50:18.634 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-20T01:50:18.668 INFO:teuthology.task.internal:Duration was 1487.631966 seconds 2024-04-20T01:50:18.668 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-20T01:50:18.706 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-20T01:50:18.706 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T01:50:18.709 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T01:50:18.712 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T01:50:18.714 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T01:50:18.763 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T01:50:18.765 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T01:50:18.768 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T01:50:18.769 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T01:50:19.020 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-20T01:50:19.020 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi044.front.sepia.ceph.com 2024-04-20T01:50:19.020 DEBUG:teuthology.orchestra.run.smithi044:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T01:50:19.048 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi121.front.sepia.ceph.com 2024-04-20T01:50:19.048 DEBUG:teuthology.orchestra.run.smithi121:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T01:50:19.074 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2024-04-20T01:50:19.075 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T01:50:19.102 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi165.front.sepia.ceph.com 2024-04-20T01:50:19.102 DEBUG:teuthology.orchestra.run.smithi165:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T01:50:19.124 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-20T01:50:19.124 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T01:50:19.127 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T01:50:19.130 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T01:50:19.144 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T01:50:19.210 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-20T01:50:19.210 DEBUG:teuthology.orchestra.run.smithi044:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T01:50:19.213 DEBUG:teuthology.orchestra.run.smithi121:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T01:50:19.215 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T01:50:19.218 DEBUG:teuthology.orchestra.run.smithi165:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T01:50:19.312 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-20T01:50:19.322 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-20T01:50:19.322 DEBUG:teuthology.orchestra.run.smithi044:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T01:50:19.353 DEBUG:teuthology.orchestra.run.smithi121:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T01:50:19.380 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T01:50:19.414 DEBUG:teuthology.orchestra.run.smithi165:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T01:50:19.441 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-20T01:50:19.453 DEBUG:teuthology.orchestra.run.smithi044:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T01:50:19.455 DEBUG:teuthology.orchestra.run.smithi121:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T01:50:19.457 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T01:50:19.460 DEBUG:teuthology.orchestra.run.smithi165:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T01:50:19.481 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = core 2024-04-20T01:50:19.487 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = core 2024-04-20T01:50:19.490 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2024-04-20T01:50:19.507 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = core 2024-04-20T01:50:19.543 DEBUG:teuthology.orchestra.run.smithi044:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T01:50:19.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T01:50:19.573 DEBUG:teuthology.orchestra.run.smithi121:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T01:50:19.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T01:50:19.590 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T01:50:19.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T01:50:19.609 DEBUG:teuthology.orchestra.run.smithi165:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T01:50:19.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T01:50:19.625 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-20T01:50:19.636 INFO:teuthology.task.internal:Transferring archived files... 2024-04-20T01:50:19.637 DEBUG:teuthology.misc:Transferring archived files from smithi044:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102/remote/smithi044 2024-04-20T01:50:19.637 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T01:50:19.685 DEBUG:teuthology.misc:Transferring archived files from smithi121:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102/remote/smithi121 2024-04-20T01:50:19.685 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T01:50:19.723 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102/remote/smithi123 2024-04-20T01:50:19.724 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T01:50:19.769 DEBUG:teuthology.misc:Transferring archived files from smithi165:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-20_01:10:46-rados-wip-yuri7-testing-2024-04-18-1351-reef-distro-default-smithi/7664102/remote/smithi165 2024-04-20T01:50:19.769 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T01:50:19.807 INFO:teuthology.task.internal:Removing archive directory... 2024-04-20T01:50:19.808 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T01:50:19.810 DEBUG:teuthology.orchestra.run.smithi121:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T01:50:19.813 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T01:50:19.815 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T01:50:19.858 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-20T01:50:19.873 INFO:teuthology.task.internal:Not uploading archives. 2024-04-20T01:50:19.873 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-20T01:50:19.885 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-20T01:50:19.885 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T01:50:19.888 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T01:50:19.891 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T01:50:19.893 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T01:50:19.905 INFO:teuthology.orchestra.run.smithi121.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 01:50 /home/ubuntu/cephtest 2024-04-20T01:50:19.906 INFO:teuthology.orchestra.run.smithi044.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 01:50 /home/ubuntu/cephtest 2024-04-20T01:50:19.911 INFO:teuthology.orchestra.run.smithi123.stdout: 262179 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 01:50 /home/ubuntu/cephtest 2024-04-20T01:50:19.914 INFO:teuthology.orchestra.run.smithi165.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 01:50 /home/ubuntu/cephtest 2024-04-20T01:50:19.915 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-20T01:50:19.927 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-20T01:50:19.954 INFO:teuthology.run:Summary data: description: rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} duration: 1487.631965637207 flavor: default owner: scheduled_yuriw@teuthology success: true 2024-04-20T01:50:19.954 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T01:50:20.020 INFO:teuthology.run:pass